Windows Build Issues and Notes

[geographika]

Many thanks for creating this project!

I'm going through the steps on a Windows machine at the moment, and have a few notes that may be of use to someone.
@timlinux - if putting these into an issue is inappropriate, please feel free to close. I'm not sure if there could be incorporated into the README, or perhaps in an additional text file in the repository.

Some basics - when running the first command - docker-compose run planet-downloader

If Docker is not currently running on the host machine, then the following error is returned:

D:\GitHub\kartoza-world-osm-tegola-map>docker-compose run planet-downloader
Traceback (most recent call last):
  File "docker\api\client.py", line 214, in _retrieve_server_version

Also the following has to be completed before running the first command:

Copy the .env.template file to .env, then edit it, replacing the placeholders with proper values as indicated.

Otherwise, the following errors are shown:

ERROR: The Compose file '.\docker-compose.yml' is invalid because:
services.db.ports contains an invalid type, it should be a number, or an object

All looked good for 10-15 minutes for the download, running to 20/30% then the host machine ground to a halt, with CPU and Disk Usage both at 100%. It turned out on Windows Docker uses as much memory as it can get, which starts leading to swap memory and disk usage. The solution was the answer at https://stackoverflow.com/questions/62154016/docker-on-wsl2-very-slow - adding a .wslconfig file with the following settings (these may be ok for the download step but may need to be modified for the import:

[wsl2] 
memory=900MB    #Limits VM memory in WSL 2 to 900MB 
processors=1    #Makes the WSL 2 VM use one virtual processors

The download completed in around 90 minutes. However, the file didn't appear in the ./conf/osm_conf folder (even when running the command from the directory containing this subfolder).
The file was created in \\wsl$\docker-desktop-data\version-pack-data\community\docker\overlay2\858d8ccadf933b6040eb07bcc9e782f9943d6448eac31d91025e6c408394786c\diff\download

I tried updating docker-compose.yml to use a full path, but with no change:

    volumes:
      # neither of these seems to create the file so it is visible in windows..
      - ./conf/osm_conf:/downloads
      #- D:/GitHub/kartoza-world-osm-tegola-map/conf/osm_conf:/downloads

In the end, I copied the file manually to ./conf/osm_conf and renamed country.pbf as per the instructions.

The docker-compose up -d db ran fine and created a gis database in the Docker image. Make sure port 5432 is not in use with an existing Postgres install - either stop this, or update POSTGRES_PUBLIC_PORT=5432 to a different port in the .env file.

The following command docker-compose up -d imposm osmupdate seemed to complete successfully with the output:

osmplanet_db_1 is up-to-date
Starting osmplanet_imposm_1    ... done
Starting osmplanet_osmupdate_1 ... done

However, nothing seems to get populated in the database. Looking at the logs with docker-compose logs -f imposm output:

imposm_1             | OSM PBF file: /home/settings/country.pbf
imposm_1             | Mapping: /home/settings/country.pbf
imposm_1             | SQL Post Import: /home/settings/post-pbf-import.sql
imposm_1             | No json files to limit import detected in /home/settings
imposm_1             | Not using QGIS default styles.
imposm_1             | No *.geojson detected, so no clipping.
imposm_1             | The checkup is OK.
imposm_1             | The database is empty. Let's import the PBF : /home/settings/country.pbf
imposm_1             | ['imposm', 'import', '-diff', '-deployproduction', '-overwritecache', '-cachedir', '/home/cache', '-srid', '4326', '-dbschema-production', 'osm', '-dbschema-import', 'osm_import', '-dbschema-backup', 'osm_backup', '-diffdir', '/home/settings', '-mapping', '/home/settings/mapping.yml', '-read', '/home/settings/country.pbf', '-write', '-connection', 'postgis://osm:xxx@xxxx@db:5432/gis?sslmode=require']
imposm_1             | [2022-09-28T14:13:50Z] 0:00:00 [error] opening database: opening db: ping Postgres DB: dial tcp: address tcp/98Od2: unknown port
imposm_1             | 50
imposm_1             | An error occured in imposm with the original file.osmplanet_imposm_1 exited with code 0

After several hours of checking and running through the steps again...it turned out my strong generated database password contained@ symbols, breaking the connection string!

Output running now - I'll see how long it takes with only 900MB of RAM available..

[timlinux]

Oh wow this is fantastic feedback, thanks!

The planet import does to take a loooooong time so brace yourself!

I will incorporate your excellent notes into the readme!

Regards

Tim