OHMG is a web application that facilitates public participation in the process of georeferencing and mosaicking historical maps. Currently, a single implementation of this software exists, at OldInsuranceMaps.net, a platform based around the Sanborn Map Collection at the Library of Congress (loc.gov/collections/sanborn-maps). More generic deployments are in the works.
- Home page: ohmg.dev
- Implementation: oldinsurancemaps.net
- Documentation: about.oldinsurancemaps.net
Please don't hesitate to open a ticket if you have trouble with the site, find a bug, or have suggestions otherwise.
OHMG uses the Django web framework for URL routing, auth, and the ORM, and Django Ninja to create an API. A newsletter is implemented with Django Newsletter.
The frontend is built (mostly) with Svelte, using OpenLayers for all map interfaces. OpenStreetMap and Mapbox are the basemap sources.
Other components include:
- Postgres + PostGIS
- Database
- GDAL (3.5+ release)
- A dependency of PostGIS, and also used directly for all warping/mosaicking operations.
- Celery + RabbitMQ
- Background task management (a handful of loading/splitting/warping processes run in the background)
- Presumably, Redis could be used for the broker instead of RabbitMQ.
- TiTiler
- Tileserver for georeferencing COGs (Cloud Optimized GeoTIFFs)
Running the application requires a number of components to be installed and configured properly. The following commands assume a Debian-based Linux distribution.
You will need the following packages (at least... I still need to figure out the exact list of these dependencies):
sudo apt install build-essential gdal-bin python3.8-dev libgdal-dev libgeos-devClone the repo and enter the local directory
git clone https://github.com/ohmg-dev/OldInsuranceMaps && cd OldInsuranceMapsMake virtual env
python3 -m venv env
source env/bin/activateInstall GDAL Python bindings. These must match exactly the version of GDAL you have on your system, so use this command:
pip install gdal=="`gdal-config --version`.*"Install the ohmg package into your virtual environment
pip install -e .[dev]Install pre-commit hook (if you will be writing code)
pre-commit installUse the .env.example to create your .env file
cp .env.example .envSee environment variables for more information.
In your running Postgres instance, create a database like this
psql -U postgres -c "CREATE USER ohmg WITH ENCRYPTED PASSWORD '$DB_PASSWORD'"
psql -U postgres -c "CREATE DATABASE oldinsurancemaps WITH OWNER ohmg;"
psql -U postgres -d oldinsurancemaps -c "CREATE EXTENSION PostGIS;"Alternatively, if you have set all of the DATABASE_* variables in .env, you can use the included script to complete the database setup:
source ./scripts/setup_database.shRun migrations and create admin user to access the Django admin panel (and login to the site once it's running)
python manage.py migrate
python manage.py createsuperuserLoad all the place objects to create geography scaffolding (this will take a minute or two)
python manage.py place import-allLoad test data into database (optional)
source ./scripts/load_dev_data.shThe frontend uses a suite of independently built svelte components. During development use the following command to auto-build these components and reload your browser.
cd ohmg/frontend/svelte
pnpm install
pnpm run devYou can now run
python manage.py runserverand view the site at http://localhost:8000.
However, a few more pieces need to be set up independently before the app will be fully functional. Complete the following sections and then restart the dev server so that any new .env values will be properly acquired.
In development, RabbitMQ can be run via Docker like so:
docker run --name rabbitmq --hostname my-rabbit \
-p 5672:5672 \
-p 15672:15672 \
-e RABBITMQ_DEFAULT_USER=username \
-e RABBITMQ_DEFAULT_PASS=password \
--rm \
rabbitmq:3-alpineFor convenience, this command is in the following script:
source ./scripts/rabbit_dev.shOnce RabbitMQ is running, update .env with the RABBITMQ_DEFAULT_USER and RABBITMQ_DEFAULT_PASS credentials you used above when creating the container.
Now you are ready to run Celery in development with:
source ./scripts/celery_dev.shTiTiler can also be run via Docker, using a slightly modified version of the official container (it is only modified to include the WMS endpoint extension):
docker run --name titiler \
-p 8008:8000 \
-e PORT=8000 \
-e MOSAIC_STRICT_ZOOM=False \
-e WORKERS_PER_CORE=1 \
--rm \
-it \
ghcr.io/mradamcox/titiler:0.11.6-ohmgOr the same command is wrapped in:
source ./scripts/titiler_dev.shThis will start a container running TiTiler and expose it to localhost:8008.
Make sure you have TITILER_HOST=http://localhost:8008 in .env (see environment variables).
During development, a separate HTTP server must be used to supply TiTiler with COG endpoints, because the Django dev server does not serve HTTP range requests (more on this here and here). The easiest workaround is to use node's http-server.
From within the repository root (alongside the uploaded directory) run:
npx http-server .All COGs will now be accessible at http://localhost:8080/uploaded/.
Make sure you have MEDIA_HOST=http://localhost:8080 in .env. MEDIA_HOST is a prefix that will be prepended to any uploaded media paths that are passed to TiTiler.
In production, you will already be using a webserver for static files so you will not need to use http-server. In this case, remove MEDIA_HOST from .env or set it to ''.
All tests are stored in ohmg/tests. Make sure you have installed dev requirements, then run:
python manage.py testTo skip the tests that make external calls to the LOC API, use the following command. Keep in mind that coverage numbers will be lower when you skip tests.
python manage.py test --exclude-tag=locsection in progress