Skip to content

Commit

Permalink
Initial release (previous work was in resolve.rs)
Browse files Browse the repository at this point in the history
  • Loading branch information
fileformat committed Aug 15, 2021
1 parent 1bd23f7 commit af408fa
Show file tree
Hide file tree
Showing 5 changed files with 188 additions and 0 deletions.
18 changes: 18 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# EditorConfig is awesome: http://EditorConfig.org

# top-most EditorConfig file
root = true

[*]
charset = utf-8
end_of_line = lf
indent_size = tab
insert_final_newline = true
tab_width = 4
trim_trailing_whitespace = true

[*.json]
indent_size = 2

[*.yaml]
indent_size = 2
39 changes: 39 additions & 0 deletions .github/workflows/release.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
name: Release

on:
schedule:
- cron: '42 0 * * 1'
workflow_dispatch:

jobs:
release:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v1

- name: Get and encrypt Maxmind databases
run: bin/update_maxmind.sh
env:
MMDB_ENCRYPTION_KEY: ${{ secrets.MMDB_ENCRYPTION_KEY }}
MAXMIND_LICENSE_KEY: ${{ secrets.MAXMIND_LICENSE_KEY }}

- name: Set release info
run: |
echo "RELEASE_DATE=$(date -u +%Y-%m-%dT%H:%M:%SZ)" >> $GITHUB_ENV
echo "RELEASE_TAG=$(date -u +%Y%m%d-%H%M%S)" >> $GITHUB_ENV
- name: Release
uses: softprops/action-gh-release@v1
with:
files: |
build/GeoLite2-ASN.mmdb.enc
build/GeoLite2-City.mmdb.enc
build/mmdb.iv
build/mmdb.md5
tag_name: ${{ env.RELEASE_TAG }}
name: Release ${{ env.RELEASE_DATE }}
body: |
Automatically created on ${{ env.RELEASE_DATE }}
draft: false
prerelease: false
11 changes: 11 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
build/
*.crt
dist/
.DS_Store
*.env
.idea/
*.key
*.log
*.mmdb
tmp/
*.tmp
34 changes: 34 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# MaxMind for Resolve.rs [<img alt="Resolve.rs Logo" src="https://resolve.rs/favicon.svg" height="96" align="right"/>](https://resolve.rs/)

The MaxMind "Lite" databases are free to use, but you cannot redistribute them, and cannot (reliably) download them every time a server starts. To get a server to start without hitting MaxMind directly, I use an encrypted version that automatically updates once a week.

## Using

This is just for [resolve.rs](https://resolve.rs), but you can do the same thing with your own fork. You need to set the following Github secrets:

* `MAXMIND_LICENSE_KEY` - from MaxMind
* `MAXMIND_ACCOUNT_ID` - also from MaxMind
* `MMDB_ENCRYPTION_KEY` - generate a 32-byte (64 hex digits) encryption key

You will use the `MMDB_ENCRYPTION_KEY` when decrypting the database files on your server.

## License

The script is licensed under the [GNU Affero General Public License v3.0](LICENSE.txt).

This product includes GeoLite2 data created by MaxMind, available from [www.maxmind.com](https://dev.maxmind.com/geoip/geolite2-free-geolocation-data/)

## Release Files

- GeoLite2-ASN.mmdb.enc: encrypted ASN lookup database
- GeoLite2-City.mmdb.enc: encrypted city lookup database
- mmdb.iv: initialization vector used for encryption
- mmdb.md5: md5 hashes of the unencrypted files (to detect updates)

## Credits

[![Bash](https://www.vectorlogo.zone/logos/gnu_bash/gnu_bash-ar21.svg)](https://www.gnu.org/software/bash/ "Scripting")
[![Git](https://www.vectorlogo.zone/logos/git-scm/git-scm-ar21.svg)](https://git-scm.com/ "Version control")
[![Github](https://www.vectorlogo.zone/logos/github/github-ar21.svg)](https://github.com/ "Code hosting")
[![MaxMind](https://www.vectorlogo.zone/logos/maxmind/maxmind-ar21.svg)](https://www.maxmind.com/ "IP geolocation and ASN databases")
[![OpenSSL](https://www.vectorlogo.zone/logos/openssl/openssl-ar21.svg)](https://www.openssl.org/ "Encryption")
86 changes: 86 additions & 0 deletions bin/update_maxmind.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
#!/usr/bin/env bash
#
# downloads a fresh copy of the MaxMind databases for ASN & geolocation lookups and makes an encrypted release
#

set -o errexit
set -o pipefail
set -o nounset

echo "INFO: starting MaxMind update at $(date -u +%Y-%m-%dT%H:%M:%SZ)"

SCRIPT_DIR="$( cd "$( dirname "$0" )" && pwd )"
REPO_DIR="$(realpath "${SCRIPT_DIR}/..")"

ENV_FILE="$(realpath "${REPO_DIR}/.env")"
if [ -f "${ENV_FILE}" ]; then
echo "INFO: loading ${ENV_FILE} into environment"
export $(cat ${ENV_FILE})
fi

TAR=tar
if [[ $OSTYPE == 'darwin'* ]]; then
echo "INFO: running on MacOS so using gtar instead of tar"
TAR=gtar
fi

BUILD_DIR="$(realpath "${REPO_DIR}/build")"
if [ ! -d "${BUILD_DIR}" ]; then
echo "INFO: creating build directory ${BUILD_DIR}"
mkdir -p "${BUILD_DIR}"
else
echo "INFO: using existing build directory ${BUILD_DIR}"
fi

TMP_ASN_FILE=$(mktemp)
echo "INFO: download MaxMind ASN database into ${TMP_ASN_FILE}"
curl --silent "https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-ASN&suffix=tar.gz&license_key=${MAXMIND_LICENSE_KEY}" >"${TMP_ASN_FILE}"
${TAR} -xzf ${TMP_ASN_FILE} --directory="${BUILD_DIR}" --wildcards --strip-components 1 "*.mmdb"
rm "${TMP_ASN_FILE}"

TMP_CITY_FILE=$(mktemp)
echo "INFO: download MaxMind City database into ${TMP_CITY_FILE}"
curl --silent "https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-City&suffix=tar.gz&license_key=${MAXMIND_LICENSE_KEY}" >"${TMP_CITY_FILE}"
${TAR} -xzf ${TMP_CITY_FILE} --directory="${BUILD_DIR}" --wildcards --strip-components 1 "*.mmdb"
rm "${TMP_CITY_FILE}"

md5sum ${BUILD_DIR}/*.mmdb | sort >${BUILD_DIR}/mmdb.md5

#LATER: curl release/download/latest/mmdb.md5
#DIFF=$(git diff --name-only "${BUILD_DIR}/mmdb.md5")
#
#if [ "${DIFF}" == "" ]; then
# echo "INFO: no changes, exiting at $(date -u +%Y-%m-%dT%H:%M:%SZ)"
# exit 0
#fi

if [ "${MMDB_ENCRYPTION_KEY:-BAD}" = "BAD" ]; then
echo "INFO: no encryption keys, exiting. (but app can still be run locally)"
exit 1
fi

#
# generate (and save) a new IV every time
#
MMDB_ENCRYPTION_IV=$(head -c 16 /dev/urandom | xxd -l 16 -c 16 -p)
echo -n ${MMDB_ENCRYPTION_IV} > ${BUILD_DIR}/mmdb.iv

ASN_FILE="${BUILD_DIR}/GeoLite2-ASN.mmdb"
echo "INFO: starting encryption of ${ASN_FILE} (file size=$(du ${ASN_FILE} | cut -f 1))"
gzip --stdout ${ASN_FILE} | openssl enc -aes-256-ctr \
-K ${MMDB_ENCRYPTION_KEY} \
-iv ${MMDB_ENCRYPTION_IV} \
-out "${ASN_FILE}.enc"
rm ${ASN_FILE}
echo "INFO: encryption complete (file size=$(du ${ASN_FILE}.enc | cut -f 1))"

CITY_FILE="${BUILD_DIR}/GeoLite2-City.mmdb"
echo "INFO: starting encryption of ${CITY_FILE} (file size=$(du ${CITY_FILE} | cut -f 1))"
gzip --stdout ${CITY_FILE} | openssl enc -aes-256-ctr \
-K ${MMDB_ENCRYPTION_KEY} \
-iv ${MMDB_ENCRYPTION_IV} \
-out "${CITY_FILE}.enc"
rm ${CITY_FILE}
echo "INFO: encryption complete (file size=$(du ${CITY_FILE}.enc | cut -f 1))"

echo "INFO: complete MaxMind update at $(date -u +%Y-%m-%dT%H:%M:%SZ)"

0 comments on commit af408fa

Please sign in to comment.