Merge pull request #3 from linuxserver/multi-host

README.md

@@ -52,7 +52,7 @@ The architectures supported by this image are:
 
 ## Application Setup
 
-You can specify mods to download via the `DOCKER_MODS` environment variable like any other container, or allow discovery through docker by mounting the docker socket into the container (or configuring a suitable alternative endpoint via DOCKER_HOST).
+You can specify mods to download via the `DOCKER_MODS` environment variable like any other container, or allow discovery through docker by mounting the docker socket into the container (or configuring a suitable alternative endpoint via the built-in `DOCKER_HOST` environment variable). Whichever option you choose the appropriate `DOCKER_MODS` environment variable must still be present on the containers that need to install them.
 
 The Modmanager container will download all needed mods on startup and then check for updates every 6 hours; if you're using docker discovery it will automatically pick up any new mods.
 
@@ -62,9 +62,48 @@ If a mod requires additional packages to be installed, each container will still
 
 Note that the Modmanager container itself does not support applying mods *or* custom files/services.
 
+**Modmanager is only supported for use with Linuxserver images built after 2025-01-01, while it may work with 3rd party containers using our images as a base we will not provide support for them.**
+
 ### Security considerations
 
-Mapping `docker.sock` is a potential security liability because docker has root access on the host and any process that has full access to `docker.sock` would also have root access on the host. Docker api has no built-in way to set limitations on access, however, you can use a proxy for the `docker.sock` via a solution like [our docker socket proxy](https://github.com/linuxserver/docker-socket-proxy), which adds the ability to limit access. Then you would just set `DOCKER_HOST=` environment variable to point to the proxy address.
+Mapping `docker.sock` is a potential security liability because docker has root access on the host and any process that has full access to `docker.sock` would therefore also have root access on the host. The docker API has no built-in way to set limitations on access, however, you can use a proxy for `docker.sock` via a solution like [our docker socket proxy](https://github.com/linuxserver/docker-socket-proxy), which adds the ability to limit API access to specific endpoints.
+
+### Multiple Hosts
+
+>[!NOTE]
+>Make sure you fully understand what you're doing before you try and set this up as there are lots of ways it can go wrong if you're just guessing.
+
+Modmanager can query & download mods for remote hosts, as well as the one on which it is installed. At a very basic level if you're just using the `DOCKER_MODS` env and not docker discovery, simply mount the `/modcache` folder on your remote host(s), ensuring it is mapped for all participating containers.
+
+If you are using docker discovery, our only supported means for connecting to remote hosts is [our socket proxy container](https://github.com/linuxserver/docker-socket-proxy/). Run an instance on each remote host:
+
+>[!WARNING]
+>DO NOT expose a socket proxy to your LAN if it allows any write operations (`POST=1`, `ALLOW_RESTART=1`, etc) or exposes any API elements that are not absolutely necessary. NEVER expose a socket proxy to your WAN.
+
+```yml
+  modmanager-dockerproxy:
+    image: lscr.io/linuxserver/socket-proxy:latest
+    container_name: modmanager-dockerproxy
+    environment:
+      - CONTAINERS=1
+      - POST=0
+    volumes:
+      - /var/run/docker.sock:/var/run/docker.sock:ro
+    tmpfs:
+      - /run:exec
+    ports:
+      - 2375:2375
+    restart: unless-stopped
+    read_only: true
+```
+
+And then add it to the `DOCKER_MODS_EXTRA_HOSTS` env using the full protocol and port, separating multiple servers with a pipe (`|`), e.g.
+
+```yaml
+  - DOCKER_MODS_EXTRA_HOSTS=tcp://host1.example.com:2375|tcp://host2.example.com:2375|tcp://192.168.0.5:2375
+```
+
+As above you will need to mount the `/modcache` folder on your remote host(s), ensuring it is mapped for all participating containers.
 
 ## Usage
 
@@ -84,6 +123,7 @@ services:
     environment:
       - DOCKER_MODS= `#optional`
       - DOCKER_HOST= `#optional`
+      - DOCKER_MODS_EXTRA_HOSTS= `#optional`
     volumes:
       - /path/to/modcache:/modcache
       - /var/run/docker.sock:/var/run/docker.sock:ro `#optional`
@@ -97,6 +137,7 @@ docker run -d \
   --name=modmanager \
   -e DOCKER_MODS= `#optional` \
   -e DOCKER_HOST= `#optional` \
+  -e DOCKER_MODS_EXTRA_HOSTS= `#optional` \
   -v /path/to/modcache:/modcache \
   -v /var/run/docker.sock:/var/run/docker.sock:ro `#optional` \
   --restart unless-stopped \
@@ -111,6 +152,7 @@ Containers are configured using parameters passed at runtime (such as those abov
 | :----: | --- |
 | `-e DOCKER_MODS=` | Pipe-delimited (`\|`) list of mods to download |
 | `-e DOCKER_HOST=` | Specify the docker endpoint to use if not using the docker.sock |
+| `-e DOCKER_MODS_EXTRA_HOSTS=` | Pipe-delimited (`\|`) list of additional hosts to query & download mods for. See app setup section for details. |
 | `-v /modcache` | Modmanager mod storage. |
 | `-v /var/run/docker.sock:ro` | Mount the host docker socket into the container. |
 
@@ -234,4 +276,5 @@ Once registered you can define the dockerfile to use with `-f Dockerfile.aarch64
 
 ## Versions
 
+* **05.01.25:** - Support multiple hosts.
 * **22.12.24:** - Initial Release.

readme-vars.yml

@@ -56,7 +56,7 @@ full_custom_readme: |
 
   ## Application Setup
 
-  You can specify mods to download via the `DOCKER_MODS` environment variable like any other container, or allow discovery through docker by mounting the docker socket into the container (or configuring a suitable alternative endpoint via DOCKER_HOST).
+  You can specify mods to download via the `DOCKER_MODS` environment variable like any other container, or allow discovery through docker by mounting the docker socket into the container (or configuring a suitable alternative endpoint via the built-in `DOCKER_HOST` environment variable). Whichever option you choose the appropriate `DOCKER_MODS` environment variable must still be present on the containers that need to install them.
 
   The Modmanager container will download all needed mods on startup and then check for updates every 6 hours; if you're using docker discovery it will automatically pick up any new mods.
 
@@ -66,9 +66,48 @@ full_custom_readme: |
 
   Note that the Modmanager container itself does not support applying mods *or* custom files/services.
 
+  **Modmanager is only supported for use with Linuxserver images built after 2025-01-01, while it may work with 3rd party containers using our images as a base we will not provide support for them.**
+
   ### Security considerations
 
-  Mapping `docker.sock` is a potential security liability because docker has root access on the host and any process that has full access to `docker.sock` would also have root access on the host. Docker api has no built-in way to set limitations on access, however, you can use a proxy for the `docker.sock` via a solution like [our docker socket proxy](https://github.com/linuxserver/docker-socket-proxy), which adds the ability to limit access. Then you would just set `DOCKER_HOST=` environment variable to point to the proxy address.
+  Mapping `docker.sock` is a potential security liability because docker has root access on the host and any process that has full access to `docker.sock` would therefore also have root access on the host. The docker API has no built-in way to set limitations on access, however, you can use a proxy for `docker.sock` via a solution like [our docker socket proxy](https://github.com/linuxserver/docker-socket-proxy), which adds the ability to limit API access to specific endpoints.
+
+  ### Multiple Hosts
+
+  >[!NOTE]
+  >Make sure you fully understand what you're doing before you try and set this up as there are lots of ways it can go wrong if you're just guessing.
+
+  Modmanager can query & download mods for remote hosts, as well as the one on which it is installed. At a very basic level if you're just using the `DOCKER_MODS` env and not docker discovery, simply mount the `/modcache` folder on your remote host(s), ensuring it is mapped for all participating containers.
+
+  If you are using docker discovery, our only supported means for connecting to remote hosts is [our socket proxy container](https://github.com/linuxserver/docker-socket-proxy/). Run an instance on each remote host:
+
+  >[!WARNING]
+  >DO NOT expose a socket proxy to your LAN if it allows any write operations (`POST=1`, `ALLOW_RESTART=1`, etc) or exposes any API elements that are not absolutely necessary. NEVER expose a socket proxy to your WAN.
+
+  ```yml
+    modmanager-dockerproxy:
+      image: lscr.io/linuxserver/socket-proxy:latest
+      container_name: modmanager-dockerproxy
+      environment:
+        - CONTAINERS=1
+        - POST=0
+      volumes:
+        - /var/run/docker.sock:/var/run/docker.sock:ro
+      tmpfs:
+        - /run:exec
+      ports:
+        - 2375:2375
+      restart: unless-stopped
+      read_only: true
+  ```
+
+  And then add it to the `DOCKER_MODS_EXTRA_HOSTS` env using the full protocol and port, separating multiple servers with a pipe (`|`), e.g.
+
+  ```yaml
+    - DOCKER_MODS_EXTRA_HOSTS=tcp://host1.example.com:2375|tcp://host2.example.com:2375|tcp://192.168.0.5:2375
+  ```
+
+  As above you will need to mount the `/modcache` folder on your remote host(s), ensuring it is mapped for all participating containers.
 
   ## Usage
 
@@ -88,6 +127,7 @@ full_custom_readme: |
       environment:
         - DOCKER_MODS= `#optional`
         - DOCKER_HOST= `#optional`
+        - DOCKER_MODS_EXTRA_HOSTS= `#optional`
       volumes:
         - /path/to/modcache:/modcache
         - /var/run/docker.sock:/var/run/docker.sock:ro `#optional`
@@ -101,6 +141,7 @@ full_custom_readme: |
     --name=modmanager \
     -e DOCKER_MODS= `#optional` \
     -e DOCKER_HOST= `#optional` \
+    -e DOCKER_MODS_EXTRA_HOSTS= `#optional` \
     -v /path/to/modcache:/modcache \
     -v /var/run/docker.sock:/var/run/docker.sock:ro `#optional` \
     --restart unless-stopped \
@@ -115,6 +156,7 @@ full_custom_readme: |
   | :----: | --- |
   | `-e DOCKER_MODS=` | Pipe-delimited (`\|`) list of mods to download |
   | `-e DOCKER_HOST=` | Specify the docker endpoint to use if not using the docker.sock |
+  | `-e DOCKER_MODS_EXTRA_HOSTS=` | Pipe-delimited (`\|`) list of additional hosts to query & download mods for. See app setup section for details. |
   | `-v /modcache` | Modmanager mod storage. |
   | `-v /var/run/docker.sock:ro` | Mount the host docker socket into the container. |
 
@@ -238,6 +280,7 @@ full_custom_readme: |
 
   ## Versions
 
+  * **05.01.25:** - Support multiple hosts.
   * **22.12.24:** - Initial Release.
 
   {%- endraw %}

root/app/update-mods.sh

@@ -1,26 +1,57 @@
 #!/usr/bin/with-contenv bash
 # shellcheck shell=bash
 
-# Main script loop
-if [[ -e "/var/run/docker.sock" ]] || [[ -n "${DOCKER_HOST}" ]]; then
+find_docker_mods() {
     # Mods provided via Docker
-    echo -e "[mod-init] Searching all containers for DOCKER_MODS..."
-    for CONTAINER in $(docker ps -q); do
-        CONTAINER_MODS=$(docker inspect "${CONTAINER}" | jq -r '.[].Config.Env | to_entries | map(select(.value | match("DOCKER_MODS="))) | .[].value')
-        CONTAINER_NAME=$(docker inspect "${CONTAINER}" | jq -r .[].Name | cut -d '/' -f2)
+    if [[ "${2}" != "default" ]]; then
+        local MOD_STATE="(${2})"
+        docker context create "${2}" --docker "host=${1}" >/dev/null 2>&1
+    fi
+    docker --context "${2}" ps -q >/dev/null 2>&1 || local DOCKER_MOD_CONTEXT_FAIL=true
+    if [[ "${DOCKER_MOD_CONTEXT_FAIL}" == "true" ]]; then
+        echo "[mod-init] (ERROR) Cannot connect to the Docker daemon at ${2}, skipping host"
+        return
+    fi
+    echo -e "[mod-init] ${MOD_STATE:+${MOD_STATE} }Searching all containers in the ${2} context for DOCKER_MODS..."
+    for CONTAINER in $(docker --context "${2}" ps -q); do
+        CONTAINER_MODS=$(docker --context "${2}"  inspect "${CONTAINER}" | jq -r '.[].Config.Env | to_entries | map(select(.value | match("DOCKER_MODS="))) | .[].value')
+        CONTAINER_NAME=$(docker --context "${2}"  inspect "${CONTAINER}" | jq -r .[].Name | cut -d '/' -f2)
         if [[ -n ${CONTAINER_MODS} ]]; then
             CONTAINER_MODS=$(awk -F '=' '{print $2}' <<< "${CONTAINER_MODS}")
             for CONTAINER_MOD in $(tr '|' '\n' <<< "${CONTAINER_MODS}"); do
                 if [[ "${DOCKER_MODS}" =~ ${CONTAINER_MOD} ]]; then
-                    echo -e "[mod-init] ${CONTAINER_MOD} already in mod list, skipping"
+                    echo -e "[mod-init] ${MOD_STATE:+${MOD_STATE} }${CONTAINER_MOD} already in mod list, skipping"
                 else
-                    echo -e "[mod-init] Found new mod ${CONTAINER_MOD} for container ${CONTAINER_NAME}"
+                    echo -e "[mod-init] ${MOD_STATE:+${MOD_STATE} }Found new mod ${CONTAINER_MOD} for container ${CONTAINER_NAME}"
                     DOCKER_MODS="${DOCKER_MODS}|${CONTAINER_MOD}"
                     DOCKER_MODS="${DOCKER_MODS#|}"
                 fi
             done
         fi
     done
+    if [[ "${2}" != "default" ]]; then
+        docker context rm "${2}" >/dev/null
+    fi
+}
+
+# Main script loop
+
+# Reset DOCKER_MODS to whatever value the user passed into the container at creation time
+DOCKER_MODS="${DOCKER_MODS_STATIC}"
+
+echo -e ""
+echo -e "[mod-init] Running check for new mods and updates."
+
+if [[ -e "/var/run/docker.sock" ]] || [[ -n "${DOCKER_HOST}" ]]; then
+    find_docker_mods "${DOCKER_HOST:-docker.sock}" "default"
+fi
+
+if [[ -n "${DOCKER_MODS_EXTRA_HOSTS}" ]]; then
+    for DOCKER_MOD_CONTEXT in $(echo "${DOCKER_MODS_EXTRA_HOSTS}" | tr '|' '\n'); do
+        DOCKER_MOD_CONTEXT_NAME="${DOCKER_MOD_CONTEXT##*//}"
+        DOCKER_MOD_CONTEXT_NAME="${DOCKER_MOD_CONTEXT_NAME%%:*}"
+        find_docker_mods "${DOCKER_MOD_CONTEXT}" "${DOCKER_MOD_CONTEXT_NAME}"
+    done
 fi
 
 if [[ -n "${DOCKER_MODS}" ]]; then

root/etc/s6-overlay/s6-rc.d/init-modmanager-config/run

@@ -5,6 +5,26 @@ CRON_MINS=$((0 + RANDOM % 59))
 
 sed -i "s/@@MINUTES@@/${CRON_MINS}/" /etc/crontabs/root
 
-echo "[mod-init] Mod updates will run every 6 hours at ${CRON_MINS} minutes past the hour"
+if [[ $(date "+%-H") == 0 && $(date "+%-M") -lt ${CRON_MINS} ]]; then
+    NEXT_HOUR=0
+elif [[ $(date "+%-H") == 6 && $(date "+%-M") -lt ${CRON_MINS} ]]; then
+    NEXT_HOUR=6
+elif [[ $(date "+%-H") == 12 && $(date "+%-M") -lt ${CRON_MINS} ]]; then
+    NEXT_HOUR=12
+elif [[ $(date "+%-H") == 18 && $(date "+%-M") -lt ${CRON_MINS} ]]; then
+    NEXT_HOUR=18
+elif [[ $(date "+%-H") -ge 0 && $(date "+%-H") -le 5 ]]; then
+    NEXT_HOUR=6
+elif [[ $(date "+%-H") -ge 6 && $(date "+%-H") -le 11 ]]; then
+    NEXT_HOUR=12
+elif [[ $(date "+%-H") -ge 12 && $(date "+%-H") -le 17 ]]; then
+    NEXT_HOUR=18
+elif [[ $(date "+%-H") -ge 18 && $(date "+%-H") -le 23 ]]; then
+    NEXT_HOUR=0
+fi
+
+echo "[mod-init] Mod updates will run every 6 hours at ${CRON_MINS} minutes past the hour. Next update will be at $(date -d${NEXT_HOUR}:${CRON_MINS} '+%H:%M')."
+
+printf %s "${DOCKER_MODS}" > /run/s6/container_environment/DOCKER_MODS_STATIC
 
 /app/update-mods.sh