From 4b9a800cf9dd6908746b9c2aedc257db6f1636be Mon Sep 17 00:00:00 2001 From: Marcin Rataj Date: Wed, 11 Dec 2024 20:42:26 +0100 Subject: [PATCH 1/3] docs: docker container limits --- docs/install/run-ipfs-inside-docker.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/docs/install/run-ipfs-inside-docker.md b/docs/install/run-ipfs-inside-docker.md index c9715bfe1..37f8aa23c 100644 --- a/docs/install/run-ipfs-inside-docker.md +++ b/docs/install/run-ipfs-inside-docker.md @@ -105,6 +105,22 @@ docker run -d --name ipfs \ See the `gateway` example on the [go-ipfs-docker-examples repository](https://github.com/ipfs-shipyard/go-ipfs-docker-examples) ::: +## Configuring resource limits + +When deploying IPFS Kubo in containerized environments, it's crucial to align the Go runtime's resource awareness with the container's defined resource constraints via environment variables: + +- `GOMAXPROCS`: Configures the maximum number of OS threads that can execute Go code concurrently (should not be bigger than the hard container limit set via `docker --cpus`) +- `GOMEMLIMIT`: Sets the soft [memory allocation limit for the Go runtime](https://tip.golang.org/doc/gc-guide#Memory_limit) (should be slightly below the hard limit set for container via `docker --memory`) + +Example: + +```shell +docker run --cpus="4.0" --memory="8000MiB" \ + --cpus="4.0" -e GOMAXPROCS=4 \ + --memory="8000m" -e GOMEMLIMIT=7500MiB \ + ipfs/kubo:latest +``` + ## Private swarms inside Docker It is possible to initialize the container with a swarm key file (`/data/ipfs/swarm.key`) using the variables `IPFS_SWARM_KEY` and `IPFS_SWARM_KEY_FILE`. The `IPFS_SWARM_KEY` creates `swarm.key` with the contents of the variable itself, while `IPFS_SWARM_KEY_FILE` copies the key from a path stored in the variable. The `IPFS_SWARM_KEY_FILE` **overwrites** the key generated by `IPFS_SWARM_KEY`. From f16b53b26ffd0b5056284828051034c3307ab7d4 Mon Sep 17 00:00:00 2001 From: Marcin Rataj Date: Wed, 11 Dec 2024 20:48:28 +0100 Subject: [PATCH 2/3] fix: lint --- .github/styles/pln-ignore.txt | 4 +++- docs/install/run-ipfs-inside-docker.md | 2 +- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/.github/styles/pln-ignore.txt b/.github/styles/pln-ignore.txt index a4025c9ae..6f0d3968c 100644 --- a/.github/styles/pln-ignore.txt +++ b/.github/styles/pln-ignore.txt @@ -193,6 +193,8 @@ reproviding requesters retrievability roadmaps +runtime +runtime's rsa sandboxed satoshi @@ -214,7 +216,7 @@ testground testnet toolkits trustlessly -uncensorable +uncensorable undialable uniswap unreachability diff --git a/docs/install/run-ipfs-inside-docker.md b/docs/install/run-ipfs-inside-docker.md index 37f8aa23c..953303511 100644 --- a/docs/install/run-ipfs-inside-docker.md +++ b/docs/install/run-ipfs-inside-docker.md @@ -115,7 +115,7 @@ When deploying IPFS Kubo in containerized environments, it's crucial to align th Example: ```shell -docker run --cpus="4.0" --memory="8000MiB" \ +docker run # (....) --cpus="4.0" -e GOMAXPROCS=4 \ --memory="8000m" -e GOMEMLIMIT=7500MiB \ ipfs/kubo:latest From 8f42568f4ddb786ee28bdb97e83b53a580bf18b1 Mon Sep 17 00:00:00 2001 From: Marcin Rataj Date: Wed, 11 Dec 2024 20:56:30 +0100 Subject: [PATCH 3/3] chore: use current-ipfs-version explicit is better than implicit --- docs/install/run-ipfs-inside-docker.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/docs/install/run-ipfs-inside-docker.md b/docs/install/run-ipfs-inside-docker.md index 953303511..06978717f 100644 --- a/docs/install/run-ipfs-inside-docker.md +++ b/docs/install/run-ipfs-inside-docker.md @@ -1,6 +1,7 @@ --- title: Install IPFS Kubo inside Docker description: You can run IPFS inside Docker to simplify your deployment processes, and horizontally scale your IPFS infrastructure. +current-ipfs-version: v0.32.1 --- # Install IPFS Kubo inside Docker @@ -20,7 +21,7 @@ You can run Kubo IPFS inside Docker to simplify your deployment processes, as we 1. Start a container running ipfs and expose ports `4001` (P2P TCP/QUIC transports), `5001` (RPC API) and `8080` (Gateway): ```shell - docker run -d --name ipfs_host -v $ipfs_staging:/export -v $ipfs_data:/data/ipfs -p 4001:4001 -p 4001:4001/udp -p 127.0.0.1:8080:8080 -p 127.0.0.1:5001:5001 ipfs/kubo:latest + docker run -d --name ipfs_host -v $ipfs_staging:/export -v $ipfs_data:/data/ipfs -p 4001:4001 -p 4001:4001/udp -p 127.0.0.1:8080:8080 -p 127.0.0.1:5001:5001 ipfs/kubo:v0.32.1 ``` ::: danger NEVER EXPOSE THE RPC API TO THE PUBLIC INTERNET @@ -70,7 +71,7 @@ You can run Kubo IPFS inside Docker to simplify your deployment processes, as we When starting a container running ipfs for the first time with an empty data directory, it will call `ipfs init` to initialize configuration files and generate a new keypair. At this time, you can choose which profile to apply using the `IPFS_PROFILE` environment variable: ```shell -docker run -d --name ipfs_host -e IPFS_PROFILE=server -v $ipfs_staging:/export -v $ipfs_data:/data/ipfs -p 4001:4001 -p 4001:4001/udp -p 127.0.0.1:8080:8080 -p 127.0.0.1:5001:5001 ipfs/kubo:latest +docker run -d --name ipfs_host -e IPFS_PROFILE=server -v $ipfs_staging:/export -v $ipfs_data:/data/ipfs -p 4001:4001 -p 4001:4001/udp -p 127.0.0.1:8080:8080 -p 127.0.0.1:5001:5001 ipfs/kubo:v0.32.1 ``` ## Customizing your node @@ -118,7 +119,7 @@ Example: docker run # (....) --cpus="4.0" -e GOMAXPROCS=4 \ --memory="8000m" -e GOMEMLIMIT=7500MiB \ - ipfs/kubo:latest + ipfs/kubo:v0.32.1 ``` ## Private swarms inside Docker @@ -126,14 +127,14 @@ docker run # (....) It is possible to initialize the container with a swarm key file (`/data/ipfs/swarm.key`) using the variables `IPFS_SWARM_KEY` and `IPFS_SWARM_KEY_FILE`. The `IPFS_SWARM_KEY` creates `swarm.key` with the contents of the variable itself, while `IPFS_SWARM_KEY_FILE` copies the key from a path stored in the variable. The `IPFS_SWARM_KEY_FILE` **overwrites** the key generated by `IPFS_SWARM_KEY`. ```shell -docker run -d --name ipfs_host -e IPFS_SWARM_KEY= -v $ipfs_staging:/export -v $ipfs_data:/data/ipfs -p 4001:4001 -p 4001:4001/udp -p 127.0.0.1:8080:8080 -p 127.0.0.1:5001:5001 ipfs/kubo:latest +docker run -d --name ipfs_host -e IPFS_SWARM_KEY= -v $ipfs_staging:/export -v $ipfs_data:/data/ipfs -p 4001:4001 -p 4001:4001/udp -p 127.0.0.1:8080:8080 -p 127.0.0.1:5001:5001 ipfs/kubo:v0.32.1 ``` The swarm key initialization can also be done using docker secrets, and requires `docker swarm` or `docker-compose`: ```shell cat your_swarm.key | docker secret create swarm_key_secret - -docker run -d --name ipfs_host --secret swarm_key_secret -e IPFS_SWARM_KEY_FILE=/run/secrets/swarm_key_secret -v $ipfs_staging:/export -v $ipfs_data:/data/ipfs -p 4001:4001 -p 4001:4001/udp -p 127.0.0.1:8080:8080 -p 127.0.0.1:5001:5001 ipfs/kubo:latest +docker run -d --name ipfs_host --secret swarm_key_secret -e IPFS_SWARM_KEY_FILE=/run/secrets/swarm_key_secret -v $ipfs_staging:/export -v $ipfs_data:/data/ipfs -p 4001:4001 -p 4001:4001/udp -p 127.0.0.1:8080:8080 -p 127.0.0.1:5001:5001 ipfs/kubo:v0.32.1 ``` ## Key rotation inside Docker @@ -142,10 +143,10 @@ It is possible to do key rotation in an ephemeral container that is temporarily ```shell # given container named 'ipfs-test' that persists repo at /path/to/persisted/.ipfs -docker run -d --name ipfs-test -v /path/to/persisted/.ipfs:/data/ipfs ipfs/kubo:latest +docker run -d --name ipfs-test -v /path/to/persisted/.ipfs:/data/ipfs ipfs/kubo:v0.32.1 docker stop ipfs-test # key rotation works like this (old key saved under 'old-self') -docker run --rm -it -v /path/to/persisted/.ipfs:/data/ipfs ipfs/kubo:latest key rotate -o old-self -t ed25519 +docker run --rm -it -v /path/to/persisted/.ipfs:/data/ipfs ipfs/kubo:v0.32.1 key rotate -o old-self -t ed25519 docker start ipfs-test # will start with the new key ```