Updating for version 3.0.0

Dockerfile

@@ -8,17 +8,14 @@ WORKDIR /root
 # ARG UPSTREAM_VER=feature/docker-version
 # ADD https://api.github.com/repos/TrueBlocks/trueblocks-core/git/refs/heads/$UPSTREAM_VER version.json
 
-
 ARG UPSTREAM_VER=master
 
 RUN git clone -b "${UPSTREAM_VER}" --single-branch --progress --depth 1 \
     https://github.com/TrueBlocks/trueblocks-core.git && \
     cd trueblocks-core && \
-    cat src/CMakeLists.txt | grep -v "examples" | grep -v dev_tools >x && \
-    cat x >src/CMakeLists.txt && \
-    rm -f x && \
     mkdir -p build && \
     cd build && \
+    ../scripts/go-work-sync.sh && \
     cmake ../src && \
     make -j 5
 
@@ -27,7 +24,6 @@ FROM alpine:latest
 RUN apk --no-cache add gzip libstdc++ libgcc libcurl python3 python3-dev procps bash curl nano findutils
 
 COPY --from=builder /root/trueblocks-core/bin /usr/local/bin
-COPY --from=builder /root/.local/bin/chifra /root/.local/bin/chifra
 COPY --from=builder /root/.local/share/trueblocks /root/.local/share/trueblocks
 
 ARG SERVE_PORT=8080

README.md

@@ -11,47 +11,46 @@
 
 - [Introduction](#introduction)
 - [Requirements](#requirements)
-- [Locally running RPC endpoints](#locally-running-rpc-endpoints)
+- [A note on RPC endpoints](#a-note-on-rpc-endpoints)
 - [Configuration](#configuration)
 - [Running the tool](#running-the-tool)
   - [Using the API](#using-the-api)
   - [Using `chifra`](#using-chifra)
-- [The unchained index](#the-unchained-index)
+- [The Unchained Index](#the-unchained-index)
 - [Data science](#data-science)
-- [Quick Alternative Method to Start the Container](#quick-alternative-method-to-start-the-container)
+- [Tutorials and examples](#tutorials-and-examples)
+- [Getting started - quick and dirty](#getting-started---quick-and-dirty)
 - [Troubleshooting](#troubleshooting)
 - [Other](#other)
 
 ## Introduction
 
-TrueBlocks is a local-first indexing / data access solution that you may use for data science or as a locally-running backend for your Web 3.0 projects.
+TrueBlocks is a local-first indexing / data access solution for EVM blockchains. This makes it very useful for data science or as a locally-running backend for your Web 3.0 applications.
 
-This docker repo is intentionally minimal. See [the core repo](https://github.com/TrueBlocks/trueblocks-core) for more information about the Unchained Index, chifra (our command-line tool), and the TrueBlocks data API.
+This `docker` repo is intentionally minimal serving as a thin layer over [TrueBlocks Core](https://github.com/TrueBlocks/trueblocks-core). Please see that repo for more information about the Unchained Index, chifra (our command-line tool), and the TrueBlocks data API and SDKs.
 
-This repo is pre-alpha and comes with no warranty implied or otherwise. Use at your own discretion.
+This repo is `alpha` software and comes with no warranty implied or otherwise. Use at your own discretion.
 
 ## Requirements
 
-`docker` and `docker-compose-plugin` are required. Latest version is suggested.
+`docker` and `docker-compose-plugin` are required. The latest versions of both are suggested.
 
-**Note:** this repo uses the Compose Plugin which uses the `docker compose` syntax NOT the Compose Standalone which uses `docker-compose`.
+**Note:** We use the `Compose Plugin` which uses the `docker compose` syntax *NOT* `Compose Standalone` which uses `docker-compose`.
 
-## Locally running RPC endpoints
+## A note on RPC endpoints
 
-While TrueBlocks and chifra work with remote RPC endpoints, it is highly recommended that you have you own locally running endpoints. An excellent way to do that is to run Erigon on a `dAppNode`.
+TrueBlocks works with remote RPC endpoints, but we highly recommended that you run your own RPC. An excellent way on a `dAppNode`.
 
-If you use a shared, rate-limited endpoint such as the many RPC-for-a-service offerings, there is a high likelihood that you will be rate limited. Because TrueBlocks was designed for direct-to-local-endpoint use cases, such rate limiting is not currently in the code.
-
-Just so that you're aware.
+If you must use a shared endpoint, it's likely that our scraper will be rate limited. This will cause the scraper to fail. You may dial down the speed of the scraper with the `--sleep` option, but of course, now you've dialed down the speed. Your call.
 
 ## Configuration
 
-To get started, create a file called `.env` in this folder. An [env.example](env.example) file explaining each setting is provided to help you. Adjust the RPC provider to point to a (preferably local) RPC endpoint.
+To get started, create a file called `.env` in the current folder. An [env.example](env.example) file explaining each setting is provided to help you. Adjust the RPC provider to point to an RPC endpoint.
 
 ```[shell]
 TB_SETTINGS_DEFAULTCHAIN=mainnet
 TB_CHAINS_MAINNET_RPCPROVIDER=http://host.docker.internal:8545
-TB_KEYS_ETHERSCAN_APIKEY=<your-key>
+TB_KEYS_ETHERSCAN_APIKEY=<your-Etherscan-api-key>
 TB_SETTINGS_CACHEPATH=/cache
 TB_SETTINGS_INDEXPATH=/unchained
 ```
@@ -64,43 +63,43 @@ The `ETHERSCAN_APIKEY` key is optional, but useful to enable the articulation fe
 
 2. Create two folders on your host machine to persist the Unchained Index and binary caches:
 
-```[shell]
-mkdir -p /Users/user/Data/docker/cache
-mkdir -p /Users/user/Data/docker/unchained
-```
+    ```shell
+    mkdir -p /Users/user/Data/docker/cache
+    mkdir -p /Users/user/Data/docker/unchained
+    ```
 
-**Note:** Adjust these paths appropriately for your machine.
+    **Note:** Adjust these paths appropriately for your machine.
 
 3. Replace the `source` paths in `docker-compose.yml` with the paths you just made in step 2.
 
-```yml
-...
-    volumes:
-      - type: bind
-        source: /Users/user/Data/docker/cache     # <--- REPLACE THIS WITH ANY VALID PATH
-        target: /cache
-      - type: bind
-        source: /Users/user/Data/docker/unchained # <--- REPLACE THIS WITH ANY VALID PATH
-        target: /unchained
-...
-```
+    ```yml
+    ...
+        volumes:
+          - type: bind
+            source: /Users/user/Data/docker/cache     # <--- REPLACE WITH A VALID PATH
+            target: /cache
+          - type: bind
+            source: /Users/user/Data/docker/unchained # <--- REPLACE WITH A VALID PATH
+            target: /unchained
+    ...
+    ```
 
-3. Run `docker compose up`
+4. Run `docker compose up`
 
+Leave this tool running in a separate terminal window. It will start the TrueBlocks API server and the scraper. You should see a note indicating the server URL and the progress of the scraper.
 
 ### Using the API
 
-Both methods start the [TrueBlocks API server](https://trueblocks.io/api/). Leave this process running and open a new terminal window.
+The above commands start TrueBlocks' [API server](https://trueblocks.io/api/). Leave the server running and open a new terminal window.
 
-Use `curl` to access the API through [http://localhost:8080](http://localhost:8080). For example, the command
+Use `curl` to test access to the API. For example, the command
 
 ```[shell]
 curl "http://localhost:8080/blocks?blocks=1-1000:10"
 ```
 
 will extract every 10th block between blocks 1 and 1,000.
 
-
 ### Using `chifra`
 
 You may also use the `chifra` command line tool directly. From the current folder, type:
@@ -119,21 +118,23 @@ will show all available chifra tools. See the [full documentation](https://trueb
 
 **Pro user tip:**
 
-If you add this to your shell's init script, you will be able to run `chifra` from any command line:
+You may add the following alias to your shell's init script to make `chifra` available from any command line:
 
-```
+```shell
 alias chifra="cd <path>/<to>/trueblocks-docker ; ./scripts/chifra.sh "
 ```
 
-## The unchained index
+If you do this, and wish to also run the `core` directly, it may cause a conflict finding `chifra`.
+
+## The Unchained Index
 
 In the future, this docker will initialize and maintain [the Unchained Index](https://trueblocks.io/papers/2022/file-format-spec-v0.40.0-beta.pdf). Until then, you must initialize it and maintain it yourself.
 
 Before doing that, please [read and understand this discussion](https://trueblocks.io/docs/install/get-the-index/). It will have an important impact on how `chifra` works for you.
 
 Once you've read the above, run one of the following two commands:
 
-```[shell]
+```shell
 # If you want to initialize the full index (recommended if you have space), or
 ./scripts/chifra.sh init --all
 
@@ -145,7 +146,7 @@ Once you've read the above, run one of the following two commands:
 
 Depending on your connection, the above will take several minutes or as much as several hours.
 
-When the initialization finishes, decide if you want to run the `scraper`. The scrape maintains the index to the front of the chain. (Note: if you're exploring older data, this step may be optional.)
+When the initialization finishes, decide if you want to run the `scraper`. The scrape maintains the index to the front of the chain. (Note: if you're exploring older data, this step is optional.)
 
 To start the scraper, do this only after the `chifra init` command finishes:
 
@@ -159,19 +160,22 @@ Allow this process to continue running in its own terminal window or `tmux` sess
 
 `chifra` is an excellent data science tool. See a few of our articles ([here](https://trueblocks.io/tags/community/), [here](https://trueblocks.io/tags/trueblocks/), and [here](https://trueblocks.io/tags/recipes/)) for ideas on how to take advantage of this very useful tool.
 
-TODO: Add tutorials.
+## Tutorials and examples
 
-## Quick Alternative Method to Start the Container
+The core repo has many examples on how to use not only the `chifra` command line, but also the `TrueBlocks API` and the `TrueBlocks SDK`. Please see the [core repo](https://github.com/TrueBlocks/trueblocks-core) for more information. Note that all three methods of using TrueBlocks are compatible with each other. Learn once, use many.
 
-This alternative uses docker volumes to persist the Unchained Index and binary caches.
-It is a great option if you intend to always use docker, you don't need need the
-Unchained index files externally, or if you want to quickly try out chifra.
+## Getting started - quick and dirty
+
+This alternative method of starting the docker image uses volumes to persist the
+Unchained Index and binary caches. It is a great option if you intend only to
+use our docker version, if you don't need need the Unchained index files
+externally to docker, or if you want to quickly try out chifra.
 
 1. Make sure your `.env` file is configured [Configuration](#configuration)
 
-2. Run the command:
+2. Run this command:
 
-```[bash]
+```bash
 docker compose -f docker-compose.yml -f docker-compose.volume-override.yml up
 ```
 
@@ -181,23 +185,23 @@ docker compose -f docker-compose.yml -f docker-compose.volume-override.yml up
 
 If you are running your RPC directly on your host machine (not within a docker container) you may get the message:
 
-```
+```text
 Could not load RPC provider: Post "http://localhost:8545": dial tcp 127.0.0.1:8545: connect: connection refused
 ```
 
 To fix this, tell docker to use your host network by adding `network_mode: "host"` to the `docker-compose.yml`:
 
 ```yml
-    volumes:
-      - type: bind
-        # The source should match the folder you made to store the cache
-        source: /Users/user/Data/docker/cache
-        target: /cache
-      - type: bind
-        # The source should match the folder you made to store the Unchained index
-        source: /Users/user/Data/docker/unchained
-        target: /unchained
-    network_mode: "host"
+volumes:
+  - type: bind
+    # The source should match the cache folder you made when getting started
+    source: /Users/user/Data/docker/cache
+    target: /cache
+  - type: bind
+    # The source should match the Unchained Index folder you made when getting started
+    source: /Users/user/Data/docker/unchained
+    target: /unchained
+network_mode: "host"
 ```
 
 **Additional property 'name' is not allowed**
@@ -221,7 +225,7 @@ We love contributors. Please see information about our [work flow](https://githu
 1. Fork this repository into your own repo.
 2. Create a branch: `git checkout -b <branch_name>`.
 3. Make changes to your local branch and commit them to your forked repo: `git commit -m '<commit_message>'`
-4. Push back to the original branch: `git push origin TrueBlocks/trueblocks-core`
+4. Push back to the original branch: `git push origin TrueBlocks/trueblocks-docker`
 5. Create the pull request.
 
 **Contact**

docker-compose.yml

@@ -20,4 +20,3 @@ services:
 volumes:
   unchained:
   cache:
-

env.example

@@ -1,33 +1,36 @@
 # This is an example `.env` file. Copy this file and rename it if you wish.
 
 #---------
-# To various degrees, TrueBlocks runs against any evm-based blockchain (although we only officially
-# support `mainnet`, `sepolia`, and `gnosis`). Specify the default chain you wish to use. The default
-# chain is the one `chifra` processes if you don't specify a `--chain` parameter.
+# TrueBlocks runs against any evm-based blockchain. Specify the default chain you
+# wish to use. The default chain is the one `chifra` processes if you don't
+# specify a `--chain` parameter.
 #---------
 TB_SETTINGS_DEFAULTCHAIN=mainnet
 
 #---------
-# For each chain you wish to support, you must instruct TrueBlocks how to connect to it. Below, we
-# configure for "mainnet" and "sepolia". If you wish to support other chains, add their RPC providers
-# below. Variables are named as follows: TB_CHAINS_<chain>_RPCPROVIDER. The value provided should be
-# the URL of the RPC provider for the chain. It's highly suggested that the URL is local. 
+# For each chain you wish to support, you must instruct TrueBlocks how to connect
+# to it. Below, we configure for "mainnet" and "sepolia". If you wish to support
+# other chains, add their RPC providers below. Variables are named as follows:
+# TB_CHAINS_<chain>_RPCPROVIDER. The value provided should be the URL of the RPC
+# provider for the chain. It's highly suggested that the URL is local. 
 #---------
-# Note: The following values most likely will not work on your machine. Replace these vaules with
-#       values that work on your machine. The values below work for us on a Mac running Docker Desktop.
+# Note: The following values most likely will not work on your machine. Replace
+#       these vaules with values that work on your machine. The values below work
+#       for us on a Mac running Docker Desktop.
 TB_CHAINS_MAINNET_RPCPROVIDER=http://host.docker.internal:8545
 TB_CHAINS_SEPOLIA_RPCPROVIDER=http://host.docker.internal:8546
 
 #---------
-# This value, which is optional, enables the articulation feature. Eventually, we will provide a
-# way to use the articulation feature without an API key. In the meantime, you can get an API
-# key from https://etherscan.io/apis.
+# This value, which is optional, enables the articulation feature. Eventually, we
+# will provide a way to use the articulation feature without an API key. In the
+# meantime, you can get an API key from https://etherscan.io/apis.
 #---------
 TB_KEYS_ETHERSCAN_APIKEY=<not-set>
 
 #---------
-# These two paths, while optional, allow us to more easily use the data produced by TrueBlocks
-# outside the docker image. We recommend you leave these settings as they are.
+# These two paths, while optional, allow us to more easily use the data produced
+# by TrueBlocks outside the docker image. We recommend you leave these settings
+# as they are.
 #---------
 TB_SETTINGS_CACHEPATH=/cache
 TB_SETTINGS_INDEXPATH=/unchained