Zcash Docker Development Environment

🛑 This is a work-in-progress and may not function reliably. It is certainly not intended to be used in production. 🛑

This repository contains a Dockerfile that will help you get started building Zcash's mobile app wallet stack. The projects' code is kept on your host filesystem and shared into the docker container, so you can keep using the editors and tools you're familiar with, and only use the container to run builds. You can also run the zcashd and lightwalletd services within the container.

The development environment currently supports building:

• zcashd
• librustzcash
• lightwalletd
• zcash-android-wallet-sdk (preview branch)
• zcash-android-wallet-poc

Quick Start

Requirements: Docker, KVM and a VNC client on the host (for the Android emulator), 30+GB free disk space. You'll need enough memory to run zcashd and the emulator, I suggest a minimum of 8GB.

OSX Support: The emulator will be insanely slow on OSX (to the point where it's not even usable) because there, it has to emulate the device's CPU.

First, checkout this repository:

git checkout <this-repository's-url>
cd zdev-docker

Next, build the Docker image. This will create an image called zdev which has all of the build dependencies pre-installed:

docker build -t zdev docker

Now, clone all of the projects you want to work on into the mount/ directory:

cd mount
git clone [email protected]:zcash/zcash.git
git clone [email protected]:zcash-hackworks/lightwalletd.git
git clone [email protected]:zcash/librustzcash.git
git clone [email protected]:zcash/zcash-android-wallet-sdk.git --branch preview
git clone [email protected]:zcash/zcash-android-wallet-poc.git
cd ..

You can now launch a new container instance of zdev and have access to your clones of zcash, lightwalletd, etc. from within:

docker run -it --mount type=bind,source="$(pwd)"/mount,target=/mount -p 5901:5901 --privileged zdev

The ./mount directory will be shared between the host and the container, accessible in the container at /mount. The changes you make to files inside /mount in the container will change the files in ./mount. The -p 5901:5901 --privileged arguments are optional; they are required for running and connecting to the Android emulator inside the container.

The previous command opened a shell inside the container. You can run build commands, for example:

cd /mount/zcash
./zcutil/build.sh -j$(nproc)
./qa/zcash/full-test-suite.py

See the individual projects' documentation for build instructions.

To open another shell into the running container, run docker container ls, copy the container ID, and then run docker exec -it <container ID> bash. You can start and stop the container with docker start and docker stop.

To update the Ubuntu packages in the zdev image or make changes to which dependencies are installed, edit the docker/install-build-dependencies.sh script and then re-build the zdev image using the same command as above. The changes will take effect in the next new container instance you launch. You can of course also just make changes directly to the container instead of updating the image every time.

Build Cheat Sheet

Note: All of the builds are independent for now, i.e. the build output of librustzcash doesn't get used as an input to the zcash-android-wallet-sdk build.

zcashd

cd /mount/zcash
./zcutil/build.sh -j$(nproc)

librustzcash

cd /mount/librustzcash
cargo build --release

lightwalletd

cd /mount/lightwalletd
go build -o server cmd/server/main.go
go build -o ingest cmd/ingest/main.go

zcash-android-wallet-sdk

cd /mount/zcash-android-wallet-sdk
./gradlew clean assembleZcashtestnetRelease

You can also build the memo sample app:

cd samples/memo
./gradlew :sdk:assemblezcashtestnetdebug :app:assembledebug

zcash-android-wallet-poc

First, generate a google-services.json file from Google Firebase and put it in zcash-android-wallet-poc/zcash-android-wallet-app/app/google-services.json. The package_name inside the file needs to be cash.z.android.wallet.testnet.

cd /mount/zcash-android-wallet-poc
cd zcash-android-wallet-app
./gradlew clean assembleZcashtestnetDebug

Running the Stack

This section assumes you've successfully run all of the builds in the Build Cheat Sheet above. We will set up:

1. A zcashd node connected to testnet.
2. A lightwalletd ingestor which receives blocks from the zcashd node.
3. A lightwalletd server which serves the blocks parsed by the ingestor to light clients.
4. A patched build of zcash-android-wallet-poc (the demo Android app), running in an emulator, connected to the lightwalletd server.

Server-Side

Note: These instructions will need to be updated once lightwalletd no longer uses ZMQ to interact with zcashd

First, start the lightwalletd ingestor:

cd /mount/lightwalletd
go run cmd/ingest/main.go -db-path database.db -log-file ingestor-log.txt

This will listen on port 28332 for a ZMQ connection from zcashd. Now put the following contents in /root/.zcash/zcash.conf:

testnet=1
addnode=testnet.z.cash
zmqpubcheckedblock=tcp://127.0.0.1:28332
rpcuser=yourusername
rpcpassword=yourpassword

You're advised to change yourusername and yourpassword to something random. This configures zcashd to sync with the testnet and send blocks to lightwalletd on localhost port 28332. Now build and run zcashd:

cd /mount/zcash
./zcutil/build.sh -j$(nproc)
./zcutil/fetch-params.sh
./src/zcashd

If you check the contents of /mount/lighwalletd/ingestor-log.txt, you should see that it is receiving blocks as the zcashd node syncs.

If you get an error message like block -1: UNIQUE constraint failed in the log file, stop the ingestor, delete database.db, and then re-start it (the error occurs when the ingestor is started before zcashd).

To serve these blocks to light clients, start the server:

cd /mount/lightwalletd
go run cmd/server/main.go -conf-file /root/.zcash/zcash.conf -db-path database.db -log-file server-log.txt

Note: If this command fails and there's an error message about database locks in server-log.txt you need to stop the ingestor, start the server, then re-start the ingestor.

After a while, zcashd will finish downloading all of the blocks, and as it does so the ingestor will ingest them and the server will be able to serve them. Once this is done, the mobile wallet will be able to connect and sync.

Client-Side

First, we'll need to patch the demo app to be able to connect to our lightwalletd server. Apply this patch:

diff --git a/zcash-android-wallet-app/app/src/main/java/cash/z/android/wallet/sample/SampleConfig.kt b/zcash-android-wallet-app/app/src/main/java/cash/z/android/wallet/sample/SampleConfig.kt
index 249da5c..d45c046 100644
--- a/zcash-android-wallet-app/app/src/main/java/cash/z/android/wallet/sample/SampleConfig.kt
+++ b/zcash-android-wallet-app/app/src/main/java/cash/z/android/wallet/sample/SampleConfig.kt
@@ -94,6 +94,7 @@ object MyWallet : WalletConfig {
 
 enum class Servers(val host: String, val displayName: String) {
     LOCALHOST("10.0.0.191", "Localhost"),
+    LOCALHOST2("10.0.2.2", "Localhost2"),
     //    WLAN("10.0.0.26"),
     WLAN1("10.0.2.24", "WLAN Conference"),
     WLAN2("192.168.1.235", "WLAN Office"),
diff --git a/zcash-android-wallet-app/app/src/main/res/values/strings.xml b/zcash-android-wallet-app/app/src/main/res/values/strings.xml
index a57ee92..560d204 100644
--- a/zcash-android-wallet-app/app/src/main/res/values/strings.xml
+++ b/zcash-android-wallet-app/app/src/main/res/values/strings.xml
@@ -108,6 +108,7 @@
         <item>WLAN Office</item>
         <item>WLAN Conference</item>
         <item>Localhost</item>
+        <item>Localhost2</item>
         <item>Custom...</item>
     </string-array>
 </resources>

Now rebuild the APK:

./gradlew clean assembleZcashtestnetDebug

To run the app in the Android emulator, first start a VNC server for the emulator to use to serve its display:

vncserver :1 -geometry 600x800 -depth 24

This will output a message along the lines of:

...
New 'X' desktop is 3fe5a1e534df:1
...

Copy the 3fe...4df part of your output and use it to replace the fff...fff in the following command to start the emulator:

DISPLAY=ffffffffffff:1 $ANDROID_HOME/emulator/emulator -avd zemu -noaudio -no-boot-anim -gpu off -qemu

You can now install the APK in the emulator:

cd /mount/zcash-android-wallet-poc
$ANDROID_HOME/platform-tools/adb install ./zcash-android-wallet-app/app/build/outputs/apk/zcashtestnet/debug/app-zcashtestnet-debug.apk

From your host system, connect to the emulator with any VNC client, e.g...

vncviewer localhost:5901

You should now be able to control the emulated device and launch the app. To connect to your lightwalletd, go in the app's settings and choose Localhost2, click the checkmark, then re-open the app.

To update the app in the emulator after you rebuild it with changes, simply uninstall it as follows and then reinstall it using the same command above. Replace whateveritis.

$ANDROID_HOME/platform-tools/adb uninstall cash.z.whateveritis

To view the adb logs:

$ANDROID_HOME/platform-tools/adb logcat

TODOs

• Fix the VNC resolution
• How to do a reindex to fix the database?
• Put .zcash-mainnet and .zcash-testnet fully loaded into the image.
• Make the builds use the output of dependencies' builds
• Verify the "running the stack" instructions are actually correct.
• Security review (are tools/code being downloaded safely? what are the consequences of using a privileged docker? etc)