-
Notifications
You must be signed in to change notification settings - Fork 15
Transaction Generator: Usage Guide
This page describes how to build transaction generator, launch it and check the results.
Currently transaction generator is a part of cardano-cli executable and corresponds to generate-txs subcommand.
$ git clone [email protected]:input-output-hk/cardano-node.git
$ cd cardano-node
$ cabal new-update
$ cabal new-build all
Now cardano-cli executable is available via cabal new-run command.
The purpose of transaction generator is to generate a lot amount of transactions, so we need a cluster from cardano-nodes to submit these transactions to. Currently we can run minimal local cluster (only 3 nodes) using scripts/shelley-testnet.sh from the root of cardano-node repository. Example of command:
$ tmux
$ ./scripts/shelley-testnet.sh
Please note that you have to launch this cluster in the tmux session. By default you will see 4 tmux panes with 3 running cardano-nodes.
To launch a generator use scripts/generator.sh script from the root of cardano-node repository. Example of command:
$ ./scripts/generator.sh --target-node-id 0
where -n 0 is an index of the node we're going to talk with. Since we already have a working cluster (see above) we have to choose some node to submit generated transactions to, by default we take the first node.
There are following CLI arguments we use to set generator up:
-
--target-node-idis an id of node we'll send transactions to. Can be specified multiple times. -
--sig-keyis a path to.key-file (which contains signing key). -
--num-of-txsis a number of benchmarking transactions we'll generate. -
--inputs-per-txis a number of inputs per 1 benchmarking transaction. -
--outputs-per-txis a number of outputs per 1 benchmarking transaction. -
--tx-feeis a fee per transaction (in Lovelaces). -
--tpsis a TPS (transactions per second) rate. -
--add-tx-sizeis an additional size of transaction, usingTxAttributes(in bytes).
Please note that currently we have to provide at least 3 keys (using --sig-key argument): for genesis address, for source address and for recipient address.
To get help info about generator-related arguments use this command:
$ stack exec -- cardano-cli generate-txs --help
or this one:
$ cabal new-run -- cardano-cli generate-txs --help
Transacrion generator generates 3 "kinds" of transactions:
- genesis transaction,
- splitting transactions,
- benchmarking transactions.
Genesis transaction is a very first transaction: we move huge amount of ADA from genesisAddress to sourceAddress. And sourceAddress will be used as a source for all further transactions.
The purpose of splitting transactions is to split huge amount of ADA on sourceAddress to sufficient number of "coins" we will use for further transactions. Technically we split initial amount of money M to N equal parts, as a result we'll have N UTxO entries, and alltogether these entries will contain the same amount M (e.g. 1 initial entry * 1000 ADA -> 10 entries * 100 ADA). These UTxO entries will be used as inputs for benchmarking transactions.
Benchmarking transactions are the main ones because they will be used for a system benchmarking. CLI arguments --num-of-txs and --outputs-per-tx specify total number of benchmarking transactions and number of outputs in each of them. Currently each benchmarking transaction has only 1 input and K outputs (corresponding to --outputs-per-tx value), each output sends fixed amount of money to recipientAddress.
Since we have a local cluster we are sending transactions to, it is possible to launch cardano-explorer-node to check our local blockchain.
It is recommended to use nix to build cardano-explorer-node:
$ git clone [email protected]:input-output-hk/cardano-explorer.git
$ cd cardano-explorer
$ nix-build -A scripts.mainnet.exporter -o launch_mainnet_exporter
Then set CARDANO_NODE_SOCKET_PATH variable:
export CARDANO_NODE_SOCKET_PATH=/path/to/socket
where /path/to/socket is an absolute path to a socket of some node from our local cluster. For example, in my case it is:
export CARDANO_NODE_SOCKET_PATH=/home/shevchenko/cardano-node/socket/node1.socket
It will be used to connect to the node and listen to the blockchain.
Now explorer can be launched by launch_mainnet_exporter script. But please make sure it contains a valid genesis hash value! cardano-explorer-node executable has a CLI argument --genesis-hash, and this value should be equal to genesis hash of the node we are going to connect to. To get this hash use following commands:
$ cd cardano-node
$ cabal new-run -v0 -- cardano-cli --log-config configuration/log-configuration.yaml --real-pbft print-genesis-hash --genesis-json configuration/33873/genesis.json --socket-path socket/node1.socket
You will see something like this:
33873aeaf8a47fefc7c2ea3f72e98a04459e07ec3edfb63c9ca709f540f69503
This value should be provided to cardano-explorer-node via CLI argument --genesis-hash.
In case of success, right after launching you will see something like this:
$ ./launch_mainnet_exporter
/home/shevchenko/Code/cardano-explorer/pgpass
[iohk.cardano.explorer-db-node:Info:1] [2019-09-25 06:56:43.97 UTC] Initial genesis distribution present and correct
[iohk.cardano.explorer-db-node:Info:1] [2019-09-25 06:56:44.00 UTC] Total genesis supply of Ada: 7998949166.000000
[iohk.cardano.explorer-db-node:Info:1] [2019-09-25 06:56:44.00 UTC] Starting node client
[iohk.cardano.explorer-db-node:Info:1] [2019-09-25 06:56:44.00 UTC] localInitiatorNetworkApplication: connecting to node via "/home/shevchenko/Code/cardano-node/socket/node1.socket"
[iohk.cardano.explorer-db-node:Info:39] [2019-09-25 06:56:44.00 UTC] Starting chainSyncClient
[iohk.cardano.explorer-db-node:Info:39] [2019-09-25 06:56:44.01 UTC] Explorer DB tip is at slot 131826
Since Cardano Explorer uses database to store an information, it is assumed that you have installed and configured PostgreSQL. You can find database schema here.
To drop Explorer's database completely:
postgres@linux:~$ dropdb cexplorer -U cexplorer
To create Explorer's database from scratch:
postgres@linux:~$ createdb cexplorer -U cexplorer
To work with this database:
postgres@linux:~$ psql cexplorer -U cexplorer
Current structure of this database is this one:
cexplorer=# \dt
List of relations
Schema | Name | Type | Owner
--------+----------------+-------+-----------
public | block | table | cexplorer
public | meta | table | cexplorer
public | schema_version | table | cexplorer
public | slot_leader | table | cexplorer
public | tx | table | cexplorer
public | tx_in | table | cexplorer
public | tx_out | table | cexplorer
(7 rows)
To check blocks:
cexplorer=# SELECT * FROM block;
As you can see, hash of the first block is equal to --genesis-hash (see above).
If you stuck with an error psql: FATAL: Peer authentication failed for user "cexplorer", you probably need to change PostgreSQL configuration files. Examples:
$ cat /etc/postgresql/11/main/pg_hba.conf
local all all ident map=explorer-users
local all shevchenko trust
local all all ident
host all all 127.0.0.1/32 md5
host all all ::1/128 md5
$ cat /etc/postgresql/11/main/pg_ident.conf
explorer-users /root cexplorer
explorer-users /postgres cexplorer
explorer-users /shevchenko cexplorer
$ cat /etc/postgresql/11/main/postgresql.conf
data_directory = '/var/lib/postgresql/11/main'
hba_file = '/etc/postgresql/11/main/pg_hba.conf'
ident_file = '/etc/postgresql/11/main/pg_ident.conf'
log_destination = 'stderr'
listen_addresses = 'localhost'
port = 5432
Running transaction generator on the local cluster (which is launched on the one computer) is the simplest possible way, but in this case we'll get "an ideal values" without any network latency. Real-life measurements can be obtained only from distributed cluster: nodes are working on the different computers in the different world regions.
There is no describing of this procedure in this document: it is assumed that such a cluster already exists and works, it is responsibility of DevOps team to do it.
As an example there is the simplest possible (physically distributed) cluster from 3 nodes working in three world regions.
Since the servers (which host running nodes) exist in AWS environment, it is possible to get an access to them from staging server. Please make sure you have an access to it.
Login to cardano-deployer server as staging:
$ ssh [email protected]
Last login: Thu Oct 3 10:24:15 2019 from 37.252.89.158
staging@cardano-deployer:~$
Currently, there's a directory dshevchenko in the root of cardano-deployer server with the following structure:
dshevchenko/
cardano-node
... Code of the node, for cardano-cli
cert/
delegation-cert.000.json
delegation-cert.001.json
delegation-cert.002.json
genesis.json
keys/
delegate-keys.000.key
delegate-keys.001.key
delegate-keys.002.key
log-config/
log-config-0.yaml
log-config-1.yaml
log-config-2.yaml
topology/
topology-node-1.json
topology-node-2.json
topology-node-3.json
cardano-node will be used to build cardano-cli, because transaction generator is a part of it.
Now it is possible to login to server which hosts some of nodes. For example, there are 3 nodes in the current cluster: a1, b1, c1. To login to the server which hosts a1 node use this command:
staging@cardano-deployer:~$ nixops ssh -d shelley-aws a1
[root@a1:~]#
Let's check if the node is running and what configuration it is running with:
[root@a1:~]# ps aux | grep cardano-node
cardano+ 23120 0.2 16.2 1074462292 80480 ? Ssl Nov07 3:19 /nix/store/aac92zm4jqy76vh626xzdfni9ndcd88a-cardano-node-exes/bin/cardano-node --genesis-file /nix/store/bpnvc57j8v5s3imc2yi7iy6050qnggbd-genesis.json --genesis-hash d3931233c9b1e22d18b94bed9713a929186ca0ef7e7a187fc2ab612a60a8e69a --log-config /nix/store/b4skln4dbvx0n4yi25khd3xl6mgzzm4i-log-config.json --database-path /var/lib/cardano-node/db-testnet --socket-dir /run/cardano-node --topology /nix/store/33mmhh97lfx7d1x70djg1ba46s6k25xh-topology.yaml --real-pbft --node-id 0 --host-addr 172.31.35.144 --port 3001 --pbft-signature-threshold 0.9 --signing-key /nix/store/y8k4ia7l4rk2caz3xlpby573a3bv3xy6-delegate-keys.000.key --delegation-certificate /nix/store/9yisxnpm6a4pva3gjv7w515nbsbv4y21-delegation-cert.000.json
It is possible to read a log produced by a node:
[root@a1:~]# journalctl -f -u cardano-node
Drop the -f flag to get the full history. You can also add --since 2019-09-01 and --until 2019-10-01 to restrict it to a time-range, for example:
[root@a1:~]# journalctl -u cardano-node --since 2019-10-01 --until 2019-10-03 > sample.txt
and then read this file:
[root@a1:~]# vi sample.txt
Since transaction generator should submit transactions to the cluster, it must connect to some "target" node to submit transactions to. Currently the only way to do it is to launch transaction generator right on the server where "target" node is running.
IMPORTANT: this behavior is not the best one, because in this case we'll get unfair results. In the future cluster's configuration will be changed and transaction generator will be launched on another server.
To launch transaction generator we have to use parameters corresponding to the running node (and we already saw these parameters from ps aux | grep cardano-node command), so:
cabal new-run -- exe:cardano-cli --log-config ../log-config/log-config-0.yaml --signing-key ../keys/delegate-keys.000.key --delegation-certificate ../cert/delegation-cert.000.json --real-pbft --genesis-file ../genesis.json --genesis-hash d3931233c9b1e22d18b94bed9713a929186ca0ef7e7a187fc2ab612a60a8e69a generate-txs --topology ../topology/topology-node-1.json --num-of-txs 2 --inputs-per-tx 1 --outputs-per-tx 1 --tx-fee 1000000 --tps 10 --add-tx-size 100 --sig-key ../keys/delegate-keys.000.key --sig-key ../keys/delegate-keys.001.key --sig-key ../keys/delegate-keys.002.key --target-node-id 0 --node-id 0
IMPORTANT: Please note that currently --sig-keys are taken from the root of current server. This is because these keys were copied manually from cardano-deployer server. It is temporary solution!
Go to cluster's subdirectory:
staging@cardano-deployer:~$ cd david/cardano-ops
Then:
staging@cardano-deployer:~$ nix-shell
Now update cardano-nodes using particular branch:
staging@cardano-deployer:~$ niv update cardano-node -b master
where master is the name of branch.
Before deploy please do this export:
export NIX_PATH="nixpkgs=$(nix eval '(import ./nix {}).path')"
Finally deploy it:
staging@cardano-deployer:~$ nixops deploy -d shelley-aws
In case of success you will see something like this:
...
shelley-aws> deployment finished successfully
Please note that servers are working on NixOS, so all the paths to executables are immutable by definition. Make sure that after re-deployment you check the new /nix/store-path to executables cardano-node and cardano-cli.
To copy delegate key to particular server (for example, a1), use this command
staging@cardano-deployer:~$ nixops scp -d shelley-aws --to a1 dshevchenko/keys/delegate-keys.000.key /root
To clean up nodes' databases, use these commands. First of all, stop the nodes:
staging@cardano-deployer:~$ nixops ssh-for-each -d shelley-aws systemctl stop cardano-node
Then wipe the state:
staging@cardano-deployer:~$ nixops ssh-for-each -d shelley-aws -- rm -rf /var/lib/cardano-node/db-testnet-0
Now start nodes again:
staging@cardano-deployer:~$ nixops ssh-for-each -d shelley-aws systemctl start cardano-node
This is a temporary draft, devops will change it soon.
$ stack build
$ cat launch_mainnet_exporter.sh
#!/bin/bash
mkdir -p log-dir
# work around a bug in cardano-node
mkdir -p configuration
cp -f /home/shevchenko/Code/cardano-node/configuration/log-configuration.yaml configuration/log-configuration.yaml
# end of work-around
stack exec -- cardano-explorer-node --log-config /home/shevchenko/Code/cardano-node/configuration/log-configuration.yaml \
--genesis-hash db60f5877ac3032b401e53bf5c91d4fe84b6cb993582ffcf4c10828407127386 \
--genesis-file /home/shevchenko/Code/cardano-node/configuration/b0109/genesis.json \
--socket-path /home/shevchenko/Code/cardano-node/socket/node1.socket \
--schema-dir /home/shevchenko/Code/cardano-explorer/schema
$ cat pgpass
localhost:5432:cexplorer:cexplorer:cexplorer
$ PGUSER=cexplorer PGHOST=localhost PGPORT=5432 PGPASSFILE=pgpass ./launch_mainnet_exporter.sh