Skip to content

Commit

Permalink
Add instances to Cli and enable remote vs local run configurations (#26)
Browse files Browse the repository at this point in the history
* add instance flag to init command

* update init and run cmds to use instance flag

* moved some file path creation to use constants

* init now creates local and remote config, run command only uses local config

* added remote and local flags to run command

* added different local vs remote binary downloads to allow for different versions for each run config

* updated remote .env and added remaining binary download versions

* update just build command

* added instance flag to clean command

* remove leading newlines

* panic on failed instance name validation

* update clean command help messages

* remove unneeded default prints

* remove cruft

* alphabetize constants

* move println to logging in init

* runAll renamed to runRun

* moved CreateDir to helpers

* flag updates

* fixed confusing cleanCmd name

* add panic to all fatal logs

* fix incorrect Infof log

* readme updates

* readme updates

* renamed CreateDir to CreateDirOfPanic

* move run configuration logic to simple switch statement

* cleanup, instance flag usage updates
  • Loading branch information
sambukowski authored Apr 3, 2024
1 parent 786e51b commit 7fc6e07
Show file tree
Hide file tree
Showing 13 changed files with 572 additions and 123 deletions.
30 changes: 28 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ your IDE, but for VS Code you can open your settings and add:
```json
{
"gopls": {
"buildFlags": ["-tags=darwin arm64 amd64 linux"]
"buildFlags": ["-tags=darwin arm64 amd64 linux"]
}
}
```
Expand All @@ -85,7 +85,7 @@ cobra-cli add <new-command>
### Available Commands

| Command | Description |
|----------------------------|-------------------------------------------------------------------------------------|
| -------------------------- | ----------------------------------------------------------------------------------- |
| `version` | Prints the cli version. |
| `help` | Show help. |
| `dev` | Root command for cli development functionality. |
Expand All @@ -95,3 +95,29 @@ cobra-cli add <new-command>
| `dev clean all` | Deletes the local data, downloaded binaries, and config files for the Astria stack. |
| `sequencer create-account` | Generate an account for the Sequencer. |
| `sequencer get-balance` | Get the balance of an account on the Sequencer. |

### Instances

The `dev init`, `dev run`, and `dev clean` commands all have an optional `--instance` flag. The value of this flag will be used as the directory name where the rollup data will be stored. Now you can run many rollups while keeping their configs and state data separate. If no value is provided, `default` is used, i.e. `~/.astria/default`.

For example, if you run:

```bash
astria-go dev init
astria-go dev init --instance hello
astria-go dev init --instance world
```

You will see the following in the `~/.astria` directory:

```bash
.astria/
default/
hello/
world/
```

Each of these directories will contain configs and binaries for
running the Astria stack. You can then update the `.env` files in the
`~/.astria/<instance name>/config-local/` or `~/.astria/<instance
name>/config-remote/` directories to suit your needs.
7 changes: 6 additions & 1 deletion cmd/devtools/binaries_config_darwin_amd64.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,14 @@ type Binary struct {
Url string
}

var Binaries = []Binary{
var LocalBinaries = []Binary{
{"cometbft", "https://github.com/cometbft/cometbft/releases/download/v0.37.4/cometbft_0.37.4_darwin_amd64.tar.gz"},
{"astria-sequencer", "https://github.com/astriaorg/astria/releases/download/sequencer-v0.9.0/astria-sequencer-x86_64-apple-darwin.tar.gz"},
{"astria-composer", "https://github.com/astriaorg/astria/releases/download/composer-v0.4.0/astria-composer-x86_64-apple-darwin.tar.gz"},
{"astria-conductor", "https://github.com/astriaorg/astria/releases/download/conductor-v0.12.0/astria-conductor-x86_64-apple-darwin.tar.gz"},
}

var RemoteBinaries = []Binary{
{"astria-composer", "https://github.com/astriaorg/astria/releases/download/composer-v0.3.1/astria-composer-x86_64-apple-darwin.tar.gz"},
{"astria-conductor", "https://github.com/astriaorg/astria/releases/download/conductor-v0.11.1/astria-conductor-x86_64-apple-darwin.tar.gz"},
}
7 changes: 6 additions & 1 deletion cmd/devtools/binaries_config_darwin_arm64.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,14 @@ type Binary struct {
Url string
}

var Binaries = []Binary{
var LocalBinaries = []Binary{
{"cometbft", "https://github.com/cometbft/cometbft/releases/download/v0.37.4/cometbft_0.37.4_darwin_arm64.tar.gz"},
{"astria-sequencer", "https://github.com/astriaorg/astria/releases/download/sequencer-v0.9.0/astria-sequencer-aarch64-apple-darwin.tar.gz"},
{"astria-composer", "https://github.com/astriaorg/astria/releases/download/composer-v0.4.0/astria-composer-aarch64-apple-darwin.tar.gz"},
{"astria-conductor", "https://github.com/astriaorg/astria/releases/download/conductor-v0.12.0/astria-conductor-aarch64-apple-darwin.tar.gz"},
}

var RemoteBinaries = []Binary{
{"astria-composer", "https://github.com/astriaorg/astria/releases/download/composer-v0.3.1/astria-composer-aarch64-apple-darwin.tar.gz"},
{"astria-conductor", "https://github.com/astriaorg/astria/releases/download/conductor-v0.11.1/astria-conductor-aarch64-apple-darwin.tar.gz"},
}
7 changes: 6 additions & 1 deletion cmd/devtools/binaries_config_linux_amd64.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,14 @@ type Binary struct {
Url string
}

var Binaries = []Binary{
var LocalBinaries = []Binary{
{"cometbft", "https://github.com/cometbft/cometbft/releases/download/v0.37.4/cometbft_0.37.4_linux_amd64.tar.gz"},
{"astria-sequencer", "https://github.com/astriaorg/astria/releases/download/sequencer-v0.9.0/astria-sequencer-x86_64-unknown-linux-gnu.tar.gz"},
{"astria-composer", "https://github.com/astriaorg/astria/releases/download/composer-v0.4.0/astria-composer-x86_64-unknown-linux-gnu.tar.gz"},
{"astria-conductor", "https://github.com/astriaorg/astria/releases/download/conductor-v0.12.0/astria-conductor-x86_64-unknown-linux-gnu.tar.gz"},
}

var RemoteBinaries = []Binary{
{"astria-composer", "https://github.com/astriaorg/astria/releases/download/composer-v0.3.1/astria-composer-x86_64-unknown-linux-gnu.tar.gz"},
{"astria-conductor", "https://github.com/astriaorg/astria/releases/download/conductor-v0.11.1/astria-conductor-x86_64-unknown-linux-gnu.tar.gz"},
}
35 changes: 18 additions & 17 deletions cmd/devtools/clean.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,37 +12,37 @@ import (
// cleanCmd represents the clean command
var cleanCmd = &cobra.Command{
Use: "clean",
Short: "Cleans the local development environment data.",
Long: `Cleans the local development environment data. Does not remove the binaries or the config files.`,
Run: func(cmd *cobra.Command, args []string) {
runClean()
},
Short: "Cleans the local development instance data.",
Long: `Cleans the local development instance data. Does not remove the binaries or the config files.`,
Run: runClean,
}

func runClean() {
func runClean(cmd *cobra.Command, args []string) {
// Get the instance name from the -i flag or use the default
instance := cmd.Flag("instance").Value.String()
IsInstanceNameValidOrPanic(instance)

homePath, err := os.UserHomeDir()
if err != nil {
log.WithError(err).Error("Error getting home dir")
panic(err)
}
defaultDataDir := filepath.Join(homePath, ".astria/data")
defaultDir := filepath.Join(homePath, ".astria")
instanceDir := filepath.Join(defaultDir, instance)
dataDir := filepath.Join(instanceDir, DataDirName)

cleanCmd := exec.Command("rm", "-rf", defaultDataDir)
if err := cleanCmd.Run(); err != nil {
rmCmd := exec.Command("rm", "-rf", dataDir)
if err := rmCmd.Run(); err != nil {
log.WithError(err).Error("Error running rm")
panic(err)
}

err = os.MkdirAll(defaultDataDir, 0755) // Read & execute by everyone, write by owner.
if err != nil {
log.WithError(err).Error("Error creating data directory")
panic(err)
}
CreateDirOrPanic(dataDir)
}

var allCmd = &cobra.Command{
Use: "all",
Short: "Clean all local data including binaries and config files.",
Short: "Delete everything in the ~/.astria directory.",
Long: "Clean all local data including binaries and config files. `dev init` will need to be run again to get the binaries and config files back.",
Run: func(cmd *cobra.Command, args []string) {
runCleanAll()
Expand All @@ -59,8 +59,8 @@ func runCleanAll() {
// TODO: allow for configuration of this directory
defaultDataDir := filepath.Join(homePath, ".astria")

cleanCmd := exec.Command("rm", "-rf", defaultDataDir)
if err := cleanCmd.Run(); err != nil {
rmCmd := exec.Command("rm", "-rf", defaultDataDir)
if err := rmCmd.Run(); err != nil {
log.WithError(err).Error("Error running rm")
panic(err)
}
Expand All @@ -69,6 +69,7 @@ func runCleanAll() {
func init() {
// top level command
devCmd.AddCommand(cleanCmd)
cleanCmd.Flags().StringP("instance", "i", DefaultInstanceName, "Choose the instance that will be cleaned.")

// subcommands
cleanCmd.AddCommand(allCmd)
Expand Down
4 changes: 1 addition & 3 deletions cmd/devtools/config/local.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -90,9 +90,7 @@ ASTRIA_CONDUCTOR_METRICS_HTTP_LISTENER_ADDR="127.0.0.1:9000"
ASTRIA_SEQUENCER_LISTEN_ADDR="127.0.0.1:26658"

# Path to rocksdb
# ASTRIA_SEQUENCER_DB_FILEPATH="/tmp/astria_db"
ASTRIA_SEQUENCER_DB_FILEPATH="~/.astria/data/astria_sequencer_db"
# ASTRIA_SEQUENCER_DB_FILEPATH="../data/astria_sequencer_db"
ASTRIA_SEQUENCER_DB_FILEPATH="~/data/astria_sequencer_db"

# Set to true to enable the mint component
# Only used if the "mint" feature is enabled
Expand Down
144 changes: 144 additions & 0 deletions cmd/devtools/config/remote.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
###########################
######## conductor ########
###########################
# Configuration options for the Astria Conductor

# the sequencer block height that the rollup's first block was in
ASTRIA_CONDUCTOR_INITIAL_SEQUENCER_BLOCK_HEIGHT=2892100

# The chain id of the chain that is being read from the astria-sequencer or the
# Data Availability layer
ASTRIA_CONDUCTOR_CHAIN_ID="clilocalastria"

# Execution RPC URL
ASTRIA_CONDUCTOR_EXECUTION_RPC_URL="http://127.0.0.1:50051"

# The URL to a fully trusted CometBFT/Sequencer to issue cometbft RPCs. Example
# RPCs are subscribing to new blocks, fetching blocks at a given level, or
# retrieving validators.
# 127.0.0.1:26657 is the default socket address in comebft's `rpc.laddr` setting.
ASTRIA_CONDUCTOR_SEQUENCER_URL="wss://rpc.sequencer.dusk-3.devnet.astria.org/websocket"

# Set the origin where blocks are pulled from and sent to the execution layer
# Setting options:
# - "SoftOnly" -> blocks are only pulled from the sequencer
# - "FirmOnly" -> blocks are only pulled from DA
# - "SoftAndFirm" -> blocks are pulled from both the sequencer and DA
ASTRIA_CONDUCTOR_EXECUTION_COMMIT_LEVEL="SoftOnly"

# Log Level
ASTRIA_CONDUCTOR_LOG="astria_conductor=info"

# The URL of the celestia node used to subscribe to new headers and fetch
# blocks from. Note that this string need not be a fully qualified URL and
# can miss the scheme part. The scheme part, if present, will be replaced by
# `ws:` or `http:` for websocket subscriptions and http GET requests,
# respectively. The following are examples of accepted strings (26658 is
# the default port of a celestia node to listen for RPCs).
# - 127.0.0.1:26658
# - ws://127.0.0.1:26658
# - http://127.0.0.1:26658
ASTRIA_CONDUCTOR_CELESTIA_NODE_URL="127.0.0.1:26658"

# set to true to enable op-stack deposit derivations
ASTRIA_CONDUCTOR_ENABLE_OPTIMISM=false

# Websocket URL of Ethereum L1 node.
# Only used if `ASTRIA_CONDUCTOR_ENABLE_OPTIMISM=true`.
ASTRIA_CONDUCTOR_ETHEREUM_L1_URL="ws://127.0.0.1:8546"

# The bearer token to retrieve sequencer blocks as blobs from Celestia.
# The token is obtained by running `celestia bridge auth <permissions>`
# on the host running the celestia node.
ASTRIA_CONDUCTOR_CELESTIA_BEARER_TOKEN="<JWT Bearer token>"

# If true disables writing to the opentelemetry OTLP endpoint.
ASTRIA_CONDUCTOR_NO_OTEL=true

# If true disables tty detection and forces writing telemetry to stdout.
# If false span data is written to stdout only if it is connected to a tty.
ASTRIA_CONDUCTOR_FORCE_STDOUT=true

# If true uses an exceedingly pretty human readable format to write to stdout.
# If false uses JSON formatted OTEL traces.
# This does nothing unless stdout is connected to a tty or
# `ASTRIA_CONDUCTOR_FORCE_STDOUT` is set to `true`.
ASTRIA_CONDUCTOR_PRETTY_PRINT=true

# If set to any non-empty value removes ANSI escape characters from the pretty
# printed output. Note that this does nothing unless `ASTRIA_CONDUCTOR_PRETTY_PRINT`
# is set to `true`.
NO_COLOR=

# Contract address of the OptimismPortal contract on L1.
# Only used if `ASTRIA_CONDUCTOR_ENABLE_OPTIMISM=true`.
ASTRIA_CONDUCTOR_OPTIMISM_PORTAL_CONTRACT_ADDRESS=""

# The block height of the Ethereum L1 chain that the
# OptimismPortal contract was deployed at.
# Only used if `ASTRIA_CONDUCTOR_ENABLE_OPTIMISM=true`.
ASTRIA_CONDUCTOR_INITIAL_ETHEREUM_L1_BLOCK_HEIGHT=1

# Set to true to enable prometheus metrics.
ASTRIA_CONDUCTOR_NO_METRICS=true

# The address at which the prometheus HTTP listener will bind if enabled.
ASTRIA_CONDUCTOR_METRICS_HTTP_LISTENER_ADDR="127.0.0.1:9000"


##########################
######## composer ########
##########################
# Configuration options of Astria Composer.

# Address of the RPC server for the sequencer chain
ASTRIA_COMPOSER_SEQUENCER_URL="https://rpc.sequencer.dusk-3.devnet.astria.org"

# A list of execution <chain_id_1>::<rpc_server_1>,<chain_id_2>::<rpc_server_2>.
# Chain IDs are not case sensitive. If an ID is repeated, the last list item is used.
ASTRIA_COMPOSER_ROLLUPS="clilocalastria::ws://127.0.0.1:8546"

# Log level. One of debug, info, warn, or error
ASTRIA_COMPOSER_LOG="astria_composer=info"

# If true disables writing to the opentelemetry OTLP endpoint.
ASTRIA_COMPOSER_NO_OTEL=true

# If true disables tty detection and forces writing telemetry to stdout.
# If false span data is written to stdout only if it is connected to a tty.
ASTRIA_COMPOSER_FORCE_STDOUT=true

# If true uses an exceedingly pretty human readable format to write to stdout.
# If false uses JSON formatted OTEL traces.
# This does nothing unless stdout is connected to a tty or
# `ASTRIA_COMPOSER_FORCE_STDOUT` is set to `true`.
ASTRIA_COMPOSER_PRETTY_PRINT=true

# If set to any non-empty value removes ANSI escape characters from the pretty
# printed output. Note that this does nothing unless `ASTRIA_COMPOSER_PRETTY_PRINT`
# is set to `true`.
NO_COLOR=

# Address of the API server
ASTRIA_COMPOSER_API_LISTEN_ADDR="0.0.0.0:0"

# Private key for the sequencer account used for signing transactions
# Must be a hex-encoded 32-byte array (64-character hex string)
ASTRIA_COMPOSER_PRIVATE_KEY="2bd806c97f0e00af1a1fc3328fa763a9269723c8db8fac4f93af71db186d6e90"

# Block time in milliseconds, used to force submitting of finished bundles.
# Should match the sequencer node configuration for 'timeout_commit', as
# specified in https://docs.tendermint.com/v0.34/tendermint-core/configuration.html
ASTRIA_COMPOSER_MAX_SUBMIT_INTERVAL_MS=2000

# Max bytes to encode into a single sequencer `SignedTransaction`, not including signature,
# public key, nonce. This is the sum of the sizes of all the `SequenceAction`s. Should be
# set below the sequencer's max block size to allow space for encoding, signature, public
# key and nonce bytes
ASTRIA_COMPOSER_MAX_BYTES_PER_BUNDLE=200000

# Set to true to enable prometheus metrics.
ASTRIA_COMPOSER_NO_METRICS=true

# The address at which the prometheus HTTP listener will bind if enabled.
ASTRIA_COMPOSER_METRICS_HTTP_LISTENER_ADDR="127.0.0.1:9000"
10 changes: 10 additions & 0 deletions cmd/devtools/constants.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
package devtools

const (
DataDirName = "data"
DefaultInstanceName = "default"
LocalBinariesDirName = "bin-local"
LocalConfigDirName = "config-local"
RemoteBinariesDirName = "bin-remote"
RemoteConfigDirName = "config-remote"
)
36 changes: 36 additions & 0 deletions cmd/devtools/helpers.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
package devtools

import (
"fmt"
"os"
"regexp"

log "github.com/sirupsen/logrus"
)

func IsInstanceNameValidOrPanic(instance string) {
re, err := regexp.Compile(`^[a-z]+[a-z0-9]*(-[a-z0-9]+)*$`)
if err != nil {
log.WithError(err).Error("Error compiling regex")
panic(err)
}
if !re.MatchString(instance) {
log.Errorf("Invalid instance name: %s", instance)
err := fmt.Errorf(`
Invalid instance name: '%s'. Instance names must be lowercase, alphanumeric,
and may contain dashes. It can't begin or end with a dash. No repeating dashes.
`, instance)
panic(err)
}
}

// CreateDirOrPanic creates a directory with the given name with 0755 permissions.
// If the directory can't be created, it will panic.
func CreateDirOrPanic(dirName string) {
err := os.MkdirAll(dirName, 0755)
if err != nil {
log.WithError(err).Error("Error creating data directory")
panic(err)
}

}
Loading

0 comments on commit 7fc6e07

Please sign in to comment.