Skip to content

Commit

Permalink
Merge pull request #270 from spacemeshos/5089-update-postcli-for-comp…
Browse files Browse the repository at this point in the history 
…atibility

Update postcli for compatibility with multismeshing
  • Loading branch information
@fasmat
fasmat authored Feb 27, 2024
2 parents 3cd36cc + 61f0fbc commit 83f3fde
Show file tree
Hide file tree
Showing 11 changed files with 202 additions and 116 deletions.
2 changes: 1 addition & 1 deletion Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ install: get-postrs-lib
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s v1.54.2
go install gotest.tools/[email protected]
go install honnef.co/go/tools/cmd/[email protected]
go install go.uber.org/mock/mockgen@v0.3.0
go install go.uber.org/mock/mockgen@v0.4.0
.PHONY: install

tidy:
Expand Down
43 changes: 26 additions & 17 deletions cmd/postcli/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -81,28 +81,37 @@ apt install nvidia-opencl-icd
Example

```bash
./postcli -provider=2 -id=c230c51669d1fcd35860131e438e234726b2bd5f9adbbd91bd88a718e7e98ecb \
./postcli -provider=2 -numUnits=4 -id=c230c51669d1fcd35860131e438e234726b2bd5f9adbbd91bd88a718e7e98ecb \
-commitmentAtxId=c230c51669d1fcd35860131e438e234726b2bd5f9adbbd91bd88a718e7e98ecb -genproof
```

### Remarks

* Both `-id` and `-commitmentAtxId` are needed to generate the PoST data.
* If `-id` isn't provided a new identity will be auto-generated. Its private key will be stored in `key.bin` in
`-datadir` with the PoST data. This file then **must** to be copied/moved with the PoST data to run a node with this
generated identity.
**NOTE:** The generated PoST data is ONLY valid for this identity!
If a public key is provided with the `-id` flag, the `key.bin` file will be NOT created. Make sure that the key file
that belongs to the identity provided to `postcli` is available in the PoST directory **before** running a node with it.
* `-commitmentAtxId`: it is recommended to look up the highest ATX by querying it from a synced node with
`grpcurl -plaintext -d '' 0.0.0.0:9093 spacemesh.v1.ActivationService.Highest | jq -r '.atx.id.id' | base64 -d | xxd -p -c 64`.
The node can be operated in "non-smeshing" mode during synchronization and when querying the highest ATX.
* Both `-numUnits` and `-commitmentAtxId` are needed to generate PoST data.
* an already started initialization can be continued without providing these flags.
* `-commitmentAtxId`: it is recommended to look up the highest ATX by querying it from a synced node with

```shell
grpcurl -plaintext -d '' 0.0.0.0:9093 spacemesh.v1.ActivationService.Highest | jq -r '.atx.id.id' | base64 -d | xxd -p -c 64`
```

The node can be operated in "non-smeshing" mode during synchronization and when querying the highest ATX.

Alternatively you can pick an ATX from the current epoch from the explorer: <https://explorer.spacemesh.io/epochs>
* If `-id` isn't provided a new identity will be auto-generated. Its private key will be stored in `identity.key` in
`-datadir` next to the PoST data. This file then **must** to be copied/moved to the `data/identities` directory of
the node managing this identity.
* **NOTE:** The generated PoST data is ONLY valid for this identity!
* If a public key is provided with the `-id` flag, the `identities.key` file will be NOT created. Make sure that the
key file that belongs to the identity provided to `postcli` is available in the `data/identities` directory for a
PoST service using the generated data to be able to connect to the node.
* The `-reset` flag can be used to clean up a previous initialization. **Careful**: This will delete data that won't be
recoverable.

## Initializing a subset of PoST data

It is possible to initialize only subset of the files. This feature is intended to be used to split initialization
It is possible to initialize only subset of the files. This feature is intended to allow splitting initialization
between many machines.

### Example - split initialization between 2 machines
Expand Down Expand Up @@ -195,11 +204,11 @@ confidence but still completes verification in a reasonable time. Suggested valu

To verify POS data:

1. locate the directory of the POS data. It should contain postdata_metadata.json and postdata_N.bin files.
1. locate the directory of the POS data. It should contain `postdata_metadata.json` and `postdata_N.bin` files.
2. run `postcli -verify -datadir <path to POS directory> -fraction <% of data to verify>`.

For example, `postcli -verify -datadir ~/post/data -fraction 0.1` will verify 0.1% of data. No additional arguments
(i.e `-id`) are required. The postcli will read all required information from postdata_metadata.json
(i.e `-id`) are required. The postcli will read all required information from `postdata_metadata.json`

If the POS data is found to be invalid, `postcli` will exit with status 1 and print the index of file and offset of the
label found to be invalid. If verification completes successfully, `postcli` exits with 0.
Expand All @@ -213,8 +222,8 @@ re-initializing the data. Postcli will need to **read** the entire POS data and

To find a lost nonce:

1. locate the directory of the POS data. It should contain postdata_metadata.json and postdata_N.bin files.
1. locate the directory of the POS data. It should contain `postdata_metadata.json` and `postdata_N.bin` files.
2. run `postcli -searchForNonce -datadir <path to POS directory>`.

The postcli will read the metadata from postdata_metadata.json and then look for the nonce in all postdata_N.bin files
one by one. If the nonce is found it will update the metadata file.
The postcli will read the metadata from `postdata_metadata.json` and then look for the nonce in all `postdata_N.bin`
files one by one. If the nonce is found it will update the metadata file.
191 changes: 130 additions & 61 deletions cmd/postcli/main.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import (
"errors"
"flag"
"fmt"
"io/fs"
"log"
"os"
"os/signal"
Expand All @@ -25,7 +26,7 @@ import (
"github.com/spacemeshos/post/verifying"
)

const edKeyFileName = "key.bin"
const edKeyFileName = "identity.key"

var (
cfg = config.MainnetConfig()
Expand All @@ -41,9 +42,7 @@ var (
fraction float64

idHex string
id []byte
commitmentAtxIdHex string
commitmentAtxId []byte
reset bool
numUnits uint64

Expand Down Expand Up @@ -102,75 +101,134 @@ func askForConfirmation() {
return
}

fmt.Println("Are you sure you want to continue (y/N)?")
log.Println("Are you sure you want to continue (y/N)?")

var answer string
_, err := fmt.Scanln(&answer)
if err != nil || !(answer == "y" || answer == "Y") {
fmt.Println("Aborting")
os.Exit(1)
log.Fatal("Aborting")
}
}

func processFlags() error {
func processFlags() {
meta, err := initialization.LoadMetadata(opts.DataDir)
switch {
case errors.Is(err, initialization.ErrStateMetadataFileMissing):
case err != nil:
log.Fatalln("failed to load metadata:", err)
default:
if idHex == "" {
idHex = hex.EncodeToString(meta.NodeId)
} else if idHex != hex.EncodeToString(meta.NodeId) {
log.Println("WARNING: it appears that", opts.DataDir, "was previously initialized with a different `id` value.")
log.Println("\tCurrent value:", hex.EncodeToString(meta.NodeId))
log.Println("\tValue passed to postcli:", idHex)
log.Fatalln("aborting")
}
}

key, err := loadKey()
switch {
case errors.Is(err, fs.ErrNotExist):
case err != nil:
log.Fatalln("failed to load key:", err)
case meta != nil && !bytes.Equal(meta.NodeId, key):
log.Fatalln("WARNING: inconsistent state:", edKeyFileName, "file does not match metadata in", opts.DataDir)
default:
if idHex == "" {
idHex = hex.EncodeToString(key)
} else if idHex != hex.EncodeToString(key) {
log.Println("WARNING: it appears that", opts.DataDir, "was previously initialized with a generated key.")
log.Println("The provided id does not match the generated key.")
log.Println("\tCurrent value:", hex.EncodeToString(key))
log.Println("\tValue passed to postcli:", idHex)
log.Fatalln("aborting")
}
}

// we require the user to explicitly pass numUnits to avoid erasing existing data
if !flagSet["numUnits"] {
return fmt.Errorf("-numUnits must be specified to perform initialization. to use the default value, "+
"run with -numUnits %d. note: if there's more than this amount of data on disk, "+
"THIS WILL ERASE EXISTING DATA. MAKE ABSOLUTELY SURE YOU SPECIFY THE CORRECT VALUE", opts.NumUnits)
if !flagSet["numUnits"] && meta != nil {
log.Fatalln("-numUnits must be specified to perform initialization.")
}

if flagSet["numUnits"] && numUnits != uint64(meta.NumUnits) {
log.Println("WARNING: it appears that", opts.DataDir, "was previously initialized with a different `numUnits` value.")
log.Println("\tCurrent value:", meta.NumUnits)
log.Println("\tValue passed to postcli:", numUnits)
if (numUnits < uint64(meta.NumUnits)) && !yes {
log.Println("CONTINUING MIGHT ERASE EXISTING DATA. MAKE ABSOLUTELY SURE YOU SPECIFY THE CORRECT VALUE.")
}
askForConfirmation()
}

if flagSet["numUnits"] && (numUnits < uint64(cfg.MinNumUnits) || numUnits > uint64(cfg.MaxNumUnits)) {
fmt.Println("WARNING: numUnits is outside of range valid for mainnet (min:",
log.Println("WARNING: numUnits is outside of range valid for mainnet (min:",
cfg.MinNumUnits, "max:", cfg.MaxNumUnits, ")")
if !yes {
log.Println("CONTINUING WILL INITIALIZE DATA INCOMPATIBLE WITH MAINNET. MAKE ABSOLUTELY SURE YOU WANT TO DO THIS.")
}
askForConfirmation()
cfg.MinNumUnits = uint32(numUnits)
cfg.MaxNumUnits = uint32(numUnits)
}

if !flagSet["provider"] {
return errors.New("-provider flag is required")
if !flagSet["commitmentAtxId"] && meta != nil {
log.Fatalln("-commitmentAtxId must be specified to perform initialization.")
}

if !flagSet["commitmentAtxId"] {
return errors.New("-commitmentAtxId flag is required")
if flagSet["commitmentAtxId"] {
commitmentAtxId, err := hex.DecodeString(commitmentAtxIdHex)
if err != nil {
log.Println("invalid commitmentAtxId:", err)
}
if meta != nil && !bytes.Equal(commitmentAtxId, meta.CommitmentAtxId) {
log.Println("WARNING: it appears that", opts.DataDir, "was previously initialized with a different `commitmentAtxId` value.")
log.Println("\tCurrent value:", hex.EncodeToString(meta.CommitmentAtxId))
log.Println("\tValue passed to postcli:", commitmentAtxIdHex)
log.Fatalln("aborting")
}
}
var err error
commitmentAtxId, err = hex.DecodeString(commitmentAtxIdHex)
if err != nil {
return fmt.Errorf("invalid commitmentAtxId: %w", err)

if !flagSet["provider"] {
log.Fatalln("-provider flag is required")
}

if flagSet["labelsPerUnit"] && (cfg.LabelsPerUnit != config.MainnetConfig().LabelsPerUnit) {
fmt.Println("WARNING: labelsPerUnit is set to a non-default value. This makes the initialization incompatible " +
"with mainnet. If you're trying to initialize for mainnet, please remove the -labelsPerUnit flag")
log.Println("WARNING: labelsPerUnit is set to a non-default value.")
log.Println("If you're trying to initialize for mainnet, please remove the -labelsPerUnit flag")
if !yes {
log.Println("CONTINUING WILL INITIALIZE DATA INCOMPATIBLE WITH MAINNET. MAKE ABSOLUTELY SURE YOU WANT TO DO THIS.")
}
askForConfirmation()
}

if flagSet["scryptN"] {
fmt.Println("WARNING: scryptN is set to a non-default value. This makes the initialization incompatible " +
"with mainnet. If you're trying to initialize for mainnet, please remove the -scryptN flag")
if flagSet["scryptN"] && (opts.Scrypt.N != config.MainnetInitOpts().Scrypt.N) {
log.Println("WARNING: scryptN is set to a non-default value.")
log.Println("If you're trying to initialize for mainnet, please remove the -scryptN flag")
if !yes {
log.Println("CONTINUING WILL INITIALIZE DATA INCOMPATIBLE WITH MAINNET. MAKE ABSOLUTELY SURE YOU WANT TO DO THIS.")
}
askForConfirmation()
}

if (opts.FromFileIdx != 0 || opts.ToFileIdx != nil) && idHex == "" {
return errors.New("-id flag is required when using -fromFile or -toFile")
log.Fatalln("-id flag is required when using -fromFile or -toFile")
}

if idHex == "" {
log.Println("cli: generating new identity")
pub, priv, err := ed25519.GenerateKey(nil)
if err != nil {
return fmt.Errorf("failed to generate identity: %w", err)
log.Fatalln("failed to generate identity:", err)
}
id = pub
log.Printf("cli: generated id %x\n", id)
return saveKey(priv)
}
id, err = hex.DecodeString(idHex)
if err != nil {
return fmt.Errorf("invalid id: %w", err)
if err := saveKey(priv); err != nil {
log.Fatalln("failed to save identity:", err)
}
idHex = hex.EncodeToString(pub)
log.Println("generated key in", edKeyFileName)
log.Println("copy this file to the `data/identities` directory of your node")
return
}
return nil
}

func main() {
Expand All @@ -187,7 +245,7 @@ func main() {

if printNumFiles {
totalFiles := opts.TotalFiles(cfg.LabelsPerUnit)
fmt.Println(totalFiles)
log.Println(totalFiles)
return
}

Expand Down Expand Up @@ -248,12 +306,16 @@ func main() {
return
}

err = processFlags()
switch {
case errors.Is(err, ErrKeyFileExists):
log.Fatalln("cli: key file already exists. This appears to be a mistake. If you're trying to initialize a new identity delete key.bin and try again otherwise specify identity with `-id` flag")
case err != nil:
log.Fatalln("failed to process flags:", err)
processFlags()

id, err := hex.DecodeString(idHex)
if err != nil {
log.Fatalf("failed to decode id %s: %s\n", idHex, err)
}

commitmentAtxId, err := hex.DecodeString(commitmentAtxIdHex)
if err != nil {
log.Fatalf("failed to decode commitmentAtxId %s: %s\n", commitmentAtxIdHex, err)
}

init, err := initialization.NewInitializer(
Expand All @@ -277,11 +339,8 @@ func main() {

err = init.Initialize(ctx)
switch {
case errors.Is(err, shared.ErrInitCompleted):
log.Panic(err.Error())
return
case errors.Is(err, context.Canceled):
log.Println("cli: initialization interrupted")
log.Fatalln("cli: initialization interrupted")
return
case err != nil:
log.Println("cli: initialization error", err)
Expand Down Expand Up @@ -312,7 +371,10 @@ func main() {
}

func saveKey(key ed25519.PrivateKey) error {
if err := os.MkdirAll(opts.DataDir, 0o700); err != nil && !os.IsExist(err) {
err := os.MkdirAll(opts.DataDir, 0o700)
switch {
case errors.Is(err, os.ErrExist):
case err != nil:
return fmt.Errorf("mkdir error: %w", err)
}

Expand All @@ -327,13 +389,11 @@ func saveKey(key ed25519.PrivateKey) error {
return nil
}

func cmdVerifyPos(opts config.InitOpts, fraction float64, logger *zap.Logger) {
log.Println("cli: verifying key.bin")

func loadKey() (ed25519.PublicKey, error) {
keyPath := filepath.Join(opts.DataDir, edKeyFileName)
data, err := os.ReadFile(keyPath)
if err != nil {
log.Fatalf("could not read private key from %s: %s\n", keyPath, err)
return nil, fmt.Errorf("could not read private key from %s: %w", keyPath, err)
}

dst := make([]byte, ed25519.PrivateKeySize)
Expand All @@ -344,21 +404,30 @@ func cmdVerifyPos(opts config.InitOpts, fraction float64, logger *zap.Logger) {
if n != ed25519.PrivateKeySize {
log.Fatalf("size of key (%d) not expected size %d\n", n, ed25519.PrivateKeySize)
}
pub := ed25519.NewKeyFromSeed(dst[:ed25519.SeedSize]).Public().(ed25519.PublicKey)
return ed25519.NewKeyFromSeed(dst[:ed25519.SeedSize]).Public().(ed25519.PublicKey), nil
}

metafile := filepath.Join(opts.DataDir, initialization.MetadataFileName)
meta, err := initialization.LoadMetadata(opts.DataDir)
if err != nil {
log.Fatalf("failed to load metadata from %s: %s\n", opts.DataDir, err)
}
func cmdVerifyPos(opts config.InitOpts, fraction float64, logger *zap.Logger) {
log.Println("cli: verifying", edKeyFileName)
pub, err := loadKey()
switch {
case errors.Is(err, fs.ErrNotExist):
case err != nil:
log.Fatalf("failed to load public key from %s: %s\n", edKeyFileName, err)
default:
metaFile := filepath.Join(opts.DataDir, initialization.MetadataFileName)
meta, err := initialization.LoadMetadata(opts.DataDir)
if err != nil {
log.Fatalf("failed to load metadata from %s: %s\n", opts.DataDir, err)
}

if !bytes.Equal(meta.NodeId, pub) {
log.Fatalf("NodeID in %s (%x) does not match public key from key.bin (%x)", metafile, meta.NodeId, pub)
if !bytes.Equal(meta.NodeId, pub) {
log.Fatalf("NodeID in %s (%x) does not match public key from %s (%x)", metaFile, meta.NodeId, edKeyFileName, pub)
}
log.Println("cli:", edKeyFileName, "is valid")
}

log.Println("cli: key.bin is valid")
log.Println("cli: verifying POS data")

params := postrs.NewScryptParams(opts.Scrypt.N, opts.Scrypt.R, opts.Scrypt.P)
verifyOpts := []postrs.VerifyPosOptionsFunc{
postrs.WithFraction(fraction),
Expand Down
Loading

0 comments on commit 83f3fde

Please sign in to comment.