Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Various fixes for upstream #1057

Merged
merged 7 commits into from
Dec 16, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .github/workflows/parachain.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,11 @@ jobs:
key: ${{ runner.os }}-cargo-${{ hashFiles('parachain/Cargo.lock') }}
restore-keys: |
${{ runner.os }}-cargo-
- uses: actions/[email protected]
with:
node-version: "18.x"
registry-url: "https://npm.pkg.github.com"
scope: "@paritytech"
- name: setup rust toolchain
run: rustup show
- name: cargo check
Expand All @@ -63,6 +68,12 @@ jobs:
working-directory: parachain
run: |
cargo install zepter -f --locked && zepter run check
- name: lint-markdown
working-directory: parachain
run: |
npm install -g markdownlint-cli
markdownlint --version
markdownlint --config .config/.markdownlint.yaml --ignore target

test:
needs: check
Expand Down
210 changes: 210 additions & 0 deletions parachain/.config/.markdownlint.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,210 @@
# Default state for all rules
default: true

# Path to configuration file to extend
extends: null

# MD001/heading-increment/header-increment - Heading levels should only increment by one level at a time
MD001: true

# MD002/first-heading-h1/first-header-h1 - First heading should be a top-level heading
MD002:
# Heading level
level: 1

# MD003/heading-style/header-style - Heading style
MD003:
# Heading style
style: "consistent"

# MD004/ul-style - Unordered list style
MD004:
# List style
style: "consistent"

# MD005/list-indent - Inconsistent indentation for list items at the same level
MD005: false

# MD006/ul-start-left - Consider starting bulleted lists at the beginning of the line
MD006: false

# MD007/ul-indent - Unordered list indentation
MD007: false

# MD009/no-trailing-spaces - Trailing spaces
MD009:
# Spaces for line break
br_spaces: 2
# Allow spaces for empty lines in list items
list_item_empty_lines: false
# Include unnecessary breaks
strict: false

# MD010/no-hard-tabs - Hard tabs
MD010: false

# MD011/no-reversed-links - Reversed link syntax
MD011: true

# MD012/no-multiple-blanks - Multiple consecutive blank lines
MD012:
# Consecutive blank lines
maximum: 2

# MD013/line-length - Line length
MD013:
# Number of characters
line_length: 120
# Number of characters for headings
heading_line_length: 120
# Number of characters for code blocks
code_block_line_length: 150
# Include code blocks
code_blocks: true
# Include tables
tables: true
# Include headings
headings: true
# Include headings
headers: true
# Strict length checking
strict: false
# Stern length checking
stern: false

# MD014/commands-show-output - Dollar signs used before commands without showing output
MD014: true

# MD018/no-missing-space-atx - No space after hash on atx style heading
MD018: true

# MD019/no-multiple-space-atx - Multiple spaces after hash on atx style heading
MD019: true

# MD020/no-missing-space-closed-atx - No space inside hashes on closed atx style heading
MD020: true

# MD021/no-multiple-space-closed-atx - Multiple spaces inside hashes on closed atx style heading
MD021: true

# MD022/blanks-around-headings/blanks-around-headers - Headings should be surrounded by blank lines
MD022: false

# MD023/heading-start-left/header-start-left - Headings must start at the beginning of the line
MD023: true

# MD024/no-duplicate-heading/no-duplicate-header - Multiple headings with the same content
MD024: false

# MD025/single-title/single-h1 - Multiple top-level headings in the same document
MD025: false

# MD026/no-trailing-punctuation - Trailing punctuation in heading
MD026:
# Punctuation characters
punctuation: ".,;:!。,;:!"

# MD027/no-multiple-space-blockquote - Multiple spaces after blockquote symbol
MD027: true

# MD028/no-blanks-blockquote - Blank line inside blockquote
MD028: true

# MD029/ol-prefix - Ordered list item prefix
MD029:
# List style
style: "one_or_ordered"

# MD030/list-marker-space - Spaces after list markers
MD030:
# Spaces for single-line unordered list items
ul_single: 1
# Spaces for single-line ordered list items
ol_single: 1
# Spaces for multi-line unordered list items
ul_multi: 1
# Spaces for multi-line ordered list items
ol_multi: 1

# MD031/blanks-around-fences - Fenced code blocks should be surrounded by blank lines
MD031: false

# MD032/blanks-around-lists - Lists should be surrounded by blank lines
MD032: false

# MD033/no-inline-html - Inline HTML
MD033: false

# MD034/no-bare-urls - Bare URL used
MD034: false

# MD035/hr-style - Horizontal rule style
MD035:
# Horizontal rule style
style: "consistent"

# MD036/no-emphasis-as-heading/no-emphasis-as-header - Emphasis used instead of a heading
MD036: false

# MD037/no-space-in-emphasis - Spaces inside emphasis markers
MD037: true

# MD038/no-space-in-code - Spaces inside code span elements
MD038: true

# MD039/no-space-in-links - Spaces inside link text
MD039: true

# MD040/fenced-code-language - Fenced code blocks should have a language specified
MD040: false

# MD041/first-line-heading/first-line-h1 - First line in a file should be a top-level heading
MD041: false

# MD042/no-empty-links - No empty links
MD042: true

# MD043/required-headings/required-headers - Required heading structure
MD043: false

# MD044/proper-names - Proper names should have the correct capitalization
MD044:
# List of proper names
names: ["Polkadot", "Substrate", "Cumulus", "Parity"]
# Include code blocks
code_blocks: false
# Include HTML elements
html_elements: false

# MD045/no-alt-text - Images should have alternate text (alt text)
MD045: false

# MD046/code-block-style - Code block style
MD046:
# Block style
style: "consistent"

# MD047/single-trailing-newline - Files should end with a single newline character
MD047: true

# MD048/code-fence-style - Code fence style
MD048:
# Code fence style
style: "consistent"

# MD049/emphasis-style - Emphasis style should be consistent
MD049: false

# MD050/strong-style - Strong style should be consistent
MD050:
# Strong style
style: "consistent"

# MD051/link-fragments - Link fragments should be valid
MD051: false

# MD052/reference-links-images - Reference links and images should use a label that is defined
MD052: false

# MD053/link-image-reference-definitions - Link and image reference definitions should be needed
MD053: false
28 changes: 18 additions & 10 deletions parachain/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@
Note: This section is not necessary for local development, as there are scripts to auto-configure the parachain in the
[test directory](../web/packages/test).

For a fully operational chain, further configuration of the initial chain spec is required. The specific configuration will depend heavily on your environment, so this guide will remain high-level.
For a fully operational chain, further configuration of the initial chain spec is required. The specific configuration will
depend heavily on your environment, so this guide will remain high-level.

After completing a release build of the parachain, build an initial spec for the snowbase runtime:

Expand All @@ -18,11 +19,13 @@ Now edit the spec and configure the following:
2. Contract addresses for the Ether, Erc20, and Dot apps.
3. Authorized principal for the basic channel

For an example configuration, consult the [setup script](https://github.com/Snowfork/snowbridge/blob/main/web/packages/test/scripts/start-services.sh) for our local development stack. Specifically the `start_polkadot_launch` bash function.
For an example configuration, consult the [setup script](https://github.com/Snowfork/snowbridge/blob/main/web/packages/test/scripts/start-services.sh)
for our local development stack. Specifically the `start_polkadot_launch` bash function.

## Tests

To run the parachain tests locally, use `cargo test --workspace`. For the full suite of tests, use `cargo test --workspace --features runtime-benchmarks`.
To run the parachain tests locally, use `cargo test --workspace`. For the full suite of tests, use
`cargo test --workspace --features runtime-benchmarks`.

Optionally exclude the top-level and runtime crates:

Expand Down Expand Up @@ -63,8 +66,8 @@ const decodedEventLog = iface.decodeEventLog(
console.log(`decoded rawLog.data: ${JSON.stringify(decodedEventLog)}`);
```

Place the `encodedLog` string in the `message.data` field in the test data. Use the `decoded rawLog.data` field to update the comments
with the decoded log data.
Place the `encodedLog` string in the `message.data` field in the test data. Use the `decoded rawLog.data` field to
update the comments with the decoded log data.

## Generating pallet weights from benchmarks

Expand Down Expand Up @@ -105,7 +108,8 @@ target/release/snowbridge benchmark pallet \

### Minimal Spec

To generate `minimal` test data and benchmarking data, make sure to start the local E2E setup to spin up a local beacon node instance to connect to:
To generate `minimal` test data and benchmarking data, make sure to start the local E2E setup to spin up a local beacon
node instance to connect to:

```bash
cd web/packages/test
Expand All @@ -117,14 +121,17 @@ Wait for output `Testnet has been initialized`.
In a separate terminal, from the `snowbridge` directory, run:

```bash
mage -d relayer build && relayer/build/snowbridge-relay generate-beacon-data --spec "minimal" && cd parachain && cargo +nightly fmt -- --config-path rustfmt.toml && cd -
mage -d relayer build && relayer/build/snowbridge-relay generate-beacon-data --spec "minimal" && cd parachain &&
cargo +nightly fmt -- --config-path rustfmt.toml && cd -
```

### Mainnet Spec

We only use the mainnet spec for generating fixtures for pallet weight benchmarks.

To generate the data we can connect to the Lodestar Goerli public node. The script already connects to the Lodestar node, so no need to start up additional services. In the event of the Lodestar node not being available, you can start up your own stack with these commands:
To generate the data we can connect to the Lodestar Goerli public node. The script already connects to the Lodestar node,
so no need to start up additional services. In the event of the Lodestar node not being available, you can start up your
own stack with these commands:

```bash
cd web/packages/test
Expand All @@ -134,10 +141,11 @@ cd web/packages/test
From the `snowbridge` directory, run:

```bash
mage -d relayer build && relayer/build/snowbridge-relay generate-beacon-data --spec "mainnet" && cd parachain && cargo +nightly fmt -- --config-path rustfmt.toml && cd -
mage -d relayer build && relayer/build/snowbridge-relay generate-beacon-data --spec "mainnet" && cd parachain &&
cargo +nightly fmt -- --config-path rustfmt.toml && cd -
```

### Benchmarking tests
### Benchmarking tests

To run the benchmark tests

Expand Down
2 changes: 1 addition & 1 deletion parachain/pallets/ethereum-beacon-client/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ version = "0.0.1"
edition = "2021"
authors = ["Snowfork <[email protected]>"]
repository = "https://github.com/Snowfork/snowbridge"
license = "PENDING/TBC"
license = "Apache-2.0"

[package.metadata.docs.rs]
targets = ["x86_64-unknown-linux-gnu"]
Expand Down
27 changes: 19 additions & 8 deletions parachain/pallets/ethereum-beacon-client/benchmark.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,25 @@
# Motivation
Demonstrate that [FastAggregateVerify](https://datatracker.ietf.org/doc/html/draft-irtf-cfrg-bls-signature-04#section-3.3.4) is the most expensive call in ethereum beacon light client, though in [#13031](https://github.com/paritytech/substrate/pull/13031) Parity team has wrapped some low level host functions for `bls-12381` but adding a high level host function specific for it is super helpful.
Demonstrate that
[FastAggregateVerify](https://datatracker.ietf.org/doc/html/draft-irtf-cfrg-bls-signature-04#section-3.3.4) is the most
expensive call in ethereum beacon light client, though in [#13031](https://github.com/paritytech/substrate/pull/13031)
Parity team has wrapped some low level host functions for `bls-12381` but adding a high level host function specific
for it is super helpful.

# Benchmark
We add several benchmarks [here](https://github.com/Snowfork/snowbridge/blob/8891ca3cdcf2e04d8118c206588c956541ae4710/parachain/pallets/ethereum-beacon-client/src/benchmarking/mod.rs#L98-L124) as following to demonstrate [bls_fast_aggregate_verify](https://github.com/Snowfork/snowbridge/blob/8891ca3cdcf2e04d8118c206588c956541ae4710/parachain/pallets/ethereum-beacon-client/src/lib.rs#L764) is the main bottleneck. Test data [here](https://github.com/Snowfork/snowbridge/blob/8891ca3cdcf2e04d8118c206588c956541ae4710/parachain/pallets/ethereum-beacon-client/src/benchmarking/data_mainnet.rs#L553-L1120) is real from goerli network which contains 512 public keys from sync committee.

We add several benchmarks
[here](https://github.com/Snowfork/snowbridge/blob/8891ca3cdcf2e04d8118c206588c956541ae4710/parachain/pallets/ethereum-beacon-client/src/benchmarking/mod.rs#L98-L124)
as following to demonstrate
[bls_fast_aggregate_verify](https://github.com/Snowfork/snowbridge/blob/8891ca3cdcf2e04d8118c206588c956541ae4710/parachain/pallets/ethereum-beacon-client/src/lib.rs#L764)
is the main bottleneck. Test data
[here](https://github.com/Snowfork/snowbridge/blob/8891ca3cdcf2e04d8118c206588c956541ae4710/parachain/pallets/ethereum-beacon-client/src/benchmarking/data_mainnet.rs#L553-L1120)
is real from goerli network which contains 512 public keys from sync committee.

## sync_committee_period_update
Base line benchmark for extrinsic [sync_committee_period_update](https://github.com/Snowfork/snowbridge/blob/8891ca3cdcf2e04d8118c206588c956541ae4710/parachain/pallets/ethereum-beacon-client/src/lib.rs#L233)


## bls_fast_aggregate_verify
Subfunction of extrinsic `sync_committee_period_update` which does what [FastAggregateVerify](https://datatracker.ietf.org/doc/html/draft-irtf-cfrg-bls-signature-04#section-3.3.4) requires.
Subfunction of extrinsic `sync_committee_period_update` which does what
[FastAggregateVerify](https://datatracker.ietf.org/doc/html/draft-irtf-cfrg-bls-signature-04#section-3.3.4) requires.

## bls_aggregate_pubkey
Subfunction of `bls_fast_aggregate_verify` which decompress and instantiate G1 pubkeys only.
Expand All @@ -22,7 +31,7 @@ Subfunction of `bls_fast_aggregate_verify` which verify the prepared signature o
# Result

## hardware spec
Run benchmark in a EC2 instance
Run benchmark in a EC2 instance
```
cargo run --release --bin polkadot-parachain --features runtime-benchmarks -- benchmark machine --base-path /mnt/scratch/benchmark

Expand Down Expand Up @@ -61,7 +70,7 @@ benchmark pallet \

|extrinsic | minimum execution time benchmarked(us) |
| --------------------------------------- |----------------------------------------|
|sync_committee_period_update | 123_126 |
|sync_committee_period_update | 123_126 |
|bls_fast_aggregate_verify| 121_083 |
|bls_aggregate_pubkey | 90_306 |
|bls_verify_message | 28_000 |
Expand All @@ -74,4 +83,6 @@ benchmark pallet \

# Conclusion

A high level host function specific for [bls_fast_aggregate_verify](https://github.com/Snowfork/snowbridge/blob/8891ca3cdcf2e04d8118c206588c956541ae4710/parachain/pallets/ethereum-beacon-client/src/lib.rs#L764) is super helpful.
A high level host function specific for
[bls_fast_aggregate_verify](https://github.com/Snowfork/snowbridge/blob/8891ca3cdcf2e04d8118c206588c956541ae4710/parachain/pallets/ethereum-beacon-client/src/lib.rs#L764)
is super helpful.
2 changes: 1 addition & 1 deletion parachain/pallets/ethereum-beacon-client/fuzz/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,5 +17,5 @@ cargo install cargo-fuzz
Note: `max-len` is necessary because the max input length is 4096 bytes. Some of our inputs are larger than this
default value. When running the tests without an increased max len parameter, no fuzz data will be generated.

The tests will keep running until a crash is found, so in our CI setup the number of runs is limited so that the
The tests will keep running until a crash is found, so in our CI setup the number of runs is limited so that the
test completes.
1 change: 1 addition & 0 deletions parachain/pallets/inbound-queue/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ version = "0.1.1"
edition = "2021"
authors = ["Snowfork <[email protected]>"]
repository = "https://github.com/Snowfork/snowbridge"
license = "Apache-2.0"

[package.metadata.docs.rs]
targets = ["x86_64-unknown-linux-gnu"]
Expand Down
Loading
Loading