A0-4318: network rate-limiter for the substrate network #1780
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- parametrized by TimeProvider instead of taking `now: Instant` as argument for `rate_limit` - simplified rate_limit method - re-aligned tests of TokenBucket - TokenBucket can be set to rate_per_second = 0 - no data will be read
- seperate RateLimiter implementations for [`tokio::io::AsyncRead`] and [`futures::AsyncRead`] - using [`std::sync::Mutex`] for TokenBucket in RateLimiter - it should allow for `global` rate-limit, not just per connection
… substrate's [`sc_client::network::NetworkWorker`]
- s/alephbft_bit_rate/alephbft_network_bit_rate
…ces `TokenBucket`
…substrate_network
… [`finality-aleph/src/network/build/transport.rs`]
…port.rs`] - removed references to [`libp2p`] from [`finality-aleph/src/network/build/mod.rs`]
fixxxedpoint
force-pushed
the
A0-4318_rate_limited_substrate_network
branch
from
04c0718 to
3e7ff62
Compare
fixxxedpoint
requested review from
ggawryal
and removed request for
Marcin-Radecki
ggawryal
reviewed
Jul 11, 2024
deuszx
reviewed
Jul 11, 2024
deuszx
reviewed
Jul 11, 2024
…substrate_network
…d of `requeste` and `available`. It should be also easier now to transform it into wait-free version.
…substrate_network
… - so far only tested locally using 7 instances of aleph-node.
…>` in `network/substrate.rs`
…substrate_network
timorleph
approved these changes
Aug 13, 2024
ggawryal
reviewed
Aug 14, 2024
ggawryal
approved these changes
Aug 14, 2024
…n `finality-aleph/src/network/build/mod.rs`
fixxxedpoint
force-pushed
the
A0-4318_rate_limited_substrate_network
branch
from
6b21677 to
21b4b73
Compare
…ant>>` in TokenBucket
…substrate_network
- added ShareTokenBucket, AsyncTokenBucket and ShredBandwidthManager - used for sharing bandwidth between multiple connections
…and Socket-based networks (sync + alephbft) - new default values for rate-limiters (sync + alephbft). 768Kib for alephbft and 5Mib for sync. These values were tested using t3a.xlarge aws instances - nodes were not able to handle more.
…docker/docker_entrypoint.sh to allow configuration of rate-limiting for both sync and alephbft networks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Implementation of the rate-limiter algorithm for the substrate-based part of our network code. It limits only the
readside of connections.Additionally, now both rate-limit parameters for alephbft and substrate networks apply to all open connection (rate-limit for all nodes, not just single connection).An exploit together with e2e tests will be provided in another PR.Type of change