Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: price algo plus price changes #174

Closed
wants to merge 54 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
e2a6616
historical fee service: working on calculating SMAs
segfault-magnet Dec 12, 2024
8eb3243
starting to add decision logic based on sma price
segfault-magnet Dec 14, 2024
f7cf478
cleanup tests
segfault-magnet Dec 14, 2024
b3ec73b
shorten tests
segfault-magnet Dec 14, 2024
6599fcf
parameterized tests
segfault-magnet Dec 14, 2024
281086b
add more tests
segfault-magnet Dec 14, 2024
8b74db3
more edge cases
segfault-magnet Dec 14, 2024
cc451cf
use sequential fees struct, rename price service to fee analytics
segfault-magnet Dec 14, 2024
fea814a
add activation fee threshold
segfault-magnet Dec 14, 2024
3bfcca0
add comparison strategy
segfault-magnet Dec 14, 2024
7e5769f
add tests for treshold strategy
segfault-magnet Dec 14, 2024
855ac26
l1 adapter for historical fees, about to test out algo on historical
segfault-magnet Dec 14, 2024
c2f1601
historical testing of algo
segfault-magnet Dec 14, 2024
c732cd7
adding in the fee acceptance scaling for being late with posting
segfault-magnet Dec 15, 2024
3a25064
checked tests validity, tweaked values to better represent the test
segfault-magnet Dec 15, 2024
c1d4824
analyzing algo config
segfault-magnet Dec 15, 2024
6d74ce3
committer compiles, unit tests fixed
segfault-magnet Dec 15, 2024
f7b44cf
tracks fragment age, tests fixed
segfault-magnet Dec 16, 2024
ac72a02
pull up the config to state committer level so it can be exported out
segfault-magnet Dec 16, 2024
b8d6631
added tests to state committer showing that the price algo is consulted
segfault-magnet Dec 16, 2024
fb6c016
all tests passing, including e2e
segfault-magnet Dec 16, 2024
c7aa8fa
cleanup
segfault-magnet Dec 16, 2024
79ee936
add tests for max fee calculation
segfault-magnet Dec 16, 2024
d4785a0
some todos
segfault-magnet Dec 16, 2024
7326055
add tests for converting fees
segfault-magnet Dec 17, 2024
c123862
move fee conversion to separate mod
segfault-magnet Dec 17, 2024
4506b80
removing fee analytics as a service, making it a helper
segfault-magnet Dec 17, 2024
4c42cb7
add check whether provider reported all requested fees
segfault-magnet Dec 17, 2024
61c6fd3
add caching and tests
segfault-magnet Dec 17, 2024
8b6993b
changed the last n block calculation, need to fix tests
segfault-magnet Dec 17, 2024
94c8979
fixed tests
segfault-magnet Dec 18, 2024
004be10
validate percentages
segfault-magnet Dec 18, 2024
963422e
add capping for discount percentage at 100%
segfault-magnet Dec 18, 2024
80b9cef
if too far back, makes decision even with a faulty fee provider
segfault-magnet Dec 18, 2024
7fa623d
add logging, remove print statements
segfault-magnet Dec 18, 2024
3ab2e93
added fees at height
segfault-magnet Dec 18, 2024
62f6f38
moving fee tracking into its own service due to a need for regular
segfault-magnet Dec 18, 2024
bfc5eb0
add helper to lessen test boilerplate
segfault-magnet Dec 18, 2024
d39838e
remove context
segfault-magnet Dec 18, 2024
f908f32
cargo fix
segfault-magnet Dec 18, 2024
d87d5cb
add fee tracker to be run periodically
segfault-magnet Dec 18, 2024
ebb8464
revert debug logs
segfault-magnet Dec 18, 2024
c34af7e
Merge remote-tracking branch 'origin/master' into feat/price_optimizi…
segfault-magnet Dec 18, 2024
06755d7
test(costs): get latest costs from api
rymnc Dec 17, 2024
d8d39de
Merge remote-tracking branch 'origin/master' into feat/price_optimizi…
segfault-magnet Dec 19, 2024
3ed11b7
Merge branch 'feat/price_optimizing_algo' into feat/price_algo_plus_p…
segfault-magnet Dec 19, 2024
c8a7ca7
enable cache
segfault-magnet Dec 19, 2024
da0ab9c
Merge branch 'feat/price_optimizing_algo' into feat/price_algo_plus_p…
segfault-magnet Dec 19, 2024
c1a9ea3
add more logging to debug
segfault-magnet Dec 19, 2024
41b3750
Merge branch 'feat/price_optimizing_algo' into feat/price_algo_plus_p…
segfault-magnet Dec 19, 2024
173a269
fix bug, minimum not maximum starting height when calculating l2 blocks
segfault-magnet Dec 23, 2024
5f56a7d
Merge branch 'feat/price_optimizing_algo' into feat/price_algo_plus_p…
segfault-magnet Dec 23, 2024
3a554e4
mult reward by INTRINSIC_GAS
segfault-magnet Dec 23, 2024
abe9e12
Merge branch 'feat/price_optimizing_algo' into feat/price_algo_plus_p…
segfault-magnet Dec 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

24 changes: 24 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

24 changes: 21 additions & 3 deletions committer/src/api.rs
Original file line number Diff line number Diff line change
Expand Up @@ -90,9 +90,17 @@ async fn metrics(registry: web::Data<Arc<Registry>>) -> impl Responder {
std::result::Result::<_, InternalError<_>>::Ok(text)
}

#[derive(Deserialize)]
#[serde(rename_all = "lowercase")]
enum HeightVariant {
Latest,
Specific,
}

#[derive(Deserialize)]
struct CostQueryParams {
from_height: u32,
variant: HeightVariant,
value: Option<u32>,
limit: Option<usize>,
}

Expand All @@ -103,8 +111,18 @@ async fn costs(
) -> impl Responder {
let limit = query.limit.unwrap_or(100);

match data.get_costs(query.from_height, limit).await {
Ok(bundle_costs) => HttpResponse::Ok().json(bundle_costs),
let response = match query.variant {
HeightVariant::Latest => data.get_latest_costs(limit).await,
HeightVariant::Specific => match query.value {
Some(height) => data.get_costs(height, limit).await,
None => Err(services::Error::Other(
"height value is required".to_string(),
)),
},
};

match response {
Ok(costs) => HttpResponse::Ok().json(costs),
Err(services::Error::Other(e)) => {
HttpResponse::from_error(InternalError::new(e, StatusCode::BAD_REQUEST))
}
Expand Down
29 changes: 28 additions & 1 deletion committer/src/config.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use std::{
net::Ipv4Addr,
num::{NonZeroU32, NonZeroUsize},
num::{NonZeroU32, NonZeroU64, NonZeroUsize},
str::FromStr,
time::Duration,
};
Expand Down Expand Up @@ -93,6 +93,9 @@ pub struct App {
/// How often to check for finalized l1 txs
#[serde(deserialize_with = "human_readable_duration")]
pub tx_finalization_check_interval: Duration,
/// How often to check for l1 prices
#[serde(deserialize_with = "human_readable_duration")]
pub l1_prices_check_interval: Duration,
/// Number of L1 blocks that need to pass to accept the tx as finalized
pub num_blocks_to_finalize_tx: u64,
/// Interval after which to bump a pending tx
Expand All @@ -111,6 +114,30 @@ pub struct App {
/// How often to run state pruner
#[serde(deserialize_with = "human_readable_duration")]
pub state_pruner_run_interval: Duration,
/// Configuration for the fee algorithm used by the StateCommitter
pub fee_algo: FeeAlgoConfig,
}

/// Configuration for the fee algorithm used by the StateCommitter
#[derive(Debug, Clone, Deserialize)]
pub struct FeeAlgoConfig {
/// Short-term period for Simple Moving Average (SMA) in block numbers
pub short_sma_blocks: NonZeroU64,

/// Long-term period for Simple Moving Average (SMA) in block numbers
pub long_sma_blocks: NonZeroU64,

/// Maximum number of unposted L2 blocks before sending a transaction regardless of fees
pub max_l2_blocks_behind: NonZeroU32,

/// Starting discount percentage applied we try to achieve if we're 0 l2 blocks behind
pub start_discount_percentage: f64,

/// Premium percentage we're willing to pay if we're max_l2_blocks_behind - 1 blocks behind
pub end_premium_percentage: f64,

/// A fee that is always acceptable regardless of other conditions
pub always_acceptable_fee: u64,
}

/// Configuration settings for managing fuel block bundling and fragment submission operations.
Expand Down
13 changes: 12 additions & 1 deletion committer/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ mod setup;
use api::launch_api_server;
use errors::{Result, WithContext};
use metrics::prometheus::Registry;
use services::fee_tracker::port::cache::CachingApi;
use setup::last_finalization_metric;
use tokio_util::sync::CancellationToken;

Expand Down Expand Up @@ -72,13 +73,22 @@ async fn main() -> Result<()> {
&metrics_registry,
);

let (fee_tracker, fee_tracker_handle) = setup::fee_tracker(
ethereum_rpc.clone(),
cancel_token.clone(),
&config,
&metrics_registry,
)?;

let state_committer_handle = setup::state_committer(
fuel_adapter.clone(),
ethereum_rpc.clone(),
storage.clone(),
cancel_token.clone(),
&config,
);
&metrics_registry,
fee_tracker,
)?;

let state_importer_handle =
setup::block_importer(fuel_adapter, storage.clone(), cancel_token.clone(), &config);
Expand All @@ -105,6 +115,7 @@ async fn main() -> Result<()> {
handles.push(state_importer_handle);
handles.push(block_bundler);
handles.push(state_listener_handle);
handles.push(fee_tracker_handle);
// Enable pruner once the issue is resolved
//TODO: https://github.com/FuelLabs/fuel-block-committer/issues/173
// handles.push(state_pruner_handle);
Expand Down
54 changes: 50 additions & 4 deletions committer/src/setup.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,10 @@ use metrics::{
};
use services::{
block_committer::{port::l1::Contract, service::BlockCommitter},
fee_tracker::{
port::cache::CachingApi,
service::{FeeThresholds, FeeTracker, SmaPeriods},
},
state_committer::port::Storage,
state_listener::service::StateListener,
state_pruner::service::StatePruner,
Expand Down Expand Up @@ -117,7 +121,9 @@ pub fn state_committer(
storage: Database,
cancel_token: CancellationToken,
config: &config::Config,
) -> tokio::task::JoinHandle<()> {
registry: &Registry,
fee_tracker: FeeTracker<CachingApi<L1>>,
) -> Result<tokio::task::JoinHandle<()>> {
let state_committer = services::StateCommitter::new(
l1,
fuel,
Expand All @@ -127,17 +133,19 @@ pub fn state_committer(
fragment_accumulation_timeout: config.app.bundle.fragment_accumulation_timeout,
fragments_to_accumulate: config.app.bundle.fragments_to_accumulate,
gas_bump_timeout: config.app.gas_bump_timeout,
tx_max_fee: config.app.tx_max_fee as u128,
},
SystemClock,
fee_tracker,
);

schedule_polling(
state_committer.register_metrics(registry);

Ok(schedule_polling(
config.app.tx_finalization_check_interval,
state_committer,
"State Committer",
cancel_token,
)
))
}

pub fn block_importer(
Expand Down Expand Up @@ -316,3 +324,41 @@ pub async fn shut_down(
storage.close().await;
Ok(())
}

pub fn fee_tracker(
l1: L1,
cancel_token: CancellationToken,
config: &config::Config,
registry: &Registry,
) -> Result<(FeeTracker<CachingApi<L1>>, tokio::task::JoinHandle<()>)> {
let fee_tracker = FeeTracker::new(
CachingApi::new(l1, 24 * 3600 / 12),
services::fee_tracker::service::Config {
sma_periods: SmaPeriods {
short: config.app.fee_algo.short_sma_blocks,
long: config.app.fee_algo.long_sma_blocks,
},
fee_thresholds: FeeThresholds {
max_l2_blocks_behind: config.app.fee_algo.max_l2_blocks_behind,
start_discount_percentage: config
.app
.fee_algo
.start_discount_percentage
.try_into()?,
end_premium_percentage: config.app.fee_algo.end_premium_percentage.try_into()?,
always_acceptable_fee: config.app.fee_algo.always_acceptable_fee as u128,
},
},
);

fee_tracker.register_metrics(registry);

let handle = schedule_polling(
config.app.tx_finalization_check_interval,
fee_tracker.clone(),
"Fee Tracker",
cancel_token,
);

Ok((fee_tracker, handle))
}
Loading
Loading