Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"contract" op code sanity checks #1453

Merged
merged 43 commits into from
Nov 15, 2023
Merged
Show file tree
Hide file tree
Changes from 34 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
8df83c3
WIP add infra for contract testing
MitchTurner Oct 25, 2023
333e74a
Restructure new bench infra, add stalling `call` bench
MitchTurner Oct 26, 2023
8fd9633
Merge branch 'master' into contract-op-code-sanity-check
MitchTurner Oct 26, 2023
864fe3b
Fix insert
MitchTurner Oct 27, 2023
1fe61c6
WIP debugging stalling benchmark
MitchTurner Oct 27, 2023
39be18e
Get simple bench working
MitchTurner Oct 27, 2023
b79b577
Get call bench actually working!
MitchTurner Oct 28, 2023
0def296
Add range of contract sizes for `call` benches
MitchTurner Oct 30, 2023
773b8e1
Add `bal` benches
MitchTurner Oct 30, 2023
dc2aa9d
Finish simple benches, bring over example code for complex ones
MitchTurner Oct 30, 2023
7e4c311
WIP add new methods without verification
MitchTurner Oct 31, 2023
31fa121
WIP Fix some compelation
MitchTurner Oct 31, 2023
919127f
WIP Fix some benches, add more
MitchTurner Nov 1, 2023
e58abf7
Add `scwq` bench, but comment out because undercosted
MitchTurner Nov 2, 2023
e43b05b
Add all but tro and other sequential
MitchTurner Nov 2, 2023
1b89fb7
Fix all but smo and tro
MitchTurner Nov 2, 2023
dd62a49
Merge branch 'master' into contract-op-code-sanity-check
MitchTurner Nov 2, 2023
0769262
Cleanup comments, update CHANGELOG
MitchTurner Nov 2, 2023
7cc0428
Fix smo
MitchTurner Nov 2, 2023
c51a092
Comment out `tro`
MitchTurner Nov 3, 2023
48ad913
Add sequential op code benches but just comment out
MitchTurner Nov 3, 2023
90db502
Fix compilation warnings
MitchTurner Nov 3, 2023
b8195ba
Revert bad fmt
MitchTurner Nov 3, 2023
ef78901
Replace removed changelog entries
MitchTurner Nov 3, 2023
faa299c
Pick some nits
MitchTurner Nov 3, 2023
e8b041d
Add docs to helper functions
MitchTurner Nov 3, 2023
88bab84
Fix name, comment out sequential op bench
MitchTurner Nov 3, 2023
6d952b7
Removed unused function, remove dbg
MitchTurner Nov 3, 2023
72319ee
Appease Clippy-sama
MitchTurner Nov 3, 2023
6ee19be
Update benches/benches/block_target_gas.rs
MitchTurner Nov 8, 2023
8fd7b89
remove dum comments
MitchTurner Nov 8, 2023
d811014
Merge branch 'master' into contract-op-code-sanity-check
MitchTurner Nov 8, 2023
b14aecb
Update CHANGELOG
MitchTurner Nov 8, 2023
f26ec0d
Remove mistake
MitchTurner Nov 8, 2023
f14aa0d
Refactor into factory thing, address some PR reviews
MitchTurner Nov 10, 2023
a3f7060
Add all review changes but `tro` stuff
MitchTurner Nov 14, 2023
afc9488
Merge branch 'master' into contract-op-code-sanity-check
xgreenx Nov 15, 2023
a66e197
Merge remote-tracking branch 'origin/master' into contract-op-code-sa…
xgreenx Nov 15, 2023
a3936c1
WIP fixing `tro`
MitchTurner Nov 15, 2023
b40f617
Merge remote-tracking branch 'origin/contract-op-code-sanity-check' i…
xgreenx Nov 15, 2023
37bfa17
Add incrementing u256 storage key
xgreenx Nov 15, 2023
e24af3e
Use updated costs for sequential opcodes.
xgreenx Nov 15, 2023
7d24aad
Merge branch 'master' into contract-op-code-sanity-check
xgreenx Nov 15, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Description of the upcoming release here.

### Added

- [#1453](https://github.com/FuelLabs/fuel-core/pull/1453): Add the majority of the "sanity" benchmarks for contract opcodes.
- [#1473](https://github.com/FuelLabs/fuel-core/pull/1473): Expose fuel-core version as a constant
- [#1469](https://github.com/FuelLabs/fuel-core/pull/1469): Added support of bloom filter for RocksDB tables and increased the block cache.
- [#1642](https://github.com/FuelLabs/fuel-core/pull/1462): Added benchmark to measure the performance of contract state and contract ID calculation; use for gas costing.
Expand Down Expand Up @@ -76,6 +77,7 @@ Description of the upcoming release here.
- [#1408](https://github.com/FuelLabs/fuel-core/pull/1408): Update gas benchmarks for storage opcodes to use a pre-populated database to get more accurate worst-case costs.

#### Breaking

- [#1464](https://github.com/FuelLabs/fuel-core/pull/1464): Avoid possible truncation of higher bits. It may invalidate the code that truncated higher bits causing different behavior on 32-bit vs. 64-bit systems. The change affects some endpoints that now require lesser integers.
- [#1432](https://github.com/FuelLabs/fuel-core/pull/1432): All subscriptions and requests have a TTL now. So each subscription lifecycle is limited in time. If the subscription is closed because of TTL, it means that you subscribed after your transaction had been dropped by the network.
- [#1407](https://github.com/FuelLabs/fuel-core/pull/1407): The recipient is a `ContractId` instead of `Address`. The block producer should deploy its contract to receive the transaction fee. The collected fee is zero until the recipient contract is set.
Expand Down
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions benches/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ ctrlc = "3.2.3"
ed25519-dalek = { version = "2.0", features = ["rand_core"] }
ethnum = "1.3"
fuel-core = { path = "../crates/fuel-core", default-features = false, features = ["rocksdb-production"] }
fuel-core-chain-config = { workspace = true }
fuel-core-services = { path = "./../crates/services" }
fuel-core-storage = { path = "./../crates/storage" }
fuel-core-sync = { path = "./../crates/services/sync", features = ["benchmarking"] }
Expand Down
228 changes: 222 additions & 6 deletions benches/benches/block_target_gas.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ use rand::SeedableRng;

use ethnum::U256;
use fuel_core_benches::*;
use fuel_core_chain_config::ContractConfig;
use fuel_core_types::{
fuel_asm::{
op,
Expand All @@ -41,8 +42,20 @@ use fuel_core_types::{
secp256r1,
*,
},
fuel_tx::UniqueIdentifier,
fuel_types::AssetId,
fuel_tx::{
ContractIdExt,
Input,
Output,
TxPointer,
UniqueIdentifier,
UtxoId,
},
fuel_types::{
AssetId,
Bytes32,
ContractId,
},
fuel_vm::checked_transaction::EstimatePredicates,
};

mod utils;
Expand All @@ -58,6 +71,10 @@ use utils::{
#[global_allocator]
static GLOBAL: tikv_jemallocator::Jemalloc = tikv_jemallocator::Jemalloc;

const STATE_SIZE: u64 = 10_000_000;
const TARGET_BLOCK_GAS_LIMIT: u64 = 100_000;
const BASE: u64 = 10_000;

fn run(
id: &str,
group: &mut BenchmarkGroup<WallTime>,
Expand All @@ -70,8 +87,6 @@ fn run(
.build()
.unwrap();
let _drop = rt.enter();
const TARGET_BLOCK_GAS_LIMIT: u64 = 100_000;
const BASE: u64 = 10_000;

let database = Database::rocksdb();
let mut config = Config::local_node();
Expand All @@ -92,8 +107,8 @@ fn run(

b.to_async(&rt).iter(|| {
let shared = service.shared.clone();

let tx = fuel_core_types::fuel_tx::TransactionBuilder::script(
// Infinite loop
script.clone().into_iter().collect(),
script_data.clone(),
)
Expand All @@ -102,7 +117,7 @@ fn run(
.add_unsigned_coin_input(
SecretKey::random(&mut rng),
rng.gen(),
u64::MAX,
u64::MAX / 2,
AssetId::BASE,
Default::default(),
Default::default(),
Expand Down Expand Up @@ -138,6 +153,207 @@ fn run(
});
}

/// Sets up a service with a full database. Returns the service with the associated Runtime.
/// The size of the database can be overridden with the `STATE_SIZE` environment variable.
fn service_with_contract_id(
contract_id: ContractId,
) -> (fuel_core::service::FuelService, tokio::runtime::Runtime) {
let rt = tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.unwrap();
Comment on lines +266 to +269
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: Mitch and I talked about this pattern, where the service and runtime are coupled together. It is not ideal and we can investigate ways to clean this up, either as part of this PR, or in a follow up issue.

let _drop = rt.enter();
let mut database = Database::rocksdb();
let mut config = Config::local_node();
config
.chain_conf
.consensus_parameters
.tx_params
.max_gas_per_tx = TARGET_BLOCK_GAS_LIMIT;
config.chain_conf.initial_state.as_mut().unwrap().contracts =
Some(vec![ContractConfig {
contract_id,
code: vec![],
salt: Default::default(),
state: None,
balances: None,
tx_id: None,
output_index: None,
tx_pointer_block_height: None,
tx_pointer_tx_idx: None,
}]);

config
.chain_conf
.consensus_parameters
.predicate_params
.max_gas_per_predicate = TARGET_BLOCK_GAS_LIMIT;
config.chain_conf.block_gas_limit = TARGET_BLOCK_GAS_LIMIT;
config.utxo_validation = false;
config.block_production = Trigger::Instant;

// Override state size if the env var is set
let state_size = std::env::var_os("STATE_SIZE")
.map(|value| {
let value = value.to_str().unwrap();
let value = value.parse::<u64>().unwrap();
println!("Overriding state size with {}", value);
value
})
.unwrap_or(STATE_SIZE);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] Accepting state_size as a parameter to this function, rather than determining it here, would be a better separation of concerns. You could calculate state_size in this way at the call site and pass it in.


database
.init_contract_state(
&contract_id,
(0..state_size).map(|k| {
let mut key = Bytes32::zeroed();
key.as_mut()[..8].copy_from_slice(&k.to_be_bytes());
(key, key)
}),
)
.unwrap();
database
.init_contract_balances(
&contract_id,
(0..state_size).map(|k| {
let key = k / 2;
let mut sub_id = Bytes32::zeroed();
sub_id.as_mut()[..8].copy_from_slice(&key.to_be_bytes());

let asset = if k % 2 == 0 {
VmBench::CONTRACT.asset_id(&sub_id)
} else {
AssetId::new(*sub_id)
};
(asset, key + 1_000)
}),
)
.unwrap();

let service = fuel_core::service::FuelService::new(database, config.clone())
.expect("Unable to start a FuelService");
service.start().expect("Unable to start the service");
(service, rt)
}

/// Runs benchmark for `script` with prepared `service` and specified contract (by 'contract_id') which should be
/// included in service
#[allow(clippy::too_many_arguments)]
fn run_with_service(
id: &str,
group: &mut BenchmarkGroup<WallTime>,
script: Vec<Instruction>,
script_data: Vec<u8>,
service: &fuel_core::service::FuelService,
contract_id: ContractId,
rt: &tokio::runtime::Runtime,
rng: &mut rand::rngs::StdRng,
) {
run_with_service_with_extra_inputs(
id,
group,
script,
script_data,
service,
contract_id,
rt,
rng,
vec![],
vec![],
);
}

/// Runs benchmark for `script` with prepared `service` and specified contract (by `contract_id`) which should be
/// included in service.
/// Also include additional inputs and outputs in transaction
#[allow(clippy::too_many_arguments)]
fn run_with_service_with_extra_inputs(
id: &str,
group: &mut BenchmarkGroup<WallTime>,
script: Vec<Instruction>,
script_data: Vec<u8>,
service: &fuel_core::service::FuelService,
contract_id: ContractId,
rt: &tokio::runtime::Runtime,
rng: &mut rand::rngs::StdRng,
extra_inputs: Vec<Input>,
extra_outputs: Vec<Output>,
) {
group.bench_function(id, |b| {

b.to_async(rt).iter(|| {
let shared = service.shared.clone();


let mut tx_builder = fuel_core_types::fuel_tx::TransactionBuilder::script(
script.clone().into_iter().collect(),
script_data.clone(),
);
tx_builder
.gas_limit(TARGET_BLOCK_GAS_LIMIT - BASE)
.gas_price(1)
.add_unsigned_coin_input(
SecretKey::random(rng),
rng.gen(),
u32::MAX as u64,
AssetId::BASE,
Default::default(),
Default::default(),
);
let input_count = tx_builder.inputs().len();

let contract_input = Input::contract(
UtxoId::default(),
Bytes32::zeroed(),
Bytes32::zeroed(),
TxPointer::default(),
contract_id,
);
let contract_output = Output::contract(input_count as u8, Bytes32::zeroed(), Bytes32::zeroed());

tx_builder
.add_input(contract_input)
.add_output(contract_output);

for input in &extra_inputs {
tx_builder.add_input(input.clone());
}

for output in &extra_outputs {
tx_builder.add_output(*output);
}
let mut tx = tx_builder.finalize_as_transaction();
tx.estimate_predicates(&shared.config.chain_conf.consensus_parameters.clone().into()).unwrap();
async move {
let tx_id = tx.id(&shared.config.chain_conf.consensus_parameters.chain_id);

let mut sub = shared.block_importer.block_importer.subscribe();
shared
.txpool
.insert(vec![std::sync::Arc::new(tx)])
.await
.into_iter()
.next()
.expect("Should be at least 1 element")
.expect("Should include transaction successfully");
let res = sub.recv().await.expect("Should produce a block");
assert_eq!(res.tx_status.len(), 2, "res.tx_status: {:?}", res.tx_status);
assert_eq!(res.sealed_block.entity.transactions().len(), 2);
assert_eq!(res.tx_status[0].id, tx_id);

let fuel_core_types::services::executor::TransactionExecutionResult::Failed {
reason,
..
} = &res.tx_status[0].result
else {
panic!("The execution should fails with out of gas")
};
assert!(reason.contains("OutOfGas"));
}
})
});
}

fn block_target_gas(c: &mut Criterion) {
let mut group = c.benchmark_group("block target estimation");

Expand Down
Loading
Loading