Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Additional testing for tracker and tx modules - mostly around serialization/deserialization #30

Merged
merged 70 commits into from
Mar 11, 2024
Merged
Changes from 1 commit
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
73e3e0e
AsyncTxStrategy should do one of the following during execute: raise …
derekpierre Mar 1, 2024
013279c
Adjust strategies to accomodate latest paradigm for execute i.e. retu…
derekpierre Mar 1, 2024
9c2ec6e
Adjust existing strategy tests since None can now be returned from ex…
derekpierre Mar 1, 2024
39b4b36
Update timeout strategy logging.
derekpierre Mar 3, 2024
cf88810
Update speedup strategy calcs to use math.ceil instead of round
derekpierre Mar 3, 2024
1c01980
Move from a fixed max tip to a max tip factor based on the current su…
derekpierre Mar 4, 2024
352f74a
Simplify legacy transaction speedup logic.
derekpierre Mar 4, 2024
0be4f52
Add tests for constructor parameters and legacy tx speed up functiona…
derekpierre Mar 4, 2024
c023dea
Update logging of gas conditions.
derekpierre Mar 4, 2024
979f3f9
Better annotations about what the strategy is doing.
derekpierre Mar 4, 2024
9589151
Don't modify tx nonce as part of strategy.
derekpierre Mar 4, 2024
b66f5af
Fix outdated comment about value of warn factor.
derekpierre Mar 4, 2024
bf13194
Clean up logging logic since old_tip, old_max_fee may or may not be p…
derekpierre Mar 4, 2024
f7bd316
Add tests for eip1559 transaction speedup.
derekpierre Mar 4, 2024
699d38f
Rename speed up strategy; while the rate value is fixed the updates a…
derekpierre Mar 4, 2024
3755667
Use constant instead of "maxPriorityFeePerGas".
derekpierre Mar 4, 2024
16c2ed3
Use constant for minimum required speedup increase percentage.
derekpierre Mar 4, 2024
1c67055
Use a max between a low default value and warn factor calc in case ti…
derekpierre Mar 4, 2024
f8cf36b
Add todo about best way of setting a cap on the speedup strategy.
derekpierre Mar 4, 2024
1f1fb28
Use reactor to determine when hook is called instead of repeatedly ca…
derekpierre Mar 4, 2024
abb8555
Add TODO to determine whether strategies can be overriden.
derekpierre Mar 4, 2024
5491a6e
Update logging category for monitor when no longer tracking tx.
derekpierre Mar 5, 2024
2a9c0b5
Use enable/disabling of auto mining for better testing.
derekpierre Mar 5, 2024
436ce64
Fix nonce logging message for strategies.
derekpierre Mar 5, 2024
c7851ae
Add the ability to update the tracker's active pending tx after a retry.
derekpierre Mar 5, 2024
7e647c0
Ensure that "from" value in tx params matches/equals signer.
derekpierre Mar 5, 2024
283ba70
Make __fire reusable by both broadcast and strategize - it is now lim…
derekpierre Mar 5, 2024
bd27cbf
Add tests to ensure that strategies are employed when a tx hasn't alr…
derekpierre Mar 5, 2024
25ad0b5
Add testing for "from" tx parameter handling when a tx is being queued.
derekpierre Mar 6, 2024
adac77d
Add test for strategies which do not make updates to parameters.
derekpierre Mar 6, 2024
dc6cda3
Add testing statements to ensure that the broadcast hook is also call…
derekpierre Mar 6, 2024
28f67f4
txhash is not optionally returned from __fire - either it is or an ex…
derekpierre Mar 6, 2024
a7eff18
Potential specialized handling of error cases when firing a tx; this …
derekpierre Mar 6, 2024
9ea695c
Commonize check for recoverable type of error after calling sendRawTr…
derekpierre Mar 6, 2024
f42264d
Insufficient funds is really a special case and its own exception, no…
derekpierre Mar 6, 2024
9013c0d
Fix firing of hook with only ordered args.
derekpierre Mar 6, 2024
f5ecc5a
Add test for unrecoverable errors encoutered when trying to broadcast…
derekpierre Mar 6, 2024
4905e07
Add test for recoverable errors encoutered when trying to broadcast a…
derekpierre Mar 7, 2024
d695078
Add test for unrecoverable errors encoutered when trying to retry a tx.
derekpierre Mar 7, 2024
e000b64
Retry failures should be treated differently than broadcast failures.…
derekpierre Mar 7, 2024
b744a93
Make methods on TxTracker public instead of protected.
derekpierre Mar 7, 2024
431462e
Use a counter for tracking number of requeue attempts for failed txs …
derekpierre Mar 7, 2024
290fd65
Use a counter for tracking number of failed retry attempts when using…
derekpierre Mar 7, 2024
5c42f80
Update existing tests given limits on requeue/retry (redo) attempts.
derekpierre Mar 7, 2024
189b2dd
Add test for exceeding retries when encountering a broadcast error.
derekpierre Mar 7, 2024
f4670ac
Add additional assertions for the requeue counter in existing tests.
derekpierre Mar 7, 2024
c370dea
Existing retry test cleanup.
derekpierre Mar 7, 2024
f59aa74
Add test for exceeding retries when encountering retry errors.
derekpierre Mar 7, 2024
86734c3
Ensure that retry_failure_counter is cleared for all exit cases fault…
derekpierre Mar 7, 2024
190ed94
Rename constant to _MAX_REDO_ATTEMPTS.
derekpierre Mar 7, 2024
f820474
Add TODO.
derekpierre Mar 7, 2024
a2ea01c
Track num requeues and num retries directly on FutureTx and PendingTx…
derekpierre Mar 7, 2024
6f0e150
Strategies must be explicitly provided by the user, if not, no strate…
derekpierre Mar 7, 2024
008c308
There will always be one TimeoutStrategy used by the machine. The tim…
derekpierre Mar 7, 2024
b76111f
Update error messages when non-active tx provided for retry update/fa…
derekpierre Mar 8, 2024
3f52121
Get hook after ensuring active tx is set.
derekpierre Mar 8, 2024
7d5b060
Inital tests for tracker.
derekpierre Mar 8, 2024
b017820
Set retries value of copy instead of incrementing it - just safer.
derekpierre Mar 8, 2024
153161f
Rename handle retry failure parameter for clarity.
derekpierre Mar 8, 2024
e8e704c
Ensure that hook is given time to be called by using a deferred with …
derekpierre Mar 8, 2024
f36fb9d
Don't forget to pop the tx before morphing in tests.
derekpierre Mar 8, 2024
018b0e2
Remove the storage of txdata from PendingTx; it isn't used at all.
derekpierre Mar 8, 2024
a0f2c74
Fix bug in restoring queue values for _TxTracker.
derekpierre Mar 8, 2024
1914e6b
Move params out of AsyncTx and into FutureTx and PendingTx, the other…
derekpierre Mar 8, 2024
7185531
TxParams which has Union data types can take different forms so to en…
derekpierre Mar 8, 2024
faa51ed
Fix tests that used hook to wait using reactor, and enable inlineCall…
derekpierre Mar 8, 2024
084e1e2
Add tests for commit/restore functionality for _TxTracker.
derekpierre Mar 8, 2024
2b52a48
Ensure that hash comparisons of txs are also made as part of tests.
derekpierre Mar 8, 2024
e1f312e
Add serialization/deserialization test for FaultedTx. May/may not rea…
derekpierre Mar 8, 2024
3245c0f
Add test about trying to perform finalize when there isn't an existin…
derekpierre Mar 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add tests for eip1559 transaction speedup.
derekpierre committed Mar 4, 2024

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
commit f7bd316194e411b5c2fd612fc5172bb3d5136226
254 changes: 251 additions & 3 deletions tests/test_strategy.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
import math
import random
from datetime import datetime, timedelta
from unittest.mock import PropertyMock

import pytest
from hexbytes import HexBytes
@@ -104,19 +106,18 @@ def test_speedup_strategy_constructor(w3):
assert speedup_strategy.speedup_factor == (1 + speedup_increase)
assert speedup_strategy.max_tip_factor == max_tip_factor

assert speedup_strategy.name == "speedup"


def test_speedup_strategy_legacy_tx(w3, legacy_transaction, mocker):
speedup_percentage = 0.112 # 11.2%
speedup_strategy = FixedRateSpeedUp(
w3, speedup_increase_percentage=speedup_percentage
)
assert speedup_strategy.name == "speedup"

pending_tx = mocker.Mock(spec=PendingTx)
pending_tx.id = 1

# legacy transaction - keep speeding up

# generated gas price < existing tx gas price
generated_gas_price = legacy_transaction["gasPrice"] - 1 # < what is in tx
mocker.patch.object(w3.eth, "generate_gas_price", return_value=generated_gas_price)
@@ -164,3 +165,250 @@ def test_speedup_strategy_legacy_tx(w3, legacy_transaction, mocker):
generated_gas_price * (1 + speedup_percentage)
)
assert updated_tx_params["nonce"] == old_nonce


def test_speedup_strategy_eip1559_tx_no_blockchain_change(
w3, eip1559_transaction, mocker
):
# blockchain conditions have not changed since
(
current_base_fee,
max_tip_factor,
pending_tx,
speedup_percentage,
speedup_strategy,
suggested_tip,
) = eip1559_setup(mocker, w3)

old_max_priority_fee_per_gas = eip1559_transaction["maxPriorityFeePerGas"]
old_max_fee_per_gas = eip1559_transaction["maxFeePerGas"]

pending_tx.params = dict(eip1559_transaction)
tx_params = speedup_strategy.execute(pending_tx)

updated_max_priority_fee_per_gas = tx_params["maxPriorityFeePerGas"]
assert updated_max_priority_fee_per_gas > old_max_priority_fee_per_gas
assert updated_max_priority_fee_per_gas == math.ceil(
old_max_priority_fee_per_gas * (1 + speedup_percentage)
)

updated_max_fee_per_gas = tx_params["maxFeePerGas"]
assert updated_max_fee_per_gas > old_max_fee_per_gas
assert updated_max_fee_per_gas == math.ceil(
old_max_fee_per_gas * (1 + speedup_percentage)
)
assert updated_max_fee_per_gas >= (
current_base_fee + updated_max_priority_fee_per_gas
)


def test_speedup_strategy_eip1559_tx_base_fee_decreased(
w3, eip1559_transaction, mocker
):
# 2) suppose current base fee decreased; calc remains simple
(
current_base_fee,
max_tip_factor,
pending_tx,
speedup_percentage,
speedup_strategy,
suggested_tip,
) = eip1559_setup(mocker, w3)

old_max_priority_fee_per_gas = eip1559_transaction["maxPriorityFeePerGas"]
old_max_fee_per_gas = eip1559_transaction["maxFeePerGas"]

new_current_base_fee = math.ceil(current_base_fee * 0.95)
mocker.patch.object(
w3.eth, "get_block", return_value={"baseFeePerGas": new_current_base_fee}
)

pending_tx.params = dict(eip1559_transaction)
tx_params = speedup_strategy.execute(pending_tx)

updated_max_priority_fee_per_gas = tx_params["maxPriorityFeePerGas"]
assert updated_max_priority_fee_per_gas > old_max_priority_fee_per_gas
assert updated_max_priority_fee_per_gas == math.ceil(
old_max_priority_fee_per_gas * (1 + speedup_percentage)
)

updated_max_fee_per_gas = tx_params["maxFeePerGas"]
assert updated_max_fee_per_gas > old_max_fee_per_gas
assert updated_max_fee_per_gas == math.ceil(
old_max_fee_per_gas * (1 + speedup_percentage)
)
assert updated_max_fee_per_gas >= (
new_current_base_fee + updated_max_priority_fee_per_gas
)


def test_speedup_strategy_eip1559_tx_base_fee_increased(
w3, eip1559_transaction, mocker
):
# suppose current base fee increase; it gets used instead of prior value
(
current_base_fee,
max_tip_factor,
pending_tx,
speedup_percentage,
speedup_strategy,
suggested_tip,
) = eip1559_setup(mocker, w3)

old_max_priority_fee_per_gas = eip1559_transaction["maxPriorityFeePerGas"]
old_max_fee_per_gas = eip1559_transaction["maxFeePerGas"]

new_current_base_fee = math.ceil(current_base_fee * 1.1)
mocker.patch.object(
w3.eth, "get_block", return_value={"baseFeePerGas": new_current_base_fee}
)

pending_tx.params = dict(eip1559_transaction)
tx_params = speedup_strategy.execute(pending_tx)

updated_max_priority_fee_per_gas = tx_params["maxPriorityFeePerGas"]
assert updated_max_priority_fee_per_gas > old_max_priority_fee_per_gas
assert updated_max_priority_fee_per_gas == math.ceil(
old_max_priority_fee_per_gas * (1 + speedup_percentage)
)

updated_max_fee_per_gas = tx_params["maxFeePerGas"]
assert updated_max_fee_per_gas > old_max_fee_per_gas
assert updated_max_fee_per_gas > math.ceil(
old_max_fee_per_gas * (1 + speedup_percentage)
)
assert updated_max_fee_per_gas == math.ceil(
new_current_base_fee * (1 + speedup_percentage)
+ updated_max_priority_fee_per_gas
)


def test_speedup_strategy_eip1559_tx_no_max_fee(w3, eip1559_transaction, mocker):
# suppose no maxFeePerGas specified - use default calc (same as web3py)
(
current_base_fee,
max_tip_factor,
pending_tx,
speedup_percentage,
speedup_strategy,
suggested_tip,
) = eip1559_setup(mocker, w3)

old_max_priority_fee_per_gas = eip1559_transaction["maxPriorityFeePerGas"]

mocker.patch.object(
w3.eth, "get_block", return_value={"baseFeePerGas": current_base_fee}
)
tx_params = dict(eip1559_transaction)
del tx_params["maxFeePerGas"]

pending_tx.params = tx_params
tx_params = speedup_strategy.execute(pending_tx)

updated_max_priority_fee_per_gas = tx_params["maxPriorityFeePerGas"]
assert updated_max_priority_fee_per_gas == math.ceil(
old_max_priority_fee_per_gas * (1 + speedup_percentage)
)

updated_max_fee_per_gas = tx_params["maxFeePerGas"]
assert updated_max_fee_per_gas == math.ceil(
(current_base_fee * 2) + updated_max_priority_fee_per_gas
)


def test_speedup_strategy_eip1559_tx_no_max_priority_fee(
w3, eip1559_transaction, mocker
):
# suppose no maxPriorityFeePerGas specified - use suggested tip
(
current_base_fee,
max_tip_factor,
pending_tx,
speedup_percentage,
speedup_strategy,
suggested_tip,
) = eip1559_setup(mocker, w3)

mocker.patch.object(
w3.eth, "get_block", return_value={"baseFeePerGas": current_base_fee}
)
tx_params = dict(eip1559_transaction)
del tx_params["maxPriorityFeePerGas"]

pending_tx.params = tx_params
tx_params = speedup_strategy.execute(pending_tx)

updated_max_priority_fee_per_gas = tx_params["maxPriorityFeePerGas"]
assert updated_max_priority_fee_per_gas == math.ceil(
suggested_tip * (1 + speedup_percentage)
)

updated_max_fee_per_gas = tx_params["maxFeePerGas"]
assert updated_max_fee_per_gas == math.ceil(
current_base_fee * (1 + speedup_percentage) + updated_max_priority_fee_per_gas
)


def test_speedup_strategy_eip1559_tx_hit_max_tip(w3, eip1559_transaction, mocker):
(
current_base_fee,
max_tip_factor,
pending_tx,
speedup_percentage,
speedup_strategy,
suggested_tip,
) = eip1559_setup(mocker, w3)

pending_tx.params = dict(eip1559_transaction)
expected_num_iterations_before_hitting_max_tip = math.ceil(
math.log(max_tip_factor) / math.log(1 + speedup_percentage)
)

# do one less iteration than expected
for i in range(expected_num_iterations_before_hitting_max_tip - 1):
tx_params = speedup_strategy.execute(pending_tx)
assert tx_params is not None
assert tx_params["maxPriorityFeePerGas"] <= (suggested_tip * max_tip_factor)
# update params
pending_tx.params = tx_params

# next attempt should cause tip to exceed max tip factor
assert (pending_tx.params["maxPriorityFeePerGas"] * (1 + speedup_percentage)) > (
suggested_tip * max_tip_factor
)
tx_params = speedup_strategy.execute(pending_tx)
# hit max factor so no more changes - return None
assert (
tx_params is None
), f"no updates after {expected_num_iterations_before_hitting_max_tip} iterations"


def eip1559_setup(mocker, w3):
speedup_percentage = round(random.randint(110, 230) / 1000, 3) # [11%, 23%]
max_tip_factor = random.randint(2, 5)
speedup_strategy = FixedRateSpeedUp(
w3,
speedup_increase_percentage=speedup_percentage,
max_tip_factor=max_tip_factor,
)
pending_tx = mocker.Mock(spec=PendingTx)
pending_tx.id = 1
pending_tx.txhash = HexBytes("0xdeadbeef")

# use consistent mocked values
current_base_fee = w3.eth.get_block("latest")["baseFeePerGas"]
suggested_tip = w3.eth.max_priority_fee
mocker.patch.object(
w3.eth, "get_block", return_value={"baseFeePerGas": current_base_fee}
)
mocker.patch(
"web3.eth.eth.Eth.max_priority_fee", PropertyMock(return_value=suggested_tip)
)
return (
current_base_fee,
max_tip_factor,
pending_tx,
speedup_percentage,
speedup_strategy,
suggested_tip,
)