runtime: validate transactions in parallel

[miloserdow]

In the old flow, validation (including signature checks) happened inside verify_and_charge_transaction(). In the new flow, we first run parallel_validate_transactions() to validate and compute the transaction cost (including signature checks) in parallel. Only after a transaction passes validation do we call verify_and_charge_transaction() with the known cost.

graph TD;
    subgraph Old_Flow
    A[Runtime::apply]
    A-->B[self.process_transactions]
    B-->C[process_transaction]
    C-->D[verify_and_charge_transaction]
    D-->E[validate_transaction incl. signature]
    E-->|Valid|F[charge fees & finalize]
    E-->|Invalid|G[reject tx]
    end

    subgraph New_Flow
    A2[Runtime::apply]
    A2-->H[self.process_transactions]
    H-->|parallel|I[parallel_validate_transactions incl. signature]
    I-->|Valid, get cost|J[process_transaction]
    J-->K[verify_and_charge_transaction]
    K-->L[charge fees & finalize]
    I-->|Invalid|M[reject tx early]
    classDef threaded fill:#3240a8,stroke:#333,stroke-width:2px;
    class I threaded;
    end

Loading

(blue = running in parallel)

Testing

The change brings ~20% throughput improvement on n2d-standard-16

1006/1006 blocks applied in 2m at a rate of 7.5129/s. 0s remaining. Skipped 0 blocks. Over last 100 blocks to height 1073: 0 empty blocks, averaging 413.00 Tgas per non-empty block

1007/1007 blocks applied in 2m at a rate of 8.9972/s. 0s remaining. Skipped 0 blocks. Over last 100 blocks to height 1073: 0 empty blocks, averaging 413.00 Tgas per non-empty block

Additionally, profiles before and after the change are available: before, after

[codecov]

Codecov Report

Attention: Patch coverage is 93.96135% with 25 lines in your changes missing coverage. Please review.

Project coverage is 70.72%. Comparing base (cf45cc5) to head (453eaf2).
Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
runtime/runtime/src/verifier.rs	96.21%	12 Missing ⚠️
.../runtime-params-estimator/src/estimator_context.rs	0.00%	9 Missing ⚠️
chain/chain/src/runtime/mod.rs	93.33%	0 Missing and 2 partials ⚠️
runtime/runtime/src/lib.rs	96.55%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #12654      +/-   ##
==========================================
- Coverage   70.72%   70.72%   -0.01%     
==========================================
  Files         848      848              
  Lines      174212   174303      +91     
  Branches   174212   174303      +91     
==========================================
+ Hits       123219   123278      +59     
- Misses      45850    45880      +30     
- Partials     5143     5145       +2     

Flag	Coverage Δ
backward-compatibility	0.16% <0.00%> (-0.01%)	⬇️
db-migration	0.16% <0.00%> (-0.01%)	⬇️
genesis-check	1.35% <0.00%> (-0.01%)	⬇️
linux	69.23% <91.54%> (-0.01%)	⬇️
linux-nightly	70.30% <93.47%> (-0.01%)	⬇️
pytests	1.65% <0.00%> (-0.01%)	⬇️
sanity-checks	1.46% <0.00%> (-0.01%)	⬇️
unittests	70.55% <93.96%> (-0.01%)	⬇️
upgradability	0.20% <0.00%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[nagisa]

As an additional note, I think it might make sense to re-introduce a helper function that combines the validate_transaction and verify_and_charge_transaction so that the tests and the code that doesn't care about parallel validation of transactions doesn't need to change as much as it did.

All that said, all of this looks quite nice, I didn't see any fundamental issues with the approach taken here. Thank you for working on this :)