Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement EIP-7805 (FOCIL) #8003

Draft
wants to merge 62 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 33 commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
caafcee
add to release spec, chainspec, engineapi
Marchhill Jan 3, 2025
92523fc
change inclusionlisttransactions type, add to block
Marchhill Jan 3, 2025
a7e59cb
implement GetInclusionListTransactions handler
Marchhill Jan 3, 2025
b71294e
fix tests
Marchhill Jan 3, 2025
df33833
use IL for block building
Marchhill Jan 4, 2025
e2557cd
check that block satisfies IL
Marchhill Jan 4, 2025
1a6c7f1
Merge remote-tracking branch 'upstream/master' into feature/eip-7805
Marchhill Jan 4, 2025
72204a1
fix engine rpc registration, fix old tests, add test
Marchhill Jan 21, 2025
3789381
formatting
Marchhill Jan 21, 2025
2f2abaf
change getInclusionList type to bytes
Marchhill Jan 23, 2025
d7b3dab
fix some tests
Marchhill Jan 23, 2025
dd73442
fix some things and add engineapi tests for inclusion lists
Marchhill Jan 28, 2025
6cf18ad
tidy
Marchhill Jan 28, 2025
99e0963
refactor inclusion list validation to block validator
Marchhill Jan 28, 2025
c26ba5e
add test for inclusion list validation, fix check if tx contained in …
Marchhill Jan 28, 2025
da6e88f
improve efficiency of checking block tx hashes
Marchhill Jan 28, 2025
5e79e5b
fix IL block production and add test
Marchhill Jan 29, 2025
ebd456e
refactor tests and tidy
Marchhill Jan 29, 2025
2300836
use txpooltxsource for inclusion list
Marchhill Jan 29, 2025
4791cb0
whitespace
Marchhill Jan 29, 2025
fd56978
Merge remote-tracking branch 'upstream/master' into feature/eip-7805
Marchhill Jan 30, 2025
274a563
remove merge conflicts
Marchhill Jan 30, 2025
9d58825
remove syncthreadtests
Marchhill Jan 30, 2025
efecc21
small test change
Marchhill Jan 30, 2025
acc500f
fix aura tests
Marchhill Jan 30, 2025
33b6412
move inclusion list validator to separate class
Marchhill Feb 3, 2025
1a00e1c
remove transactionProcessor from blockValidator
Marchhill Feb 3, 2025
180f85a
formatting
Marchhill Feb 3, 2025
8afa7e6
tidy up old tests
Marchhill Feb 3, 2025
2eec53c
tidy
Marchhill Feb 3, 2025
c82a9c5
fix auramergeengine newpayload IL test, improve hash mismatch error m…
Marchhill Feb 3, 2025
e859eca
restore refactor
Marchhill Feb 3, 2025
a9ff72f
revert whitespace
Marchhill Feb 3, 2025
0339125
limit IL size to 8kb
Marchhill Feb 13, 2025
7ccde5e
avoid recalculating header hash
Marchhill Feb 13, 2025
d5e7fad
refactor and decode IL txs earlier
Marchhill Feb 14, 2025
86ea63a
Merge remote-tracking branch 'upstream/master' into feature/eip-7805
Marchhill Feb 14, 2025
b4a285f
Fix formatting
rubo Feb 14, 2025
8f97620
don't decode inclusion list again in txsource
Marchhill Feb 17, 2025
439e725
cleanup IL validator
Marchhill Feb 17, 2025
00fec89
IL satisfied if no gas for simple transfer
Marchhill Feb 17, 2025
e1f004b
use txpooltxsource explicitly in GetInclusionListHandler
Marchhill Feb 17, 2025
f4e0894
max IL size 8192, use ILDecode for encoding, comment on ordering by p…
Marchhill Feb 17, 2025
f1a359c
avoid allocating for creating IL trie, move constants to file
Marchhill Feb 17, 2025
8f1bb99
make txpooltxsource nullable
Marchhill Feb 17, 2025
e2fae0c
add 7805 constants file
Marchhill Feb 17, 2025
4b7cfd1
Merge remote-tracking branch 'upstream/master' into feature/eip-7805
Marchhill Feb 17, 2025
b038b4d
check if InclusionListTransactions null in payloadattributes
Marchhill Feb 17, 2025
b3d1c9b
fix payloadattributes syntax
Marchhill Feb 17, 2025
3c6f62a
fix formatting
Marchhill Feb 17, 2025
54f3baa
fix inclusion list satisfied test
Marchhill Feb 18, 2025
9b34e14
fit as many txs as possible into inclusion list
Marchhill Feb 18, 2025
e10df74
use upper bound estimate for IL packing
Marchhill Feb 18, 2025
9fe0f74
rename getInclusionList to have V1
Marchhill Feb 18, 2025
7acb872
add dummy version updatePayloadWithInclusionList
Marchhill Feb 18, 2025
e8080b9
add new file
Marchhill Feb 18, 2025
93357c7
comment FCU changes
Marchhill Feb 18, 2025
b9d7896
force rebuild payload for updateInclusionListHandler
Marchhill Feb 19, 2025
e0802dc
inclusionlisttxsource pass to update handler and add to additional so…
Marchhill Feb 20, 2025
063d0da
taiko, optimism
Marchhill Feb 20, 2025
d57586f
add cts and ILtxsource to tests
Marchhill Feb 20, 2025
b40e077
INVALID_INCLUSION_LIST error on newpayload
Marchhill Feb 21, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
// SPDX-FileCopyrightText: 2025 Demerzel Solutions Limited
// SPDX-License-Identifier: LGPL-3.0-only

using Nethermind.Consensus.Validators;
using Nethermind.Core;
using Nethermind.Core.Extensions;
using Nethermind.Core.Specs;
using Nethermind.Core.Test.Builders;
using Nethermind.Evm;
using Nethermind.Evm.Tracing;
using Nethermind.Evm.TransactionProcessing;
using Nethermind.Serialization.Rlp;
using Nethermind.Specs.Forks;
using Nethermind.Specs.Test;
using NSubstitute;
using NUnit.Framework;

namespace Nethermind.Blockchain.Test.Validators;

public class InclusionListValidatorTests
{
private ITransactionProcessor _transactionProcessor;
private ISpecProvider _specProvider;
private InclusionListValidator _inclusionListValidator;
private Transaction _validTx;
private byte[] _validTxBytes;

[SetUp]
public void Setup()
{
_transactionProcessor = Substitute.For<ITransactionProcessor>();
_specProvider = new CustomSpecProvider(((ForkActivation)0, Osaka.Instance));
_inclusionListValidator = new InclusionListValidator(
_specProvider,
_transactionProcessor);

_validTx = Build.A.Transaction
.WithGasLimit(100_000)
.WithGasPrice(10.GWei())
.WithNonce(1)
.WithValue(100.Ether())
.WithTo(TestItem.AddressA)
.SignedAndResolved(TestItem.PrivateKeyA)
.TestObject;

_validTxBytes = Rlp.Encode(_validTx).Bytes;
}

[Test]
public void When_block_full_then_accept()
{
var block = Build.A.Block
.WithGasLimit(30_000_000)
.WithGasUsed(30_000_000)
.WithInclusionListTransactions([_validTxBytes])
.TestObject;

bool isValid = _inclusionListValidator.ValidateInclusionList(block, out string? error);
Assert.That(isValid, Is.True);
Assert.That(error, Is.Null);
}

[Test]
public void When_all_inclusion_list_txs_included_then_accept()
{
var block = Build.A.Block
.WithGasLimit(30_000_000)
.WithGasUsed(1_000_000)
.WithTransactions(_validTx)
.WithInclusionListTransactions([_validTxBytes])
.TestObject;

bool isValid = _inclusionListValidator.ValidateInclusionList(block, out string? error);
Assert.Multiple(() =>
{
Assert.That(isValid, Is.True);
Assert.That(error, Is.Null);
});
}

[Test]
public void When_valid_tx_excluded_then_reject()
{
_transactionProcessor.BuildUp(Arg.Any<Transaction>(), Arg.Any<BlockExecutionContext>(), Arg.Any<ITxTracer>())
.Returns(TransactionResult.Ok);

var block = Build.A.Block
.WithGasLimit(30_000_000)
.WithGasUsed(1_000_000)
.WithInclusionListTransactions([_validTxBytes])
.TestObject;

bool isValid = _inclusionListValidator.ValidateInclusionList(block, out string? error);
Assert.Multiple(() =>
{
Assert.That(isValid, Is.False);
Assert.That(error, Is.EqualTo("Block excludes valid inclusion list transaction"));
});
}

[Test]
public void When_no_inclusion_list_then_reject()
{
var block = Build.A.Block
.WithGasLimit(30_000_000)
.WithGasUsed(1_000_000)
.TestObject;

bool isValid = _inclusionListValidator.ValidateInclusionList(block, out string? error);
Assert.Multiple(() =>
{
Assert.That(isValid, Is.False);
Assert.That(error, Is.EqualTo("Block did not have inclusion list"));
});
}
}
1 change: 1 addition & 0 deletions src/Nethermind/Nethermind.Consensus/EngineApiVersions.cs
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,5 @@ public static class EngineApiVersions
public const int Shanghai = 2;
public const int Cancun = 3;
public const int Prague = 4;
public const int Osaka = 5;
}
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ public static string HeaderBlobGasMismatch(ulong? expected, ulong? actual) =>
"RequestsNotEnabled: Requests must be null in block when EIP-6110 and EIP-7002 are not activated.";

public static string InvalidRequestsHash(Hash256? expected, Hash256? actual) =>
$"InvalidRequestsHash: Requests hash hash mismatch in block: expected {expected}, got {actual}";
$"InvalidRequestsHash: Requests hash mismatch in block: expected {expected}, got {actual}";

public const string InvalidRequestsOrder =
"InvalidRequestsOrder: Requests are not in the correct order in block.";
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,8 @@ public partial class BlockProcessor(
private readonly IBlockProcessor.IBlockTransactionsExecutor _blockTransactionsExecutor = blockTransactionsExecutor ?? throw new ArgumentNullException(nameof(blockTransactionsExecutor));
private readonly IBlockhashStore _blockhashStore = blockHashStore ?? throw new ArgumentNullException(nameof(blockHashStore));
private readonly IExecutionRequestsProcessor _executionRequestsProcessor = executionRequestsProcessor ?? new ExecutionRequestsProcessor(transactionProcessor);
private readonly ITransactionProcessor _transactionProcessor = transactionProcessor ?? throw new ArgumentNullException(nameof(transactionProcessor));
private readonly IInclusionListValidator _inclusionListValidator = new InclusionListValidator(specProvider, transactionProcessor);
private Task _clearTask = Task.CompletedTask;

private const int MaxUncommittedBlocks = 64;
Expand Down Expand Up @@ -296,7 +298,12 @@ private void RestoreBranch(Hash256 branchingPointStateRoot)
// TODO: block processor pipeline
private void ValidateProcessedBlock(Block suggestedBlock, ProcessingOptions options, Block block, TxReceipt[] receipts)
{
if (!options.ContainsFlag(ProcessingOptions.NoValidation) && !_blockValidator.ValidateProcessedBlock(block, receipts, suggestedBlock, out string? error))
block.InclusionListTransactions = suggestedBlock.InclusionListTransactions;

if (
!options.ContainsFlag(ProcessingOptions.NoValidation) &&
(!_blockValidator.ValidateProcessedBlock(block, receipts, suggestedBlock, out string? error) ||
!_inclusionListValidator.ValidateInclusionList(block, out error)))
{
if (_logger.IsWarn) _logger.Warn(InvalidBlockHelper.GetMessage(suggestedBlock, "invalid block after processing"));
throw new InvalidBlockException(suggestedBlock, error);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -252,7 +252,7 @@ protected virtual Block PrepareBlock(BlockHeader parent, PayloadAttributes? payl

IEnumerable<Transaction> transactions = _txSource.GetTransactions(parent, header.GasLimit, payloadAttributes);

return new BlockToProduce(header, transactions, Array.Empty<BlockHeader>(), payloadAttributes?.Withdrawals);
return new BlockToProduce(header, transactions, Array.Empty<BlockHeader>(), payloadAttributes?.Withdrawals, payloadAttributes?.InclusionListTransactions);
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,8 @@ protected virtual ITxSource CreateTxSourceForProducer(
ILogManager logManager)
{
TxPoolTxSource txPoolSource = CreateTxPoolTxSource(processingEnv, txPool, blocksConfig, transactionComparerProvider, logManager);
return additionalTxSource.Then(txPoolSource);
InclusionListTxSource inclusionListTxSource = new(_specProvider.ChainId);
return additionalTxSource.Then(txPoolSource).Then(inclusionListTxSource);
}

protected virtual TxPoolTxSource CreateTxPoolTxSource(
Expand All @@ -132,13 +133,7 @@ protected virtual TxPoolTxSource CreateTxPoolTxSource(
IBlocksConfig blocksConfig,
ITransactionComparerProvider transactionComparerProvider,
ILogManager logManager)
{
ITxFilterPipeline txSourceFilterPipeline = CreateTxSourceFilter(blocksConfig);
return new TxPoolTxSource(txPool, _specProvider, transactionComparerProvider, logManager, txSourceFilterPipeline);
}

protected virtual ITxFilterPipeline CreateTxSourceFilter(IBlocksConfig blocksConfig) =>
TxFilterPipelineBuilder.CreateStandardFilteringPipeline(_logManager, _specProvider, blocksConfig);
=> new TxPoolTxSourceFactory(txPool, _specProvider, transactionComparerProvider, blocksConfig, logManager).Create();

protected virtual BlockProcessor CreateBlockProcessor(
IReadOnlyTxProcessingScope readOnlyTxProcessingEnv,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,10 +32,10 @@ public class BlockToProduce : Block
public BlockToProduce(BlockHeader blockHeader,
IEnumerable<Transaction> transactions,
IEnumerable<BlockHeader> uncles,
IEnumerable<Withdrawal>? withdrawals = null)
: base(blockHeader, Array.Empty<Transaction>(), uncles, withdrawals)
IEnumerable<Withdrawal>? withdrawals = null,
IEnumerable<byte[]>? inclusionListTransactions = null)
: base(blockHeader, transactions, uncles, withdrawals, inclusionListTransactions)
{
Transactions = transactions;
}

public override Block WithReplacedHeader(BlockHeader newHeader) => new BlockToProduce(newHeader, Transactions, Uncles, Withdrawals);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@
using Nethermind.Core.Specs;
using Nethermind.State.Proofs;
using Nethermind.Trie;
using System.Linq;
using Nethermind.Serialization.Rlp;

namespace Nethermind.Consensus.Producers;

Expand All @@ -26,6 +28,8 @@ public class PayloadAttributes

public Hash256? ParentBeaconBlockRoot { get; set; }

public byte[][]? InclusionListTransactions { get; set; }

public virtual long? GetGasLimit() => null;

public override string ToString() => ToString(string.Empty);
Expand All @@ -47,6 +51,11 @@ public string ToString(string indentation)
sb.Append($", {nameof(ParentBeaconBlockRoot)} : {ParentBeaconBlockRoot}");
}

if (InclusionListTransactions is not null)
{
sb.Append($", {nameof(InclusionListTransactions)} count: {InclusionListTransactions.Length}");
}

sb.Append('}');

return sb.ToString();
Expand All @@ -71,7 +80,8 @@ protected virtual int ComputePayloadIdMembersSize() =>
+ Keccak.Size // prev randao
+ Address.Size // suggested fee recipient
+ (Withdrawals is null ? 0 : Keccak.Size) // withdrawals root hash
+ (ParentBeaconBlockRoot is null ? 0 : Keccak.Size); // parent beacon block root
+ (ParentBeaconBlockRoot is null ? 0 : Keccak.Size) // parent beacon block root
+ (InclusionListTransactions is null ? 0 : Keccak.Size); // inclusion list transactions root hash

protected static string ComputePayloadId(Span<byte> inputSpan)
{
Expand Down Expand Up @@ -110,6 +120,16 @@ protected virtual int WritePayloadIdMembers(BlockHeader parentHeader, Span<byte>
position += Keccak.Size;
}

if (InclusionListTransactions is not null)
{
Transaction[] txs = InclusionListTransactions.Select(tx => Rlp.Decode<Transaction>(tx)).ToArray();
Hash256 inclusionListTransactionsRootHash = InclusionListTransactions.Length == 0
? PatriciaTree.EmptyTreeHash
: new TxTrie(txs).RootHash;
inclusionListTransactionsRootHash.Bytes.CopyTo(inputSpan.Slice(position, Keccak.Size));
position += Keccak.Size;
}

return position;
}

Expand Down Expand Up @@ -166,6 +186,7 @@ public static class PayloadAttributesExtensions
public static int GetVersion(this PayloadAttributes executionPayload) =>
executionPayload switch
{
{ InclusionListTransactions: not null } => EngineApiVersions.Osaka,
{ ParentBeaconBlockRoot: not null, Withdrawals: not null } => EngineApiVersions.Cancun,
{ Withdrawals: not null } => EngineApiVersions.Shanghai,
_ => EngineApiVersions.Paris
Expand All @@ -174,6 +195,7 @@ public static int GetVersion(this PayloadAttributes executionPayload) =>
public static int ExpectedPayloadAttributesVersion(this IReleaseSpec spec) =>
spec switch
{
{ IsEip7805Enabled: true } => EngineApiVersions.Osaka,
{ IsEip4844Enabled: true } => EngineApiVersions.Cancun,
{ WithdrawalsEnabled: true } => EngineApiVersions.Shanghai,
_ => EngineApiVersions.Paris
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
// SPDX-FileCopyrightText: 2025 Demerzel Solutions Limited
// SPDX-License-Identifier: LGPL-3.0-only

using Nethermind.Config;
using Nethermind.Consensus.Comparers;
using Nethermind.Consensus.Transactions;
using Nethermind.Core.Specs;
using Nethermind.Logging;
using Nethermind.TxPool;

namespace Nethermind.Consensus.Producers
{
public class TxPoolTxSourceFactory
{
private readonly ITxPool _txPool;
private readonly ISpecProvider _specProvider;
private readonly ITransactionComparerProvider _transactionComparerProvider;
private readonly IBlocksConfig _blocksConfig;
private readonly ILogManager _logManager;

public TxPoolTxSourceFactory(
ITxPool txPool,
ISpecProvider specProvider,
ITransactionComparerProvider transactionComparerProvider,
IBlocksConfig blocksConfig,
ILogManager logManager)
{
_txPool = txPool;
_specProvider = specProvider;
_transactionComparerProvider = transactionComparerProvider;
_blocksConfig = blocksConfig;
_logManager = logManager;
}

public virtual TxPoolTxSource Create()
{
ITxFilterPipeline txSourceFilterPipeline = TxFilterPipelineBuilder.CreateStandardFilteringPipeline(_logManager, _specProvider, _blocksConfig);
return new TxPoolTxSource(_txPool, _specProvider, _transactionComparerProvider, _logManager, txSourceFilterPipeline);
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
// SPDX-FileCopyrightText: 2025 Demerzel Solutions Limited
// SPDX-License-Identifier: LGPL-3.0-only

using System;
using System.Collections.Generic;
using System.Linq;
using Nethermind.Consensus.Producers;
using Nethermind.Core;
using Nethermind.Crypto;
using Nethermind.Serialization.Rlp;

namespace Nethermind.Consensus.Transactions;

public class InclusionListTxSource(ulong chainId) : ITxSource
{
private readonly EthereumEcdsa _ecdsa = new(chainId);

public IEnumerable<Transaction> GetTransactions(BlockHeader parent, long gasLimit, PayloadAttributes? payloadAttributes = null)
=> payloadAttributes?.InclusionListTransactions?.Select(tx => DecodeTransaction(tx)) ?? [];

private Transaction DecodeTransaction(ReadOnlySpan<byte> txBytes)
{
Transaction tx = TxDecoder.Instance.Decode(txBytes, RlpBehaviors.SkipTypedWrapping);
tx.SenderAddress = _ecdsa.RecoverAddress(tx, true);
return tx;
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
// SPDX-FileCopyrightText: 2025 Demerzel Solutions Limited
// SPDX-License-Identifier: LGPL-3.0-only

using Nethermind.Core;

namespace Nethermind.Consensus.Validators;

public interface IInclusionListValidator
{
/// <summary>
/// Validates that the block satisfies the inclusion list
/// the <see href="https://eips.ethereum.org/EIPS/eip-7805">EIP-7805</see>.
/// </summary>
/// <param name="inclusionListTransactions">The inclusion list transactions to validate.</param>
/// <param name="block">The block to validate.</param>
/// <param name="error">The validation error message if any.</param>
/// <returns>
/// <c>true</c> if the block's inclusion list is satisfied according to EIP-7805;
/// otherwise, <c>false</c>.
/// </returns>
bool ValidateInclusionList(Block block)
=> ValidateInclusionList(block, out _);

/// <summary>
/// Validates that the block satisfies the inclusion list
/// the <see href="https://eips.ethereum.org/EIPS/eip-7805">EIP-7805</see>.
/// </summary>
/// <param name="inclusionListTransactions">The inclusion list transactions to validate.</param>
/// <param name="block">The block to validate.</param>
/// <param name="error">The validation error message if any.</param>
/// <returns>
/// <c>true</c> if the block's inclusion list is satisfied according to EIP-7805;
/// otherwise, <c>false</c>.
/// </returns>
bool ValidateInclusionList(Block block, out string? error);
}
Loading