Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

txdag: dag file analyzer #208

Draft
wants to merge 90 commits into
base: feature/TxDAG-PEVM
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
90 commits
Select commit Hold shift + click to select a range
573f766
Feat: Parallel Transaction Execution Implementation
sunny2022da Jul 29, 2024
355a7df
Fix: fix incorrectly set origin storage.
DavidZangNR Jul 3, 2024
f16563a
Feat: TxDAG: support TxDAG
galaio Jul 5, 2024
03eb072
fix several UT with racing issues (#5)
DavidZangNR Jul 16, 2024
0861c3f
TxDAG: support PEVM static dispatch; (#6)
galaio Jul 18, 2024
c2adba6
fix UT test and contention issue (#7)
DavidZangNR Jul 18, 2024
57a249f
Fix: dead lock issue
sunny2022da Jul 18, 2024
a6fd6a9
Fix: avoid update the stateObjects at conflict check phase
sunny2022da Jul 18, 2024
d65873a
TxDAG: support TxDAG transfer, it can be used in QA performance testi…
galaio Jul 24, 2024
0051110
txdag: support write & read TxDAG from file; (#9)
galaio Jul 24, 2024
4e2c10b
FIX: redundancy execution and incorrect merge of dirty object (#12)
DavidZangNR Jul 29, 2024
6952f84
pevm: support delay gas fee calculation & Uts; (#11)
galaio Jul 29, 2024
438f0dd
FIX: issue in fixUpOriginAndResetPendingStorage (#14)
DavidZangNR Jul 29, 2024
e6bc402
Fix: racying issue in fixUpOriginAndResetPendingStorage
sunny2022da Jul 30, 2024
69e9db3
txdag: remove legacy TxDAG transfer logic; (#16)
galaio Jul 30, 2024
f4d9d87
txdag: opt txdag logic & clean todos; (#17)
galaio Jul 31, 2024
d5fdff6
refine lock and fix racying issue (#18)
DavidZangNR Aug 2, 2024
24763c4
txdag: opt TxDAG rwset collecting & generating; (#19)
galaio Aug 2, 2024
fd2a810
contention issue fix (#21)
DavidZangNR Aug 5, 2024
1bd615f
txdag: support multi flags, and supported in pevm; (#22)
galaio Aug 6, 2024
9c3f869
pevm: opt slot trigger mechanism; (#24)
galaio Aug 6, 2024
0eee69b
fix addBalance for delayGasFee (#25)
DavidZangNR Aug 7, 2024
81a22bc
txdag: opt read txdag file and validation logic; (#26)
galaio Aug 8, 2024
2c5c254
pevm: opt read large txdag logic and add conflict metrics; (#29)
galaio Aug 13, 2024
0441024
feat: avoid parallel process for block with few txs
sunny2022da Aug 7, 2024
c6bf957
feat: avoid parallel process for block with few txs
sunny2022da Aug 7, 2024
921469f
fix: avoid rewrite readsCache in slotDB
sunny2022da Aug 8, 2024
07c8ff1
Fix: incorrect GetState of obsoleted data caused by createObject
sunny2022da Aug 8, 2024
a16d6b9
fix: DAG disable access unconfirmedDB
sunny2022da Aug 11, 2024
5dc2adc
do not check special addr 0x1 for destruct
sunny2022da Aug 11, 2024
facc1b2
fix log issue
sunny2022da Aug 12, 2024
a878018
fix: false report of conflict
sunny2022da Aug 13, 2024
029fc23
txDAG transfer (#28)
andyzhang2023 Aug 13, 2024
219e1d1
txdag: using pending writes to accelerate txdag generation, add more …
galaio Aug 13, 2024
bfe5931
feat: code cleanup (#23)
DavidZangNR Aug 13, 2024
64f4388
recover test case
sunny2022da Aug 14, 2024
039ce25
fix ut of txDAG (#32)
andyzhang2023 Aug 14, 2024
ffedb33
txdag: support reset txdag reader when SetHead; (#31)
galaio Aug 14, 2024
837af3b
txdag: fix system tx finalise issue; (#33)
galaio Aug 14, 2024
f9034e2
fix: refine the log level of PEVM (#34)
DavidZangNR Aug 14, 2024
43fe917
mvstates: fix async dep gen deadlock issue & opt mining txdag generat…
galaio Aug 15, 2024
4f5aee3
fix parallel Num (#36)
DavidZangNR Aug 15, 2024
9a5215c
fix: remove unnecessary locks for stateobjects (#37)
DavidZangNR Aug 15, 2024
7d375ff
mvstates: opt async dep generation; (#38)
galaio Aug 16, 2024
0aa6d95
mvstates: fix oom issue when mining is enabled; (#40)
galaio Aug 16, 2024
01283bb
reduce overhead of slotDB initialize (#39)
DavidZangNR Aug 19, 2024
8638a82
mvstates: fix oom issue when mining is enabled;
galaio Aug 16, 2024
7e766fe
Fix: contention issue of Trie for PEVM (#41)
DavidZangNR Aug 20, 2024
554a07f
worker: fix TxDAG generation issues when mining block; (#43)
galaio Aug 20, 2024
311a8b0
pevm-opt: Enable parallel kv conflict check
sunny2022da Aug 21, 2024
10af2f3
pevm-opt: Add conflict check cache
sunny2022da Aug 21, 2024
d03c576
PEVM-fix: avoid checkout old tx in stage2 conflict check
sunny2022da Aug 22, 2024
03c4e9e
pevm: add a side slot to trigger next tx advance;
galaio Aug 23, 2024
44695a1
pevm: fix some bad check & support to fallback to serial processor;
galaio Aug 27, 2024
f26ca12
pevm: fix some bad check & support to fallback to serial processor;
galaio Aug 27, 2024
d830032
async the merge phase
sunny2022da Aug 28, 2024
6f8f4b0
disable parallel if parallel.num is low
sunny2022da Aug 30, 2024
e199776
do CompareAndSwap only necessary
sunny2022da Aug 30, 2024
e0416b5
remove uncessary memory overhead and reuse SyncPool
sunny2022da Sep 2, 2024
f441829
fix putSyncPool and GC issue
sunny2022da Sep 3, 2024
918cf50
parallelDBManager global
sunny2022da Sep 4, 2024
fbfe416
pevm-opt: lock free localstateObjects (#167)
sunny2022da Sep 9, 2024
1f5836f
PEVM-fix: assesslist append and optimize mergeSlotDB (#168)
sunny2022da Sep 10, 2024
9fd6995
PEVM-opt: parallel Txs Prepare (#176)
sunny2022da Sep 12, 2024
ea7cd4e
fix issue after rebase
sunny2022da Sep 25, 2024
ccea87d
fix code hash issue after rebase
sunny2022da Sep 26, 2024
c763509
fix issue in createObject after rebase
sunny2022da Sep 26, 2024
62ddd18
fix: test case issue after rebase (#190)
sunny2022da Sep 29, 2024
6408876
fix getcode issue after rebase
sunny2022da Sep 29, 2024
0622f66
fix trie prefetch issue after rebase
sunny2022da Oct 8, 2024
b40a4f9
PEVM-fix: check in ValidatePlainTxDAG (#195)
sunny2022da Oct 11, 2024
502f75f
Pevm rebased to v0.5.5 (#192)
andyzhang2023 Oct 11, 2024
b6e5253
Fix/pevm v0.5.0 invalid gas used (#199)
andyzhang2023 Oct 18, 2024
f717e27
rename pevm option (#200)
andyzhang2023 Oct 18, 2024
9c7bd05
pevm: unordered merge mode flag (#202)
welkin22 Oct 22, 2024
743ca54
pevm: remove the ParallelLegacy (#201)
sunny2022da Oct 22, 2024
0fb3f73
If TxDAG is nil, then use the serial processor to handle it.
welkin22 Oct 22, 2024
c5f1c9e
Merge pull request #203 from welkin22/feature/TxDAG-PEVM-patch
welkin22 Oct 22, 2024
3973f55
fix: use ParallelTxNum config and uber automaxprocs (#204)
welkin22 Oct 23, 2024
ea3460b
fix: Transient Storage should set its prev value but not delete it wh…
Oct 23, 2024
3af645b
bugfix: it panic when accessing storage from a reverted object which …
Oct 25, 2024
92b2317
bugfix: 'invalid mercle root' after reverting an object which had bee…
Oct 25, 2024
938ac30
UncommittedDB of PEVM shutdown when an invalid snapshot id found (jus…
Oct 25, 2024
8fbb333
fix: do initParallelRunner() only once no matter how many chains are …
Oct 25, 2024
3fd45f3
fixut: init the runner before running the cases of PevmProcessor, oth…
Oct 25, 2024
5ae6ffd
fixut: err should be returned by a seprated method ConflictsToMaindb(…
Oct 25, 2024
d58b8e0
Merge pull request #206 from andyzhang2023/fix/pevm-tstore-selfdestru…
andyzhang2023 Oct 25, 2024
efcc653
increase the conflict counter only if the state of uncommitted db con…
andyzhang2023 Oct 25, 2024
27a8096
update
welkin22 Oct 29, 2024
c8dd01a
add tx count
welkin22 Oct 30, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion cmd/evm/blockrunner.go
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ func blockTestCmd(ctx *cli.Context) error {
fmt.Println(string(state.Dump(nil)))
}
}
}); err != nil {
}, "", true); err != nil {
return fmt.Errorf("test %v: %w", name, err)
}
}
Expand Down
6 changes: 6 additions & 0 deletions cmd/geth/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,12 @@ var (
utils.RollupComputePendingBlock,
utils.RollupHaltOnIncompatibleProtocolVersionFlag,
utils.RollupSuperchainUpgradesFlag,
utils.ParallelTxFlag,
utils.ParallelTxUnorderedMergeFlag,
utils.ParallelTxNumFlag,
utils.ParallelTxDAGFlag,
utils.ParallelTxDAGFileFlag,
utils.ParallelTxDAGSenderPrivFlag,
configFileFlag,
utils.LogDebugFlag,
utils.LogBacktraceAtFlag,
Expand Down
68 changes: 67 additions & 1 deletion cmd/utils/flags.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@ import (
"encoding/hex"
"errors"
"fmt"
"github.com/ethereum/go-ethereum/core/txpool/bundlepool"
"math"
"math/big"
"net"
Expand All @@ -35,6 +34,8 @@ import (
"strings"
"time"

"github.com/ethereum/go-ethereum/core/txpool/bundlepool"

pcsclite "github.com/gballet/go-libpcsclite"
gopsutil "github.com/shirou/gopsutil/mem"
"github.com/urfave/cli/v2"
Expand Down Expand Up @@ -1093,11 +1094,49 @@ Please note that --` + MetricsHTTPFlag.Name + ` must be set to start the server.
Category: flags.MetricsCategory,
}

ParallelTxFlag = &cli.BoolFlag{
Name: "parallel",
Usage: "Enable the experimental parallel transaction execution mode, only valid in full sync mode (default = false)",
Category: flags.VMCategory,
}

ParallelTxUnorderedMergeFlag = &cli.BoolFlag{
Name: "parallel.unordered-merge",
Usage: "Enable unordered merge mode, during the parallel confirm phase, merge transaction execution results without following the transaction order.",
Category: flags.VMCategory,
}

ParallelTxNumFlag = &cli.IntFlag{
Name: "parallel.num",
Usage: "Number of slot for transaction execution, only valid in parallel mode (runtime calculated, no fixed default value)",
Category: flags.VMCategory,
}

ParallelTxDAGFlag = &cli.BoolFlag{
Name: "parallel.txdag",
Usage: "Enable the experimental parallel TxDAG generation, only valid in full sync mode (default = false)",
Category: flags.VMCategory,
}

ParallelTxDAGFileFlag = &cli.StringFlag{
Name: "parallel.txdagfile",
Usage: "It indicates the TxDAG file path",
Value: "./parallel-txdag-output.csv",
Category: flags.VMCategory,
}

VMOpcodeOptimizeFlag = &cli.BoolFlag{
Name: "vm.opcode.optimize",
Usage: "enable opcode optimization",
Category: flags.VMCategory,
}

ParallelTxDAGSenderPrivFlag = &cli.StringFlag{
Name: "parallel.txdagsenderpriv",
Usage: "private key of the sender who sends the TxDAG transactions",
Value: "",
Category: flags.VMCategory,
}
)

var (
Expand Down Expand Up @@ -1983,6 +2022,33 @@ func SetEthConfig(ctx *cli.Context, stack *node.Node, cfg *ethconfig.Config) {
cfg.EnablePreimageRecording = ctx.Bool(VMEnableDebugFlag.Name)
}

if ctx.IsSet(ParallelTxFlag.Name) {
cfg.ParallelTxMode = ctx.Bool(ParallelTxFlag.Name)
}

if ctx.IsSet(ParallelTxUnorderedMergeFlag.Name) {
cfg.ParallelTxUnorderedMerge = ctx.Bool(ParallelTxUnorderedMergeFlag.Name)
}

if ctx.IsSet(ParallelTxNumFlag.Name) {
cfg.ParallelTxNum = ctx.Int(ParallelTxNumFlag.Name)
}

if ctx.IsSet(ParallelTxDAGFlag.Name) {
cfg.EnableParallelTxDAG = ctx.Bool(ParallelTxDAGFlag.Name)
}

if ctx.IsSet(ParallelTxDAGFileFlag.Name) {
cfg.ParallelTxDAGFile = ctx.String(ParallelTxDAGFileFlag.Name)
}

if ctx.IsSet(ParallelTxDAGSenderPrivFlag.Name) {
priHex := ctx.String(ParallelTxDAGSenderPrivFlag.Name)
if cfg.Miner.ParallelTxDAGSenderPriv, err = crypto.HexToECDSA(priHex); err != nil {
Fatalf("Failed to parse txdag private key of %s, err: %v", ParallelTxDAGSenderPrivFlag.Name, err)
}
}

if ctx.IsSet(VMOpcodeOptimizeFlag.Name) {
cfg.EnableOpcodeOptimizing = ctx.Bool(VMOpcodeOptimizeFlag.Name)
if cfg.EnableOpcodeOptimizing {
Expand Down
Loading
Loading