Optimize mempool maps #873
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Previously when adding a replacement tx to the local mempool, when the txid's nested map becomes empty, we forgot to delete the key from the root map. And when adding the same tx to the `txids` map(which happens in ZMQ), we may treat the tx as a replacement tx. These are now fixed and the related unit tests are added/updated to reflect the fix.
This commit moves the coinbase from updating inputs to adding txes so it never enters the mempool's internal state. A unit test is patched to guard the updates.
This commit creates a new interface for `rpcClient.Client` so it can be easily mocked in mempool's unit tests.
This commit removes adding tx to mempool when received from `rawtx` zmq to prevent the same tx gets to be added multiple times since it will be notified multiple times. This also means we need to send the new tx in `mempoolPoller` to make sure the subscriber won't miss the event. The two good cases, 1. subscribe the input, `LookupInputSpend` return false 2. tx seen by `rawtx` or mempool poller 3. notified in `filterTx` 1. tx seen by `rawtx` or mempool poller 2. subscribe the input 3. will be notified via `LookupInputSpend` The bad case is, 1. sent by `rawtx`, which marks the tx as seen in `filterTx` 2. subscribe the input, `LoopupInputSpend` return false 3. sent by the mempool poller, `filterTx` will ignore this tx, thus the subscriber won't be notified
This commit changes the data structure of the cachedInputs to more efficiently store the data.
|
Unfortunately the optimization doesn't work based pprof, will investigate more. |
|
Replaced by #877 which focuses on removing the redundant |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Depends on #872, this PR removes a redundant
Add(tx)for bitcoind ZMQ events, and optimized the memory usage of mempool based on the fact that, multiple inputs may share the same tx hash that leaves room for optimization.