Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: minor improvements: logging, comments, locks #726

Merged
merged 127 commits into from
Mar 13, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
127 commits
Select commit Hold shift + click to select a range
8fc733f
feat: socket client uses pririority queue to determine processing order
lklimek Jan 8, 2024
7311bde
chore(consensus): add timing debug to makeProposal
lklimek Jan 8, 2024
4ef1833
build: update go.mod/go.sum
lklimek Jan 8, 2024
ea83b4f
chore: socket client debug
lklimek Jan 10, 2024
98025f6
chore: change abci client priorities
lklimek Jan 11, 2024
0bbe9bd
feat: abci client prometheus support
lklimek Jan 11, 2024
91aa568
fix(e2e): panic when no key provided in e2e test
lklimek Jan 11, 2024
0568723
chore: fix typo
lklimek Jan 11, 2024
73f5c57
chore: add missing NoopCloser
lklimek Jan 11, 2024
e86d67b
chore: abciclient metrics
lklimek Jan 11, 2024
f61dfd5
chore: abci metrics remove SentMessagesTotal which is a duplicate
lklimek Jan 11, 2024
ff8c2f3
refactor!: remove consensus params commit timeout
lklimek Jan 11, 2024
b9c498a
chore: remove unused consensus params - backward-compat way
lklimek Jan 11, 2024
1eefd46
chore: proto abci version revert
lklimek Jan 11, 2024
5e41958
feat(p2p): throttled channel
lklimek Jan 11, 2024
cdde233
feat: limit mempool broadcast to 5/s
lklimek Jan 11, 2024
f356737
refactor(p2p): move chan descs to p2p.channel_params and tune priorities
lklimek Jan 12, 2024
6a142e7
feat: detect non-deterministic ResponsePrepareProposal
lklimek Jan 12, 2024
1f0e9d9
chore: decrease log size
lklimek Jan 12, 2024
5b2a393
chore: more debug
lklimek Jan 12, 2024
6be0eaa
fix(consensus): propose locked block when it is available
lklimek Jan 12, 2024
f75ae58
chore: improve logs
lklimek Jan 15, 2024
51317c5
fix(consensus): ValidBlock must be equal LockedBlock
lklimek Jan 15, 2024
07beb3f
fix(consensus): publish valid block event if it's updated
lklimek Jan 15, 2024
cc457a6
chore: improve log readability
lklimek Jan 15, 2024
b0224ef
chore: log num txs of complete proposal block
lklimek Jan 15, 2024
2109237
chore(mempool): log when txs are processed
lklimek Jan 15, 2024
80f3c9c
chore(mempool): add timeout of checktx for p2p messages
lklimek Jan 15, 2024
66186bd
chore(p2p): remove unneeded stopMtx from MConnection
lklimek Jan 15, 2024
23cb8c6
chore(p2p): further debugging
lklimek Jan 15, 2024
c39ef2c
revert: remove checktx timeout as it needs more careful error handling
lklimek Jan 15, 2024
052af38
feat(abciclient): expire checktx abci calls on messages from p2p
lklimek Jan 16, 2024
ed4c661
feat: socket client prio queue improved
lklimek Jan 16, 2024
cefd014
test: socket client priority queue
lklimek Jan 16, 2024
96d9aa3
chore(mempool): p2p checktx timeout 1s
lklimek Jan 16, 2024
68a9a7a
chore(rpc): broadcasttx timeout 1s
lklimek Jan 16, 2024
1769d2e
chore(kvstore): check ctx.Err in handlers for testing
lklimek Jan 16, 2024
06dd3ce
chore(mempool,rpc): checktx timeout 1s using const
lklimek Jan 16, 2024
f6f934e
fix(consensus): proposed block validity conditions
lklimek Jan 16, 2024
e2bc0b7
chore(mempool): send rate limits
lklimek Jan 16, 2024
5c08a9c
Merge remote-tracking branch 'origin/v0.14-dev' into fix-mempool-spam
lklimek Jan 30, 2024
92e634f
Merge remote-tracking branch 'origin/v0.14-dev' into fix-mempool-spam
lklimek Feb 2, 2024
f336485
chore: fix after merge
lklimek Feb 2, 2024
e0c447c
chore: incoming mempool channel size eq to mempool size
lklimek Feb 2, 2024
cfa50c2
feat: timeout when trying to enqueue incoming mempool messages
lklimek Feb 2, 2024
27caa0a
chore: disable rate limits on mempool
lklimek Feb 2, 2024
310ad0b
chore: fix panic
lklimek Feb 2, 2024
bc991ff
chore: increase enqueue timeout to 10ms
lklimek Feb 2, 2024
3bba22f
chore: improve error desc
lklimek Feb 2, 2024
11791dc
chore: improve logging
lklimek Feb 2, 2024
4a0c564
chore(consensus): a bit better logging
lklimek Feb 5, 2024
3cac6e5
refactor: remove unused LastPrecommits
lklimek Feb 6, 2024
854b145
chore: minor fix of multierror
lklimek Feb 12, 2024
a7c7a44
chore(mempool): change locking of addNewTransaction
lklimek Feb 12, 2024
5c59e34
fix(p2p): EnqueueTimeout breaks unrelated channels
lklimek Feb 12, 2024
1065a66
refactor(mempool): relaxed locking of mempool addNewTransaction
lklimek Feb 12, 2024
b42b653
fix: wrong condition
lklimek Feb 12, 2024
5a05e1d
Merge remote-tracking branch 'origin/v0.14-dev' into fix-mempool-spam
lklimek Feb 12, 2024
5a27b68
Merge branch 'feat/relaxed-mempool-locking' into fix-mempool-spam
lklimek Feb 12, 2024
e2718ef
chore: fix condition again
lklimek Feb 12, 2024
8101f0c
Merge branch 'feat/relaxed-mempool-locking' into fix-mempool-spam
lklimek Feb 12, 2024
63743b6
fix(rpc): broadcastasync is broken due to invalid ctx cancellation
lklimek Feb 12, 2024
e75aeb2
chore(p2p): more logging of disconnect errors
lklimek Feb 13, 2024
03aa814
refactor(p2p): refactor routePeer and mConn shutdown
lklimek Feb 13, 2024
c90576e
chore(p2p): drait EnqueueTimeout chan safely
lklimek Feb 13, 2024
feeae96
fix(mempool): mempool size not decreased on RemoveTxByKey
lklimek Feb 13, 2024
5bf16e2
refactor(mempool): use tmsync.Waker for available txs notifications
lklimek Feb 14, 2024
1870ef9
fix(mempool): mark tx as delivered after it's sent
lklimek Feb 14, 2024
718339a
fix(p2p): peerManager.Disconnected context expires too soon
lklimek Feb 14, 2024
3a37576
doc(config): improve mempool.cache-size docs
lklimek Feb 14, 2024
eb3b0bb
Revert "fix(mempool): mark tx as delivered after it's sent"
lklimek Feb 14, 2024
7f254b9
chore: mempool channel rate limit to 5/sec
lklimek Feb 14, 2024
1d5f72f
chore(mempool): add some TODOs
lklimek Feb 14, 2024
52c6983
test(abciclient): test parallel execution of grpc abci client and server
lklimek Feb 15, 2024
c77d710
chore(mempool): remove unnecessary tx hash calculation
lklimek Feb 15, 2024
e843216
fix(mempool): cancel previous mempool run when starting new one
lklimek Feb 15, 2024
f7f7ce7
feat(p2p): channel recv rate limiting - not tested
lklimek Feb 16, 2024
38f9b36
chore: regenerate mocks
lklimek Feb 16, 2024
623706e
fix(mempool): panic due to uninitialized txsAvailable
lklimek Feb 16, 2024
54e00f7
feat(p2p): channel recv rate limiting - continued
lklimek Feb 16, 2024
0d1c114
chore: improve logging
lklimek Feb 21, 2024
a3c9635
chore: don't use go-deadlock in clist due to performance issues
lklimek Feb 23, 2024
50a8283
chore: increase log level to info for "sleeping" message
lklimek Feb 27, 2024
2aa6d6d
test(consensus): minor improvements to TestEmitNewValidBlockEventOnCo…
lklimek Mar 5, 2024
62ca27a
Merge remote-tracking branch 'origin/v0.14-dev' into fix-mempool-spam
lklimek Mar 5, 2024
0c53728
chore(abciclient): fixes after merge
lklimek Mar 5, 2024
229c387
Squashed commit of the following:
lklimek Mar 5, 2024
ae74ce4
Revert "feat: socket client uses pririority queue to determine proces…
lklimek Mar 6, 2024
3c48f75
Revert "feat: abci client prometheus support"
lklimek Mar 6, 2024
8da9d89
Revert "feat: socket client prio queue improved"
lklimek Mar 6, 2024
20d767f
chore: revert priority queue - manual fixes
lklimek Mar 6, 2024
5995351
Merge remote-tracking branch 'origin/v0.14-dev' into fix-mempool-spam
lklimek Mar 6, 2024
54f7951
test(e2e): increase TestApp_TxTooBig timeout to 2 mins
lklimek Mar 6, 2024
ca0b50f
chore: fix linter warnings
lklimek Mar 6, 2024
787ea9c
chore: fix linter warnings
lklimek Mar 6, 2024
40d36b1
chore: fix linter warnings
lklimek Mar 6, 2024
bb54a2f
Merge branch 'chore/fix-linter' into fix-mempool-spam
lklimek Mar 6, 2024
3cd0919
Merge remote-tracking branch 'origin/v0.14-dev' into fix-mempool-spam
lklimek Mar 7, 2024
0aa9937
feat(p2p): throttled channel
lklimek Jan 11, 2024
dfab3a4
feat: limit mempool broadcast to 5/s
lklimek Jan 11, 2024
c06656c
feat(p2p): channel recv rate limiting - not tested
lklimek Feb 16, 2024
73286e9
feat(p2p): channel recv rate limiting - continued
lklimek Feb 16, 2024
c128637
chore(p2p): regen channel mocks
lklimek Mar 8, 2024
5ffb024
feat(config): mempool tx-send-rate-limit, tx-recv-rate-limit, tx-recv…
lklimek Mar 8, 2024
aa82ab9
chore: lint
lklimek Mar 8, 2024
7fef019
chore(mempool): burst recv twice as big as burst send
lklimek Mar 8, 2024
1c1631b
chore: lint
lklimek Mar 8, 2024
7d6aba7
chore: remove not needed log
lklimek Mar 8, 2024
9d537f0
Merge remote-tracking branch 'origin/v0.14-dev' into fix-mempool-spam
lklimek Mar 12, 2024
ae5d8a6
chore: fix after merge
lklimek Mar 12, 2024
50dfc1c
Merge remote-tracking branch 'origin/v0.14-dev' into feat/rate-limiti…
lklimek Mar 12, 2024
e75088d
chore: fixes after merge
lklimek Mar 12, 2024
f7e507d
revert(kvstore): kvstore ctx.Done() checks are not needed
lklimek Mar 12, 2024
d2bf391
refactor(p2p): move chan descs to p2p.channel_params and tune priorities
lklimek Jan 12, 2024
cc6bfde
chore: fix after merge
lklimek Mar 12, 2024
5a10b36
Merge branch 'refactor/move-channel-definitions' into fix-mempool-spam
lklimek Mar 12, 2024
33f54c2
chore(p2p): remove some logs
lklimek Mar 12, 2024
9e5cf6c
chore(statesync): fix linter issues - remove unused consts
lklimek Mar 12, 2024
a90f1b1
Merge branch 'refactor/move-channel-definitions' into fix-mempool-spam
lklimek Mar 12, 2024
5b44f5a
chore: self-review
lklimek Mar 12, 2024
3749b9d
Merge remote-tracking branch 'origin/v0.14-dev' into fix-mempool-spam
lklimek Mar 12, 2024
5fb90a7
Merge remote-tracking branch 'origin/v0.14-dev' into fix-mempool-spam
lklimek Mar 13, 2024
39dedb6
chore: fix linter issues
lklimek Mar 13, 2024
4a4e28b
Merge remote-tracking branch 'origin/v0.14-dev' into fix-mempool-spam
lklimek Mar 13, 2024
8e527cd
chore: lint
lklimek Mar 13, 2024
2fb9bf7
fix(sync): race condition in ConcurrentSlice debug
lklimek Mar 13, 2024
9eeddf9
fix(consensus): invalid block height gossiped to lagging peer
lklimek Mar 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
chore: improve log readability
  • Loading branch information
lklimek committed Jan 15, 2024
commit cc457a6a4b50bb395883b5b414618967eb8de0a6
25 changes: 25 additions & 0 deletions internal/state/current_round_state.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import (
tmbytes "github.com/dashpay/tenderdash/libs/bytes"
tmtypes "github.com/dashpay/tenderdash/proto/tendermint/types"
"github.com/dashpay/tenderdash/types"
"github.com/rs/zerolog"
)

const (
Expand Down Expand Up @@ -236,6 +237,30 @@ func (rp RoundParams) ToProcessProposal() *abci.ResponseProcessProposal {
}
}

func (rp *RoundParams) MarshalZerologObject(e *zerolog.Event) {
if rp == nil {
e.Bool("nil", true)
return
}

e.Str("app_hash", rp.AppHash.ShortString())

e.Int("tx_results_len", len(rp.TxResults))
for i, txResult := range rp.TxResults {
e.Interface(fmt.Sprintf("tx_result[%d]", i), txResult)
if i >= 20 {
e.Str("tx_result[...]", "...")
break
}
}

e.Interface("consensus_param_updates", rp.ConsensusParamUpdates)
e.Interface("validator_set_update", rp.ValidatorSetUpdate)
e.Interface("core_chain_lock", rp.CoreChainLock)
e.Str("source", rp.Source)
e.Int32("round", rp.Round)
}

// RoundParamsFromPrepareProposal creates RoundParams from ResponsePrepareProposal
func RoundParamsFromPrepareProposal(resp *abci.ResponsePrepareProposal, round int32) (RoundParams, error) {
rp := RoundParams{
Expand Down
11 changes: 11 additions & 0 deletions types/block.go
Original file line number Diff line number Diff line change
Expand Up @@ -1034,6 +1034,17 @@ func (data *Data) ToProto() tmproto.Data {
return *tp
}

func (data *Data) MarshalZerologObject(e *zerolog.Event) {
if data == nil {
e.Bool("nil", true)
return
}

e.Str("hash", data.Hash().ShortString())
e.Int("num_txs", len(data.Txs))
e.Array("txs", &data.Txs)
}

// DataFromProto takes a protobuf representation of Data &
// returns the native type.
func DataFromProto(dp *tmproto.Data) (Data, error) {
Expand Down
11 changes: 11 additions & 0 deletions types/core_chainlock.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import (

"github.com/dashpay/tenderdash/crypto"
tmproto "github.com/dashpay/tenderdash/proto/tendermint/types"
"github.com/rs/zerolog"
)

type CoreChainLock struct {
Expand Down Expand Up @@ -107,6 +108,16 @@ func (cl *CoreChainLock) IsZero() bool {
return cl == nil || (len(cl.CoreBlockHash) == 0 && len(cl.Signature) == 0 && cl.CoreBlockHeight == 0)
}

func (cl *CoreChainLock) MarshalZerologObject(e *zerolog.Event) {
if cl == nil {
e.Bool("nil", true)
return
}
e.Hex("core_block_hash", cl.CoreBlockHash)
e.Uint32("core_block_height", cl.CoreBlockHeight)
e.Hex("signature", cl.Signature)
}

// FromProto sets a protobuf Header to the given pointer.
// It returns an error if the chain lock is invalid.
func CoreChainLockFromProto(clp *tmproto.CoreChainLock) (*CoreChainLock, error) {
Expand Down
15 changes: 15 additions & 0 deletions types/tx.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import (
"github.com/dashpay/tenderdash/crypto/merkle"
tmbytes "github.com/dashpay/tenderdash/libs/bytes"
tmproto "github.com/dashpay/tenderdash/proto/tendermint/types"
"github.com/rs/zerolog"
)

// Tx is an arbitrary byte array.
Expand Down Expand Up @@ -103,6 +104,20 @@ func (txs Txs) ToSliceOfBytes() [][]byte {
return txBzs
}

func (txs *Txs) MarshalZerologArray(e *zerolog.Array) {
if txs == nil {
return
}

for i, tx := range *txs {
e.Str(tx.Hash().ShortString())
if i >= 20 {
e.Str("...")
return
}
}
}

// TxRecordSet contains indexes into an underlying set of transactions.
// These indexes are useful for validating and working with a list of TxRecords
// from the PrepareProposal response.
Expand Down
Loading