Peer Management Enhancements (issue #488) #491

peterbroadhurst · 2024-12-23T18:16:29Z

Implements #488

Update transport manager with new responsibilities
- Reliable message DB tables
- Peer management / activation
- Peer inactivity reaping / deactivation
- Fire+forget send short retry with Go channels
- Reliable send from local DB tables, using sequence to track unsent messages
- Ack/Nack sending
- Ack/Nack storage
- Move protox.Message to protox.PaladinMsg to remove some confusion in the code with gRPC Message
Reconciling private TX manager data duplicate data structures and redundant code
- Reconcile components.TransportMessage and protox.PaladinMsg
- Move list of Components to protox.PaladinMsg enum called Component
- Change export/import for domain context to include un-written state data
  - Remove state distribution from Complete() on assemble, as states should flow in endorse payload
- Update private TX manager to call new TransportManager APIs
Assembling / storing data types delivered via reliable message types
- StateDistribution
  - Payload contains the domain/schema/id/nullifier info
  - State data itself is not duplicated in the DB, and only stored in the StateManager
- Receipt
- PreparedTransaction

JSON/RPC interface

transport_peers
transport_peerInfo
Add struct docs (and run gradle generateDocs)

Including statistics in the pldapi.PeerInfo structure

paladin/toolkit/go/pkg/pldapi/peerinfo.go

Lines 21 to 38 in 2b62d39

    
           type PeerInfo struct { 
        
           	Name          string            `docstruct:"PeerInfo" json:"name"` 
        
           	Stats         PeerStats         `docstruct:"PeerInfo" json:"stats"` 
        
           	Activated     tktypes.Timestamp `docstruct:"PeerInfo" json:"activated"` 
        
           	Outbound      map[string]any    `docstruct:"PeerInfo" json:"outbound,omitempty"` 
        
           	OutboundError error             `docstruct:"PeerInfo" json:"outboundError,omitempty"` 
        
           } 
        
           type PeerStats struct { 
        
           	SentMsgs            uint64             `docstruct:"PeerStats" json:"sentMsgs"` 
        
           	ReceivedMsgs        uint64             `docstruct:"PeerStats" json:"receivedMsgs"` 
        
           	SentBytes           uint64             `docstruct:"PeerStats" json:"sentBytes"` 
        
           	ReceivedBytes       uint64             `docstruct:"PeerStats" json:"receivedBytes"` 
        
           	LastSend            *tktypes.Timestamp `docstruct:"PeerStats" json:"lastSend"` 
        
           	LastReceive         *tktypes.Timestamp `docstruct:"PeerStats" json:"lastReceive"` 
        
           	ReliableHighestSent uint64             `docstruct:"PeerStats" json:"reliableHighestSent"` 
        
           	ReliableAckBase     uint64             `docstruct:"PeerStats" json:"reliableAckBase"` 
        
           }

Signed-off-by: Peter Broadhurst <[email protected]>

hosie

LGTM. A couple of minor comments but on the whole, this is a great improvement.

core/go/internal/privatetxnmgr/transaction_flow_mutators.go

hosie · 2025-01-06T16:14:59Z

toolkit/go/pkg/plugintk/plugin_type_transport.go

@@ -28,6 +28,8 @@ type TransportAPI interface {
 	ConfigureTransport(context.Context, *prototk.ConfigureTransportRequest) (*prototk.ConfigureTransportResponse, error)
 	SendMessage(context.Context, *prototk.SendMessageRequest) (*prototk.SendMessageResponse, error)
 	GetLocalDetails(context.Context, *prototk.GetLocalDetailsRequest) (*prototk.GetLocalDetailsResponse, error)
+	ActivateNode(context.Context, *prototk.ActivateNodeRequest) (*prototk.ActivateNodeResponse, error)


Minor spelling observation. ActivateNode could be misleading esp. if seen out of context. ActivatePeering might be more accurate.

Good point - went for ActivatePeer/DeactivatePeer

Signed-off-by: Peter Broadhurst <[email protected]>

peterbroadhurst · 2025-01-06T17:35:16Z

Merging is dependent on #498, because of a small tweak to the prepare behavior in this PR - it now only sends the prepared transaction to the address designated as the from in that prepared transaction.

The E2E test / tutorial in example/bond is affected by this, but not after the refactor that was already in plan.

@awrichar and I discussed whether we should remove prepared transaction entirely in this PR, as they no longer have an active role, but we decided to decouple that change.

peterbroadhurst · 2025-01-06T17:40:23Z

Hit intermittent build failure #410 - re-running

Signed-off-by: Peter Broadhurst <[email protected]>

Fix the bond example to retrieve prepared transactions from the node that prepared them, which may not be the node that requested them. TODO: need to revisit if "prepare" is still a valid flow, as it's become complex Signed-off-by: Andrew Richardson <[email protected]>

Signed-off-by: Andrew Richardson <[email protected]>

peterbroadhurst · 2025-01-08T17:29:21Z

Thanks @awrichar for addressing the E2E

peterbroadhurst added 19 commits December 23, 2024 13:15

Initial framework for peer mgmt code and new queued msg tables

e64f2ab

Signed-off-by: Peter Broadhurst <[email protected]>

Spelling for reliable delivery API

beae08f

Signed-off-by: Peter Broadhurst <[email protected]>

Update transport proto interface for activate/deactive lifecycle

9f34a57

Signed-off-by: Peter Broadhurst <[email protected]>

Close out on gRPC transport changes for activate/deactivate

6f15517

Signed-off-by: Peter Broadhurst <[email protected]>

State distribution message building and sender loop

44defaf

Signed-off-by: Peter Broadhurst <[email protected]>

Working through refactor of transport interface

1725f86

Signed-off-by: Peter Broadhurst <[email protected]>

Kick off test reconcile for updated transport manager

6ea9915

Signed-off-by: Peter Broadhurst <[email protected]>

Work through send loop logic

1169abc

Signed-off-by: Peter Broadhurst <[email protected]>

Fix gorm annotations

582b72f

Signed-off-by: Peter Broadhurst <[email protected]>

Original tests all re-instated

3342176

Signed-off-by: Peter Broadhurst <[email protected]>

Reconcile logic for node target validation

92bd1f1

Signed-off-by: Peter Broadhurst <[email protected]>

First real DB test of reliable re-delivery

a8fa88b

Signed-off-by: Peter Broadhurst <[email protected]>

Work through additional scenarios

01f1fa7

Signed-off-by: Peter Broadhurst <[email protected]>

Close out test of send path, ready for receive

dc0ca39

Signed-off-by: Peter Broadhurst <[email protected]>

Allow active peer without sender connected and add stats

7eaa162

Signed-off-by: Peter Broadhurst <[email protected]>

Initial work on receive message processing with acks

253859c

Signed-off-by: Peter Broadhurst <[email protected]>

Peer inactivity reaper

ffb4d12

Signed-off-by: Peter Broadhurst <[email protected]>

Work through DB test of state receipt and ack return

8bdfed8

Signed-off-by: Peter Broadhurst <[email protected]>

Remove quiesce logic now we have simpler reaper based close

d168cbe

Signed-off-by: Peter Broadhurst <[email protected]>

peterbroadhurst mentioned this pull request Jan 3, 2025

Peer management enhancements #488

Open

peterbroadhurst added 10 commits January 3, 2025 10:22

Close out on receive/ack logic

a2123a3

Signed-off-by: Peter Broadhurst <[email protected]>

Add RPC interface for query

6fdfc2d

Signed-off-by: Peter Broadhurst <[email protected]>

Add peer RPC functions to client/docs

2b62d39

Signed-off-by: Peter Broadhurst <[email protected]>

Reconcile interfaces for prepared TX

5fe2739

Signed-off-by: Peter Broadhurst <[email protected]>

Add support for receipt and prepared TX reliable msgs

60f76c4

Signed-off-by: Peter Broadhurst <[email protected]>

Add a ReceivedMessage object for component delivery

55db3c0

Signed-off-by: Peter Broadhurst <[email protected]>

Add activate/deactivate node functions

4c93d19

Signed-off-by: Peter Broadhurst <[email protected]>

Update types in private TX manager and remove redundant logic/interfaces

3962e63

Signed-off-by: Peter Broadhurst <[email protected]>

Fix migrations

f36b2f8

Signed-off-by: Peter Broadhurst <[email protected]>

Allow sender to specify message ID for correlation

0747909

Signed-off-by: Peter Broadhurst <[email protected]>

peterbroadhurst added 4 commits January 5, 2025 20:14

Update DomainContext export/import to be full snapshot with states

3393a64

Signed-off-by: Peter Broadhurst <[email protected]>

Generate nullifiers when writing received states

da143aa

Signed-off-by: Peter Broadhurst <[email protected]>

Merge branch 'main' of https://github.com/kaleido-io/paladin into peers

c1c3d62

Fix SQLite migration

a7c5aa9

Signed-off-by: Peter Broadhurst <[email protected]>

peterbroadhurst marked this pull request as ready for review January 6, 2025 03:00

peterbroadhurst requested a review from hosie January 6, 2025 03:00

peterbroadhurst added 5 commits January 5, 2025 22:03

Fix down migrations for later deletion of table

456023b

Signed-off-by: Peter Broadhurst <[email protected]>

Up coverage

59ae8f9

Signed-off-by: Peter Broadhurst <[email protected]>

Fix client for struct change

133e1fd

Signed-off-by: Peter Broadhurst <[email protected]>

Add docs for peer JSON/RPC apis

2c9655b

Signed-off-by: Peter Broadhurst <[email protected]>

Remove unused fields after activate/deactivate API update

95d72eb

Signed-off-by: Peter Broadhurst <[email protected]>

hosie approved these changes Jan 6, 2025

View reviewed changes

Update naming from review

b06c28e

Signed-off-by: Peter Broadhurst <[email protected]>

peterbroadhurst and others added 3 commits January 6, 2025 14:03

Remove map order indeterminism in test

98619a5

Signed-off-by: Peter Broadhurst <[email protected]>

pente: propagate required signer for prepared transactions

a58fe5d

Signed-off-by: Andrew Richardson <[email protected]>

peterbroadhurst enabled auto-merge January 8, 2025 17:29

Merge branch 'main' into peers

3dcb438

peterbroadhurst merged commit 7d67e28 into main Jan 8, 2025
7 checks passed

peterbroadhurst deleted the peers branch January 8, 2025 17:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Peer Management Enhancements (issue #488) #491

Peer Management Enhancements (issue #488) #491

peterbroadhurst commented Dec 23, 2024 •

edited

Loading

hosie left a comment

hosie Jan 6, 2025

peterbroadhurst Jan 6, 2025

peterbroadhurst commented Jan 6, 2025

peterbroadhurst commented Jan 6, 2025

peterbroadhurst commented Jan 8, 2025

	type PeerInfo struct {
	Name string `docstruct:"PeerInfo" json:"name"`
	Stats PeerStats `docstruct:"PeerInfo" json:"stats"`
	Activated tktypes.Timestamp `docstruct:"PeerInfo" json:"activated"`
	Outbound map[string]any `docstruct:"PeerInfo" json:"outbound,omitempty"`
	OutboundError error `docstruct:"PeerInfo" json:"outboundError,omitempty"`
	}

	type PeerStats struct {
	SentMsgs uint64 `docstruct:"PeerStats" json:"sentMsgs"`
	ReceivedMsgs uint64 `docstruct:"PeerStats" json:"receivedMsgs"`
	SentBytes uint64 `docstruct:"PeerStats" json:"sentBytes"`
	ReceivedBytes uint64 `docstruct:"PeerStats" json:"receivedBytes"`
	LastSend *tktypes.Timestamp `docstruct:"PeerStats" json:"lastSend"`
	LastReceive *tktypes.Timestamp `docstruct:"PeerStats" json:"lastReceive"`
	ReliableHighestSent uint64 `docstruct:"PeerStats" json:"reliableHighestSent"`
	ReliableAckBase uint64 `docstruct:"PeerStats" json:"reliableAckBase"`
	}

Peer Management Enhancements (issue #488) #491

Peer Management Enhancements (issue #488) #491

Conversation

peterbroadhurst commented Dec 23, 2024 • edited Loading

hosie left a comment

Choose a reason for hiding this comment

hosie Jan 6, 2025

Choose a reason for hiding this comment

peterbroadhurst Jan 6, 2025

Choose a reason for hiding this comment

peterbroadhurst commented Jan 6, 2025

peterbroadhurst commented Jan 6, 2025

peterbroadhurst commented Jan 8, 2025

peterbroadhurst commented Dec 23, 2024 •

edited

Loading