Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Shuffle #157

Closed
wants to merge 32 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
7164e58
multi: remove ChainHash from lnwire.NetAddress
ellemouton Nov 8, 2024
d3b76db
graph: remove unused ForEachNode method
ellemouton Nov 8, 2024
2d53fe1
graph: let FetchNodeFeatures take an optional read tx
ellemouton Nov 8, 2024
afb6d02
graph: create an abstract graphdb.RTx interface
ellemouton Nov 8, 2024
f3dc00a
graph: separate ForEachNode and ForEachNodeCBTx
ellemouton Nov 8, 2024
2502c1b
graph: remove read TX from ForEachNodeChannel callback
ellemouton Nov 8, 2024
97d1da6
graph: let NewPathFindTx take a context
ellemouton Nov 8, 2024
69c2110
multi: pass context to AddrSource
ellemouton Nov 8, 2024
0396ea0
graph: let IsPublicNode take a context
ellemouton Nov 8, 2024
71c8832
multi: Contexts for FetchChannelEdgesByID
ellemouton Nov 8, 2024
8eefb54
context for HasLnNode
ellemouton Nov 8, 2024
7c84a49
let FetchLightningNode take a context
ellemouton Nov 8, 2024
501a98a
let ForEachNodeChannel take a context
ellemouton Nov 8, 2024
36065ef
pass context to ForEachNodeDirectedChannel
ellemouton Nov 8, 2024
67601b2
pass context to FetchNodeFeatures
ellemouton Nov 8, 2024
baee817
let FetchChannelEdgesByOutpoint take a context
ellemouton Nov 8, 2024
0e9253c
let LookupAlias take a context
ellemouton Nov 8, 2024
c4cfcbb
let NumZombies take a context
ellemouton Nov 8, 2024
1032de0
let ForEachChannel take a context
ellemouton Nov 8, 2024
f60b645
let ForEachNode take a context
ellemouton Nov 8, 2024
5bd4a6e
add and make use of StatsCollector
ellemouton Nov 8, 2024
38b9dce
invoicesrpc: remove invoicerpc server's access to ChannelGraph pointer
ellemouton Nov 8, 2024
1886f87
multi: add GraphSource interface and GraphProvider to ImplementationC…
ellemouton Nov 3, 2024
d0ccc5c
lnrpc: add IsPublic to lnrpc.NodeInfo
ellemouton Nov 8, 2024
9387dab
graphrpc server
ellemouton Nov 8, 2024
05f7f89
add exclude lists to GetNetworkInfo calls
ellemouton Nov 8, 2024
2fe92e5
graph mux and remote client
ellemouton Nov 8, 2024
f283058
multi: add a `--gossip.no-sync` option
ellemouton Nov 4, 2024
9ab9912
address TODO
ellemouton Nov 8, 2024
cd8dc27
remote graph config options
ellemouton Nov 9, 2024
9acda1c
itest :tada:
ellemouton Nov 9, 2024
75afb5c
no not doing this.
ellemouton Nov 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
multi: add a --gossip.no-sync option
When set, LND will not advertise the gossip queries feature bit and it
will not initiate gossip syncing with any peer.
ellemouton committed Nov 8, 2024
commit f283058c31dc397bf0a9cf9496a7e4014a0a6929
31 changes: 31 additions & 0 deletions discovery/gossiper.go
Original file line number Diff line number Diff line change
@@ -75,6 +75,13 @@ var (
// the remote peer.
ErrGossipSyncerNotFound = errors.New("gossip syncer not found")

// ErrUnexpectedGossipQueries is an error that is returned if we receive
// gossip queries from a peer when we have disabled gossip syncing and
// in other words have not advertised that we support gossip queries.
ErrUnexpectedGossipQueries = errors.New(
"unexpected gossip queries received",
)

// emptyPubkey is used to compare compressed pubkeys against an empty
// byte array.
emptyPubkey [33]byte
@@ -360,6 +367,11 @@ type Config struct {
// updates for a channel and returns true if the channel should be
// considered a zombie based on these timestamps.
IsStillZombieChannel func(time.Time, time.Time) bool

// NoGossipSync is true if gossip syncing has been disabled. It
// indicates that we have not advertised the gossip queries feature bit,
// and so we should not receive any gossip queries from our peers.
NoGossipSync bool
}

// processedNetworkMsg is a wrapper around networkMsg and a boolean. It is
@@ -820,6 +832,25 @@ func (d *AuthenticatedGossiper) ProcessRemoteAnnouncement(msg lnwire.Message,
// TODO(elle): let ProcessRemoteAnnouncement take a context.
ctx := context.TODO()

// If gossip syncing has been disabled, we expect not to receive any
// gossip queries from our peer.
if d.cfg.NoGossipSync {
switch m := msg.(type) {
case *lnwire.QueryShortChanIDs,
*lnwire.QueryChannelRange,
*lnwire.ReplyChannelRange,
*lnwire.ReplyShortChanIDsEnd,
*lnwire.GossipTimestampRange:

log.Warnf("Gossip syncing was disabled, "+
"skipping message: %v", m)
errChan <- ErrUnexpectedGossipQueries

return errChan
default:
}
}

// For messages in the known set of channel series queries, we'll
// dispatch the message directly to the GossipSyncer, and skip the main
// processing loop.
7 changes: 7 additions & 0 deletions feature/manager.go
Original file line number Diff line number Diff line change
@@ -66,6 +66,9 @@ type Config struct {
// NoTaprootOverlay unsets the taproot overlay channel feature bits.
NoTaprootOverlay bool

// NoGossipQueries unsets the gossip queries feature bit.
NoGossipQueries bool

// CustomFeatures is a set of custom features to advertise in each
// set.
CustomFeatures map[Set][]lnwire.FeatureBit
@@ -199,6 +202,10 @@ func newManager(cfg Config, desc setDesc) (*Manager, error) {
raw.Unset(lnwire.SimpleTaprootOverlayChansOptional)
raw.Unset(lnwire.SimpleTaprootOverlayChansRequired)
}
if cfg.NoGossipQueries {
raw.Unset(lnwire.GossipQueriesOptional)
raw.Unset(lnwire.GossipQueriesRequired)
}
for _, custom := range cfg.CustomFeatures[set] {
if custom > set.Maximum() {
return nil, fmt.Errorf("feature bit: %v "+
2 changes: 2 additions & 0 deletions lncfg/gossip.go
Original file line number Diff line number Diff line change
@@ -18,6 +18,8 @@ type Gossip struct {
ChannelUpdateInterval time.Duration `long:"channel-update-interval" description:"The interval used to determine how often lnd should allow a burst of new updates for a specific channel and direction."`

SubBatchDelay time.Duration `long:"sub-batch-delay" description:"The duration to wait before sending the next announcement batch if there are multiple. Use a small value if there are a lot announcements and they need to be broadcast quickly."`

NoSync bool `long:"no-sync" description:"If set, lnd will not request graph updates from its peers and wont advertise the gossip queries feature bit. This is useful if LND has been provided with an external graph source that it may use for pathfinding."`
}

// Parse the pubkeys for the pinned syncers.
8 changes: 8 additions & 0 deletions peer/brontide.go
Original file line number Diff line number Diff line change
@@ -426,6 +426,9 @@ type Config struct {
// used to modify the way the co-op close transaction is constructed.
AuxChanCloser fn.Option[chancloser.AuxChanCloser]

// NoGossipSync is true if we should not sync gossip with this peer.
NoGossipSync bool

// Quit is the server's quit channel. If this is closed, we halt operation.
Quit chan struct{}
}
@@ -869,6 +872,11 @@ func (p *Brontide) Start() error {
// initGossipSync initializes either a gossip syncer or an initial routing
// dump, depending on the negotiated synchronization method.
func (p *Brontide) initGossipSync() {
// If gossip syncing has explicitly been disabled, we'll exit early.
if p.cfg.NoGossipSync {
return
}

// If the remote peer knows of the new gossip queries feature, then
// we'll create a new gossipSyncer in the AuthenticatedGossiper for it.
if p.remoteFeatures.HasFeature(lnwire.GossipQueriesOptional) {
4 changes: 4 additions & 0 deletions sample-lnd.conf
Original file line number Diff line number Diff line change
@@ -1721,6 +1721,10 @@
; be broadcast quickly.
; gossip.sub-batch-delay=5s

; If set, LND will not request graph updates from its peers and wont advertise
; the gossip queries feature bit. This is useful if LND has been provided with
; an external graph source that it may use for pathfinding.
; gossip.no-sync=false

[invoices]

2 changes: 2 additions & 0 deletions server.go
Original file line number Diff line number Diff line change
@@ -578,6 +578,7 @@ func newServer(ctx context.Context, cfg *Config, listenAddrs []net.Addr,
NoTaprootChans: !cfg.ProtocolOptions.TaprootChans,
NoTaprootOverlay: !cfg.ProtocolOptions.TaprootOverlayChans,
NoRouteBlinding: cfg.ProtocolOptions.NoRouteBlinding(),
NoGossipQueries: cfg.Gossip.NoSync,
})
if err != nil {
return nil, err
@@ -4195,6 +4196,7 @@ func (s *server) peerConnected(ctx context.Context, conn net.Conn,
MsgRouter: s.implCfg.MsgRouter,
AuxChanCloser: s.implCfg.AuxChanCloser,
AuxResolver: s.implCfg.AuxContractResolver,
NoGossipSync: s.cfg.Gossip.NoSync,
}

copy(pCfg.PubKeyBytes[:], peerAddr.IdentityKey.SerializeCompressed())

Unchanged files with check annotations Beta

// Next, fetch the node from the database to ensure everything was
// serialized properly.
dbNode, err := graph.FetchLightningNode(testPub)

Check failure on line 128 in graph/db/graph_test.go

GitHub Actions / run unit tests (btcd unit-cover)

not enough arguments in call to graph.FetchLightningNode

Check failure on line 128 in graph/db/graph_test.go

GitHub Actions / run unit tests (unit tags="kvdb_sqlite")

not enough arguments in call to graph.FetchLightningNode

Check failure on line 128 in graph/db/graph_test.go

GitHub Actions / run unit tests (unit tags="kvdb_etcd")

not enough arguments in call to graph.FetchLightningNode

Check failure on line 128 in graph/db/graph_test.go

GitHub Actions / run unit tests (unit tags="kvdb_postgres")

not enough arguments in call to graph.FetchLightningNode

Check failure on line 128 in graph/db/graph_test.go

GitHub Actions / run unit tests (btcd unit-race)

not enough arguments in call to graph.FetchLightningNode
require.NoError(t, err, "unable to locate node")
_, exists, err := graph.HasLightningNode(dbNode.PubKeyBytes)

Check failure on line 131 in graph/db/graph_test.go

GitHub Actions / run unit tests (btcd unit-cover)

not enough arguments in call to graph.HasLightningNode

Check failure on line 131 in graph/db/graph_test.go

GitHub Actions / run unit tests (unit tags="kvdb_sqlite")

not enough arguments in call to graph.HasLightningNode

Check failure on line 131 in graph/db/graph_test.go

GitHub Actions / run unit tests (unit tags="kvdb_etcd")

not enough arguments in call to graph.HasLightningNode

Check failure on line 131 in graph/db/graph_test.go

GitHub Actions / run unit tests (unit tags="kvdb_postgres")

not enough arguments in call to graph.HasLightningNode

Check failure on line 131 in graph/db/graph_test.go

GitHub Actions / run unit tests (btcd unit-race)

not enough arguments in call to graph.HasLightningNode
if err != nil {
t.Fatalf("unable to query for node: %v", err)
} else if !exists {
// Finally, attempt to fetch the node again. This should fail as the
// node should have been deleted from the database.
_, err = graph.FetchLightningNode(testPub)

Check failure on line 152 in graph/db/graph_test.go

GitHub Actions / run unit tests (btcd unit-cover)

not enough arguments in call to graph.FetchLightningNode

Check failure on line 152 in graph/db/graph_test.go

GitHub Actions / run unit tests (unit tags="kvdb_sqlite")

not enough arguments in call to graph.FetchLightningNode

Check failure on line 152 in graph/db/graph_test.go

GitHub Actions / run unit tests (unit tags="kvdb_etcd")

not enough arguments in call to graph.FetchLightningNode

Check failure on line 152 in graph/db/graph_test.go

GitHub Actions / run unit tests (unit tags="kvdb_postgres")

not enough arguments in call to graph.FetchLightningNode

Check failure on line 152 in graph/db/graph_test.go

GitHub Actions / run unit tests (btcd unit-race)

not enough arguments in call to graph.FetchLightningNode
if err != ErrGraphNodeNotFound {
t.Fatalf("fetch after delete should fail!")
}
// Next, fetch the node from the database to ensure everything was
// serialized properly.
dbNode, err := graph.FetchLightningNode(testPub)

Check failure on line 180 in graph/db/graph_test.go

GitHub Actions / run unit tests (btcd unit-cover)

not enough arguments in call to graph.FetchLightningNode

Check failure on line 180 in graph/db/graph_test.go

GitHub Actions / run unit tests (unit tags="kvdb_sqlite")

not enough arguments in call to graph.FetchLightningNode

Check failure on line 180 in graph/db/graph_test.go

GitHub Actions / run unit tests (unit tags="kvdb_etcd")

not enough arguments in call to graph.FetchLightningNode

Check failure on line 180 in graph/db/graph_test.go

GitHub Actions / run unit tests (unit tags="kvdb_postgres")

not enough arguments in call to graph.FetchLightningNode

Check failure on line 180 in graph/db/graph_test.go

GitHub Actions / run unit tests (btcd unit-race)

not enough arguments in call to graph.FetchLightningNode
require.NoError(t, err, "unable to locate node")
_, exists, err := graph.HasLightningNode(dbNode.PubKeyBytes)

Check failure on line 183 in graph/db/graph_test.go

GitHub Actions / run unit tests (btcd unit-cover)

not enough arguments in call to graph.HasLightningNode

Check failure on line 183 in graph/db/graph_test.go

GitHub Actions / run unit tests (unit tags="kvdb_sqlite")

not enough arguments in call to graph.HasLightningNode

Check failure on line 183 in graph/db/graph_test.go

GitHub Actions / run unit tests (unit tags="kvdb_etcd")

not enough arguments in call to graph.HasLightningNode

Check failure on line 183 in graph/db/graph_test.go

GitHub Actions / run unit tests (unit tags="kvdb_postgres")

not enough arguments in call to graph.HasLightningNode

Check failure on line 183 in graph/db/graph_test.go

GitHub Actions / run unit tests (btcd unit-race)

not enough arguments in call to graph.HasLightningNode
if err != nil {
t.Fatalf("unable to query for node: %v", err)
} else if !exists {
// Finally, attempt to fetch the node again. This should fail as the
// node should have been deleted from the database.
_, err = graph.FetchLightningNode(testPub)

Check failure on line 211 in graph/db/graph_test.go

GitHub Actions / run unit tests (btcd unit-cover)

not enough arguments in call to graph.FetchLightningNode

Check failure on line 211 in graph/db/graph_test.go

GitHub Actions / run unit tests (unit tags="kvdb_sqlite")

not enough arguments in call to graph.FetchLightningNode

Check failure on line 211 in graph/db/graph_test.go

GitHub Actions / run unit tests (unit tags="kvdb_etcd")

not enough arguments in call to graph.FetchLightningNode

Check failure on line 211 in graph/db/graph_test.go

GitHub Actions / run unit tests (unit tags="kvdb_postgres")

not enough arguments in call to graph.FetchLightningNode

Check failure on line 211 in graph/db/graph_test.go

GitHub Actions / run unit tests (btcd unit-race)

not enough arguments in call to graph.FetchLightningNode
if err != ErrGraphNodeNotFound {
t.Fatalf("fetch after delete should fail!")
}
// the one which the test node was assigned.
nodePub, err := testNode.PubKey()
require.NoError(t, err, "unable to generate pubkey")
dbAlias, err := graph.LookupAlias(nodePub)

Check failure on line 238 in graph/db/graph_test.go

GitHub Actions / run unit tests (btcd unit-cover)

not enough arguments in call to graph.LookupAlias

Check failure on line 238 in graph/db/graph_test.go

GitHub Actions / run unit tests (unit tags="kvdb_sqlite")

not enough arguments in call to graph.LookupAlias

Check failure on line 238 in graph/db/graph_test.go

GitHub Actions / run unit tests (unit tags="kvdb_etcd")

not enough arguments in call to graph.LookupAlias

Check failure on line 238 in graph/db/graph_test.go

GitHub Actions / run unit tests (unit tags="kvdb_postgres")

not enough arguments in call to graph.LookupAlias

Check failure on line 238 in graph/db/graph_test.go

GitHub Actions / run unit tests (btcd unit-race)

not enough arguments in call to graph.LookupAlias
require.NoError(t, err, "unable to find alias")
if dbAlias != testNode.Alias {
t.Fatalf("aliases don't match, expected %v got %v",
require.NoError(t, err, "unable to create test node")
nodePub, err = node.PubKey()
require.NoError(t, err, "unable to generate pubkey")
_, err = graph.LookupAlias(nodePub)

Check failure on line 250 in graph/db/graph_test.go

GitHub Actions / run unit tests (btcd unit-cover)

not enough arguments in call to graph.LookupAlias

Check failure on line 250 in graph/db/graph_test.go

GitHub Actions / run unit tests (unit tags="kvdb_sqlite")

not enough arguments in call to graph.LookupAlias

Check failure on line 250 in graph/db/graph_test.go

GitHub Actions / run unit tests (unit tags="kvdb_etcd")

not enough arguments in call to graph.LookupAlias

Check failure on line 250 in graph/db/graph_test.go

GitHub Actions / run unit tests (unit tags="kvdb_postgres")

not enough arguments in call to graph.LookupAlias

Check failure on line 250 in graph/db/graph_test.go

GitHub Actions / run unit tests (btcd unit-race)

not enough arguments in call to graph.LookupAlias
if err != ErrNodeAliasNotFound {
t.Fatalf("alias lookup should fail for non-existent pubkey")
}
assertEdgeWithNoPoliciesInCache(t, graph, &edgeInfo)
// Ensure that both policies are returned as unknown (nil).
_, e1, e2, err := graph.FetchChannelEdgesByID(chanID)

Check failure on line 338 in graph/db/graph_test.go

GitHub Actions / run unit tests (btcd unit-cover)

not enough arguments in call to graph.FetchChannelEdgesByID

Check failure on line 338 in graph/db/graph_test.go

GitHub Actions / run unit tests (unit tags="kvdb_sqlite")

not enough arguments in call to graph.FetchChannelEdgesByID

Check failure on line 338 in graph/db/graph_test.go

GitHub Actions / run unit tests (unit tags="kvdb_etcd")

not enough arguments in call to graph.FetchChannelEdgesByID

Check failure on line 338 in graph/db/graph_test.go

GitHub Actions / run unit tests (unit tags="kvdb_postgres")

not enough arguments in call to graph.FetchChannelEdgesByID

Check failure on line 338 in graph/db/graph_test.go

GitHub Actions / run unit tests (btcd unit-race)

not enough arguments in call to graph.FetchChannelEdgesByID
if err != nil {
t.Fatalf("unable to fetch channel edge")
}
// Ensure that any query attempts to lookup the delete channel edge are
// properly deleted.
_, _, _, err = graph.FetchChannelEdgesByOutpoint(&outpoint)

Check failure on line 355 in graph/db/graph_test.go

GitHub Actions / run unit tests (btcd unit-cover)

not enough arguments in call to graph.FetchChannelEdgesByOutpoint

Check failure on line 355 in graph/db/graph_test.go

GitHub Actions / run unit tests (unit tags="kvdb_sqlite")

not enough arguments in call to graph.FetchChannelEdgesByOutpoint

Check failure on line 355 in graph/db/graph_test.go

GitHub Actions / run unit tests (unit tags="kvdb_etcd")

not enough arguments in call to graph.FetchChannelEdgesByOutpoint

Check failure on line 355 in graph/db/graph_test.go

GitHub Actions / run unit tests (unit tags="kvdb_postgres")

not enough arguments in call to graph.FetchChannelEdgesByOutpoint

Check failure on line 355 in graph/db/graph_test.go

GitHub Actions / run unit tests (btcd unit-race)

not enough arguments in call to graph.FetchChannelEdgesByOutpoint
if err == nil {
t.Fatalf("channel edge not deleted")
}
graphSessFactory := newMockGraphSessionFactoryFromChanDB(graph)
graphSess, closeGraphSess, err := graphSessFactory.NewGraphSession()

Check failure on line 3122 in routing/pathfind_test.go

GitHub Actions / check commits

not enough arguments in call to graphSessFactory.NewGraphSession
if err != nil {
return nil, err
}
route, err := session.RequestRoute(
payment.Amount, payment.FeeLimit, 0, height,
lnwire.CustomRecords{

Check failure on line 237 in routing/payment_session_test.go

GitHub Actions / check commits

not enough arguments in call to session.RequestRoute
lnwire.MinCustomRecordsTlvType + 123: []byte{1, 2, 3},
},
)