multi: add BuildOnion, SendOnion, and TrackOnion RPCs

[calvinrzachman]

Change Description

Initial exploration into possible approach for the addition of new SendOnion (and associated) RPCs to the daemon.

• Opted for very slim wrapper around direct delivery of UpdateAddHTLC to the HTLCSwitch for forwarding, eg: no extra tracking by way of ChannelRouter and the ControlTower structures. This may be suitable given intended use by central proxy component with instantiated ChannelRouter component, but does put more burden on users of the RPC which do not re-use code for attempt tracking/persistence in this manner.

TODO

• Decide whether its's preferable to allow users of SendOnion to specify the first hop by public key and map from that to a valid short_channel_id on the server. Messed around with some htlcswitch package interfaces for this and might be worthy of another pass to consider the options here.
• Figure out the communication of error information (eg: code + failure message) via RPC.
• Finalize contents of RPC proto messages.
• moar tests

[github-actions]

Pull reviewers stats

Stats of the last 30 days for lnd:

	User	Total reviews	Time to review	Total comments

[calvinrzachman]

Trying to figure out the error handling across RPC boundary between server and client. Seems we could either use the gRPC status package or the (Send/Track)OnionResponse proto message itself to transport "rich error detail" (code + message) between server and client.

Depending on the RPC client (eg: lightning proxy with central instantiation of ChannelRouter, self-sovereign client with hosted node operator service acting as oblivious sender, etc.), these errors would be handled differently.

[calvinrzachman]

NOTE: Maybe this is the wrong way to go about this as I think I am duplicating the checks done within the Switch internally. Also, should the routerrpc server define it's own interfaces locally within package? Then it could specify exactly what behavior it requires and we wouldn't have to modify the interfaces of the htlcswitch package 🤔

I think all we need is a simple mapping from public key to any scid. I don't think we actually need to do the additional checks as the Switch will pick the correct channel to forward over internally.

[yyforyongyu]

I'm still learning the goals here - so in this test setup, we always want to have Bob being the first hop, and the rest can be anyone?

If the goal is to ensure payment goes through Alice -> Bob, I think we can do something like this instead,

1. Bob queries a route to Dave via QueryRoutes.
2. Alice inserts a hop in the above route, and builds the full path via BuildRoute
3. Alice sends the payment via SendToRouteV2.

[calvinrzachman]

I would say the goal is more to verify that Alice's node is able to correctly accept and initiate a payment using the SendOnion RPC. This is how I have conceptualized the differences between various payment sending RPCS:

• SendPaymentV2:
  • Accepts either an invoice or a destination public key in the case of a key send.
  • The payment route is computed by the daemon.
  • Onion is computed by the daemon.
  • Caller does not have privacy from daemon operator.
• SendToRouteV2:
  • Accepts a route - a list of hops in JSON - to use for the payment.
  • Payment route is computed by the caller (implies access to channel graph).
  • Onion is computed by the daemon.
  • Caller does not have privacy from daemon operator.
• SendOnion:
  • Payment route is computed by the caller.
  • Onion is computed by the caller.
  • Caller does have privacy from daemon operator (?) The daemon just forwards the onion out the first hop!

There are a couple envisioned uses for a SendOnion style RPC that I know of:

1. If we allow the ChannelRouter to run in a separate process from the main lnd binary, then path finding would be done by this external process (eg: lightning proxy) and then the onion delivered to lnd.
   • NOTE: This is the result of trying to re-use the ChannelRouter code with minimal changes. It is possible that there are other ways to do this, however they may require either heavier lift/change set to the ChannelRouter or that we avoid re-using the existing ChannelRouter code all together.
2. A hosted node provider could run the node on behalf of its customers. These customers could perform path finding and onion construction on their own device and then submit the onions to be sent by their node running in the hosted infrastructure. I believe this is what Core Lightning calls "oblivious sends" and they market as a way to maintain privacy from the provider in something like Green Light.

[yyforyongyu]

Here are my thoughts on how to enable SendToOnion as a third option to send payments, aside from sending directly or using a customized route.

There's this method we can change sendAttempt, in which we can move the onion construction outside of this method

lnd/routing/payment_lifecycle.go

Lines 676 to 681 in 399ea86

	// Generate the raw encoded sphinx packet to be included along
	// with the htlcAdd message that we send directly to the
	// switch.
	onionBlob, _, err := generateSphinxPacket(
	&rt, attempt.Hash[:], attempt.SessionKey(),
	)

And I think we need to save the onion in db, which means we need a new field in HTLCAttemptInfo, to make sure we can resume sending the payment from restart.

The rest would be similar to this sendtoRoute - the easiest way is probably to add a third param onionBlob here:

lnd/routing/payment_lifecycle.go

Lines 578 to 579 in 399ea86

	func (p *paymentLifecycle) registerAttempt(rt *route.Route,
	remainingAmt lnwire.MilliSatoshi) (*channeldb.HTLCAttempt, error) {

This means the htlc attempt is still managed by lnd, and users can easily build their own MPP via this new API.