Probing for more reliable route fee estimation #8136

hieblmi · 2023-10-31T14:51:16Z

This PR implements #7916

Overview

In this PR we extend the implementation of the graph basedEstimateRouteFee RPC with a payment probe based approach to obtain a more reliable routing fee estimate.

Notes on implementation

The new fee estimation API now optionally accepts an invoice from the target node and a probe payment timeout.
Probing distinguishes two cases, (1) probing to a public destination key specified in the invoice or (2), probing a private invoice destination reachable through public hop hints in the invoice.
If multiple hop hints are provided then, for each of the public ones, a probe is dispatched and each of the fee estimates is returned. Private hints are skipped.
lncli adds support to use the graph-based and probe-based fee estimation:

graph-based:

lncli estimateroutefee --dest 0266a18ed969ef95c8a5aa314b443b2b3b8d91ed1d9f8e95476f5f4647efdec079 --amt 100000

probe-based:

lncli estimateroutefee reg_zane estimateroutefee --pay_req lnbcrt1m1pj... --timeout 60s

JssDWt · 2023-10-31T19:20:47Z

There is currently a problem with prepayment probing in combination with some lightning service providers. Basically prepayment probes are not compatible with the current LSP protocols. Since nodes behind a LSP can be a significant portion of destinations, I think it's good to keep the following considerations in mind. In below context, assume the payee is a lightning node behind a LSP.

In the current state I generally recommend against sending prepayment probes when a LSP is involved. Or if you do really want to probe, probe up to the LSP node, and manually add the fees for the ultimate hop. These fees can be taken from the route hint in the invoice. I guess the question is: how do you determine whether a LSP is involved? Let's outline the protocols and their caveats.

Present: Breez LSP

Prepayment probes with a random payment hash will, in some situations around just-in-time channels, fail with unknown_next_peer returned by the LSP, even though the actual payment with the real payment hash would succeed.
There is an option to re-hash the payment hash prefixed with probing-01, like here. When used with the current Breez LSP, the Breez LSP will return incorrect_or_unknown_payment_details in the same situations as where the previous bullet would return unknown_next_peer.

Soon: LSPS2

Very soon Breez and multiple other LSPs will implement the LSPS2 protocol. This is a protocol for just-in-time channels. This initial version of the protocol is incompatible with prepayment probes. There are no payment hash tricks here.

These nodes should have feature bit 729 (option_supports_lsps) set in their node announcement. Recommendation is to send the prepayment probe to the node in the route hint, rather than the destination node, if this feature bit is set for the route hint node and the destination node is only reachable through the route hint node (quite a mouthful, sorry about that).

Medium term: LSPS4

The LSP spec group is currently in the process of designing a LSP protocol LSPS4 for just-in-time channels. When LSPS4 is fully implemented, prepayment probes won't be much of an issue. This is not going to be implemented soon, however.

Other thoughts

Prepayment probes don't work together with MPP very well. At least there is no indication whether you were able to deliver the full amount for the specified fee, even if all payment parts return with incorrect_or_unknown_payment_details. Point is the parts do not all arrive at the same time, like they would in an actual payment. The previous parts locking up liquidity in the route on an actual payment may very well change the path chosen for next parts.

alexbosworth · 2023-11-05T15:53:41Z

lnrpc/routerrpc/router.proto

+    An invoice of the target node that the route fee request is intended for.
+    Its parameters are input to probe payments that estimate routing fees.
+     */
+    string invoice = 3;


normally these are called payment_request when it's a bolt 11 payreq input

lnrpc/routerrpc/router.proto

hieblmi · 2023-11-06T10:53:12Z

Thanks for the detailed rundown on LSP mechanics @JssDWt.

The primary focus of this PR is to improve the currently active graph based method of estimating routing fees. It is unreliable because the gossip announcements might not accurately reflect the network topology or fees at the time when a payment occurs. So in this first step the desired outcome is to improve the fee estimation for payments to publicly known nodes.

Follow-up PRs will focus on MPP/AMP as noted in the linked issue #7916. This seems to have gotten possible since the release of AMP by invalidating the last shard of a AMP, see #4219 (comment).

I think LSP specific probing protocols will also land with a separate PR.

saubyk · 2023-11-28T17:39:14Z

assigned @bitromortac for approach ack

bitromortac

Nice, I think the api looks good, except that we could return an lnrpc.PaymentFailureReason to find out more about why a probe failed. Also added a suggestion for how to handle the streaming responses differently.

Another important point is backwards compatibility, I think we need to support the old ux for some time, as the new api now has a time component. So I'd suggest to keep the old code with the pubkey/amount input and only use the new estimation if an invoice is provided. This can then be removed after some time when the rpc fields are removed.

lnrpc/routerrpc/router.proto

lnrpc/routerrpc/router_server.go

Roasbeef · 2023-12-13T18:56:45Z

Prepayment probes don't work together with MPP very well. At least there is no indication whether you were able to deliver the full amount for the specified fee,

AMP solves that. You can send the N-1 shards, then make the Nth shared fail by giving it an invalid secret share.

Roasbeef · 2023-12-13T19:27:46Z

@JssDWt

These nodes should have feature bit 729 (option_supports_lsps) set in their node announcement. Recommendation is to send the prepayment probe to the node in the route hint, rather than the destination node, if this feature bit is set for the route hint node and the destination node is only reachable through the route hint node (quite a mouthful, sorry about that).

How will that work if a probing hash is used, and also only a single shard is sent that may be below the actual payment amount? The LSP just decides on a value to use for channel opening? What if the sender doesn't retry again, the LSP just eats the chain fees cost? What about if the user already has a channel open?

JssDWt · 2023-12-13T21:54:45Z

@Roasbeef

Assuming you're asking about probing all the way to the destination node, and not up until the LSP. Let me answer these one by one.

How will that work if a probing hash is used, and also only a single shard is sent that may be below the actual payment amount?

In that case the single shard is failed back by the LSP after a timeperiod similar to the MPP timeout. The recipient's invoice amount is the threshold for letting htlcs through and trigger channel open.

The LSP just decides on a value to use for channel opening?

The recipient communicates the intended amount to receive to the LSP before creating the invoice. The invoice then contains a route hint with a scid specifically for this payment/channel order.

What about if the user already has a channel open?

In that case there are no issues. The destination works like any other lightning node. Only if the recipient ordered a cchannel for that payment will this LSPS2 spec be used.

Roasbeef · 2024-01-05T02:09:23Z

@hieblmi is this ready to leave draft?

hieblmi · 2024-01-05T07:23:50Z

@hieblmi is this ready to leave draft?

@Roasbeef I will move to 'ready' once I have incorporated the hop hint fee estimation and refactor suggestions. Will focus on it.

hieblmi · 2024-02-21T09:47:34Z

Once again thanks for the thorough review @ProofOfKeags + @bitromortac, I've addressed all your comments.
Most notably I restructured isLsp to only check what its name implies, and added prepareLspRouteHints to handle the extraction of adjustedRouteHints which allow probing the LSP. I've also made the requested changes to tests and itests.

This PR is ready for another review round.

hieblmi · 2024-02-21T10:13:13Z

lnrpc/routerrpc/router_server.go

+func maxLspFee(routeHints [][]zpay32.HopHint, amt lnwire.MilliSatoshi) (uint32,
+	uint32) {
+
+	var maxFeePpm uint32


nit: put in var block

bitromortac

I have some more nits and the itests needs an update to test real probing to the receiver which needs to take the bits of the destination node into account (read from the invoice as discussed).

lnrpc/routerrpc/router.proto

lnrpc/routerrpc/router_server.go

lntest/node/harness_node.go

itest/lnd_estimate_route_fee_test.go

bitromortac

LGTM 🎉. There's only one remark left concerning the api docs.

bitromortac · 2024-03-05T07:07:34Z

lnrpc/routerrpc/router.proto

+    take, denoted in seconds. Although the probing function is aborted if the
+    timeout is reached, the underlying payment attempt is not cancelled. This
+    might leave the node with pending outgoing htlcs that wait for resolution.


nit: we should be more accurate here, suggestion:
The probing payment loop is aborted if this timeout is reached. Note that the probing process itself can take longer than the timeout if the HTLC becomes delayed or stuck. Canceling the context of this call will not cancel the payment loop, the duration is only controlled by the timeout parameter.

conf: fee estimation timeout in sample config

In this commit the mission control based fee estimation is supplemented with a payment probe estimation which can lead to more accurate estimation results. The probing utilizes a hop-hint heurisic to detect routes through LSPs in order to manually estimate fees to destinations behind an LSP that would otherwise block the payment probe.

hieblmi · 2024-03-05T08:32:18Z

Once again thank you @bitromortac for this thorough review journey. The handling of route hints by lnd has been especially insightful for me.

hieblmi self-assigned this Oct 31, 2023

hieblmi marked this pull request as draft October 31, 2023 14:51

saubyk linked an issue Oct 31, 2023 that may be closed by this pull request

[feature]: Change EstimateRouteFee rpc to improve fee estimation functionality #7916

Closed

saubyk added this to the v0.18.0 milestone Oct 31, 2023

hieblmi force-pushed the fee-estimation-probe branch 4 times, most recently from e90bf4a to a0e1522 Compare October 31, 2023 17:13

alexbosworth reviewed Nov 5, 2023

View reviewed changes

lnrpc/routerrpc/router.proto Outdated Show resolved Hide resolved

hieblmi force-pushed the fee-estimation-probe branch from a0e1522 to e8ce139 Compare November 6, 2023 09:56

hieblmi force-pushed the fee-estimation-probe branch from e8ce139 to adc76f8 Compare November 7, 2023 12:48

hieblmi force-pushed the fee-estimation-probe branch from adc76f8 to 1cf5900 Compare November 28, 2023 15:21

saubyk requested a review from bitromortac November 28, 2023 17:39

bitromortac reviewed Dec 11, 2023

View reviewed changes

lnrpc/routerrpc/router.proto Outdated Show resolved Hide resolved

lnrpc/routerrpc/router_server.go Outdated Show resolved Hide resolved

hieblmi force-pushed the fee-estimation-probe branch from 1cf5900 to 715f90a Compare January 4, 2024 20:19

hieblmi force-pushed the fee-estimation-probe branch 4 times, most recently from cae529d to 0e85acb Compare January 9, 2024 14:40

hieblmi marked this pull request as ready for review January 9, 2024 14:40

hieblmi force-pushed the fee-estimation-probe branch from 0e85acb to 7806868 Compare January 9, 2024 18:16

hieblmi force-pushed the fee-estimation-probe branch from 6b9fe34 to 5923576 Compare February 21, 2024 09:41

hieblmi requested review from bitromortac and ProofOfKeags February 21, 2024 09:43

hieblmi force-pushed the fee-estimation-probe branch from 5923576 to 6d0617a Compare February 21, 2024 09:48

hieblmi commented Feb 21, 2024

View reviewed changes

ProofOfKeags approved these changes Feb 21, 2024

View reviewed changes

bitromortac reviewed Feb 22, 2024

View reviewed changes

hieblmi force-pushed the fee-estimation-probe branch from 6d0617a to 9d4ea1c Compare February 22, 2024 18:35

hieblmi requested a review from bitromortac February 22, 2024 18:47

hieblmi force-pushed the fee-estimation-probe branch 2 times, most recently from 51eb85d to 5b133ee Compare February 23, 2024 08:24

bitromortac reviewed Feb 29, 2024

View reviewed changes

itest/lnd_estimate_route_fee_test.go Outdated Show resolved Hide resolved

itest/lnd_estimate_route_fee_test.go Outdated Show resolved Hide resolved

itest/lnd_estimate_route_fee_test.go Outdated Show resolved Hide resolved

itest/lnd_estimate_route_fee_test.go Show resolved Hide resolved

hieblmi force-pushed the fee-estimation-probe branch 2 times, most recently from 3d953cd to 49284e4 Compare March 1, 2024 15:40

hieblmi requested a review from bitromortac March 1, 2024 17:31

saubyk mentioned this pull request Mar 3, 2024

Queryroutes function should have an optional source node that's different from the local node Ride-The-Lightning/RTL#1362

Closed

bitromortac approved these changes Mar 5, 2024

View reviewed changes

hieblmi force-pushed the fee-estimation-probe branch from 49284e4 to fba6fda Compare March 5, 2024 08:24

hieblmi added 7 commits March 5, 2024 09:24

routerrpc: overhaul RouteFeeRequest with probing parameters.

721038d

multi: fee estimation timeout parameters

24080c5

conf: fee estimation timeout in sample config

itest: preparatory fee estimation changes

ef069b6

itest: estimate route fee tests

e79b8b2

lncli: add estimateroutefee command

814e1a7

docs: update release notes

fba6fda

guggero merged commit 6cef367 into lightningnetwork:master Mar 5, 2024
24 of 25 checks passed

hieblmi mentioned this pull request Mar 8, 2024

[feature]: Allow for cancellation of EstimateRouteFee #8534

Closed

kaloudis mentioned this pull request Apr 5, 2024

Issues paying from Strike -> ZEUS Pay ZeusLN/zeus#2021

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Probing for more reliable route fee estimation #8136

Probing for more reliable route fee estimation #8136

hieblmi commented Oct 31, 2023 •

edited

Loading

JssDWt commented Oct 31, 2023 •

edited

Loading

alexbosworth Nov 5, 2023

hieblmi commented Nov 6, 2023

saubyk commented Nov 28, 2023

bitromortac left a comment •

edited

Loading

Roasbeef commented Dec 13, 2023

Roasbeef commented Dec 13, 2023

JssDWt commented Dec 13, 2023

Roasbeef commented Jan 5, 2024

hieblmi commented Jan 5, 2024

hieblmi commented Feb 21, 2024

hieblmi Feb 21, 2024

bitromortac left a comment

bitromortac left a comment

bitromortac Mar 5, 2024

hieblmi commented Mar 5, 2024

Probing for more reliable route fee estimation #8136

Probing for more reliable route fee estimation #8136

Conversation

hieblmi commented Oct 31, 2023 • edited Loading

Overview

Notes on implementation

JssDWt commented Oct 31, 2023 • edited Loading

Present: Breez LSP

Soon: LSPS2

Medium term: LSPS4

Other thoughts

alexbosworth Nov 5, 2023

Choose a reason for hiding this comment

hieblmi commented Nov 6, 2023

saubyk commented Nov 28, 2023

bitromortac left a comment • edited Loading

Choose a reason for hiding this comment

Roasbeef commented Dec 13, 2023

Roasbeef commented Dec 13, 2023

JssDWt commented Dec 13, 2023

Roasbeef commented Jan 5, 2024

hieblmi commented Jan 5, 2024

hieblmi commented Feb 21, 2024

hieblmi Feb 21, 2024

Choose a reason for hiding this comment

bitromortac left a comment

Choose a reason for hiding this comment

bitromortac left a comment

Choose a reason for hiding this comment

bitromortac Mar 5, 2024

Choose a reason for hiding this comment

hieblmi commented Mar 5, 2024

hieblmi commented Oct 31, 2023 •

edited

Loading

JssDWt commented Oct 31, 2023 •

edited

Loading

bitromortac left a comment •

edited

Loading