[bugfix] Process fee updates as any other update message #2397

halseth · 2019-01-02T13:04:54Z

Instead of special casing the UpdateFee messages, we instead add them to
the update logs like any other HTLC update message. This lets us avoid
having to keep an extra set of variables to keep track of the fee
updates, and instead reuse the commit/ack logic used for other updates.

This fixes a bug where we would reset the pendingFeeUpdate variable
after signing our next commitment, which would make us calculate the new
fee incorrectly if the remote sent a commitment concurrently.

A test to exercise the previous failure case is added.

Fixes #2348
Fixes #1908

halseth · 2019-01-02T13:08:17Z

Possibly also fixes #1908

Roasbeef

Solid set of changes! We should've implemented it like this from the very start, the resulting logic is also much easier to follow as there're no longer any special cases for fee updates.

Roasbeef · 2019-01-02T20:38:34Z

lnwallet/channel.go

@@ -421,6 +421,11 @@ func PayDescsFromRemoteLogUpdates(chanID lnwire.ShortChannelID, height uint64,
 					Index:  uint16(i),
 				},
 			}
+
+		// NOTE: UpdateFee is not expected since they are not forwarded.


Return an error instead of panicing perhaps?

lnwallet/channel.go

Roasbeef · 2019-01-02T20:58:47Z

lnwallet/channel.go

-	theirBalance *lnwire.MilliSatoshi, nextHeight uint64,
-	remoteChain, mutateState bool) *htlcView {
+	theirBalance *lnwire.MilliSatoshi, nextHeight uint64, remoteChain,
+	mutateState bool, feePerKw SatPerKWeight) (*htlcView, SatPerKWeight) {


An alternative to returning an extra parameter would be to add a feePerKw field directly to the htlcView. This seems more natural as the fee rate itself is a part of the view as it results from evaluating the log.

Yes, that's much better!

Roasbeef · 2019-01-08T01:30:59Z

New set of tests failures with the latest rebase.

Roasbeef

It looks like the fee is being accounted for twice causing us to overdraw the amount in channel in the case of a potential close.

Roasbeef · 2019-01-08T01:34:25Z

lnwallet/channel.go

+	// is denominated in msat we won't lose precision when storing the
+	// sat/kw denominated feerate.
+	case *lnwire.UpdateFee:
+		pd = &PaymentDescriptor{


addCommitHeightRemote isn't set. We need to properly record this in order to handle the retransmission case.

--- FAIL: TestChannelRetransmissionFeeUpdate (0.13s) channel_test.go:3998: unable to complete bob's state transition: rejected commitment: commit_height=2, invalid_commit_sig=30440220682b6dd35e569da6cbe26568e5388bae4c76a9ed61162087bc77471c4558ae9102207a8dad8ffd4a8297c8f50463c2c8a1177ded151f2ea003ee8bedf83f20bdfc1b, commit_tx=0200000001b794385f2d1ef7ab4d9273d1906381b44f2f6f2588a3efb96a491883319847530000000000d1af1c8003204e0000000000002200204e62432fb93d883e0910926ed09e70556f2ca6d62fa74096d016fc389ec55cdbe016cd1d00000000220020d7ba6761e1ac17cbb0722c5818bb61d312ea660cf3505f0eeef4b39f81a2d4330065cd1d000000001600144e8187664a78afec5b8c6991dbc52d575ef5e27116539e20, sig_hash=9ffec9eac9de4aaeb680da3c95bb650468f48a2b57f7ef7eac81caa5bd6863ee

You're right. Test failure was due to 516a6a7, which is fixed in 821cb23

Roasbeef · 2019-01-08T01:42:03Z

lnwallet/channel.go

+	// Initiate feePerKw to the last committed fee for this chain as we'll
+	// need this to determine which HTLCs are dust, and also the final fee
+	// rate.
+	view.feePerKw = commitChain.tip().feePerKw


This seems incorrect. We only know the final fee rate for this view after we fully evaluate it (as we do below).

Yeah, reverted this, as it was the culprit of the test failures

lnwallet/channel.go

halseth · 2019-01-08T10:24:24Z

One of the fixup commits introduced the travis test failure. Should be fixed now. Reminder to self: check travis before squashing commits!

cfromknecht

Very clean PR! Gave this a thorough pass and LGTM. Made sure to double check that these changes are backwards compatible with the forwarding packages, which I believe they are! Stoked to have a fix in for this bug 🎉

cfromknecht · 2019-01-10T02:55:27Z

lnwallet/channel_test.go

@@ -1999,6 +1999,88 @@ func TestUpdateFeeFail(t *testing.T) {

 }

+// TestUpdateFeeConcurrentSig tests that the channel can properly handle a fee
+// update that it receives concurrently with signing its next commitment.
+func TestUpdateFeeConcurrentSig(t *testing.T) {


Solid test!

cfromknecht · 2019-01-10T02:56:49Z

lnwallet/channel.go

+
+	// For fee updates we'll create a FeeUpdate type to add to the log. We
+	// reuse the amount field to hold the fee rate. Since the amount field
+	// is denominated in msat we won't lose precision when storing the


I was wondering this myself until I looked it up :)

Roasbeef · 2019-01-10T03:20:46Z

lnwallet/channel.go

@@ -1571,7 +1571,8 @@ func (lc *LightningChannel) logUpdateToPayDesc(logUpdate *channeldb.LogUpdate,
 			Amount: lnwire.NewMSatFromSatoshis(
 				btcutil.Amount(wireMsg.FeePerKw),
 			),
-			EntryType: FeeUpdate,
+			EntryType:                FeeUpdate,
+			removeCommitHeightRemote: commitHeight,


Shouldn't this be add?

It should indeed.... Fixed!

Thninking a bit more about this, maybe it should really be remove.. To make the feeUpdates being removed when compacting logs, then the remove height must be set. lmk what you think.

Since FeeUpdates doesn't really have the same property of getting added and removed at different heights, I decided to just set both fields at the same time.

When compacting logs we'll now properly remove it if the remove height has passed: 277949c

Also added a test to check that it gets removed during log compacting: e6ee835

Cool yeah it's slightly different since it doesn't have the same add/remove semantics as the others. However, I think we can emulate that based on the two add heights. Nice was going to recommend we extend the tests to ensure that the log is empty after fully processing a fee update, will check out those extra commits!

Yeah I thought about this as well, but it makes sense since we want the fee update to be "insta-removed" in a sense, plus there's so secondary message to cleanup the feeupdate add. might it be correct to set both to the same height?

This commit adds a new updateType that can be used for PaymentDescriptors: FeeUpdate. We repurpose the fields of the existing PaymentDescriptor struct such that we can easily re-use the commit/ack logic used for other update types also for fee updates.

This commit adds conversion between the lnwire.UpdateFee message and the new FeeUpdate PaymentDescriptor. We re-purpose the existing Amount field in the PaymentDescriptor stuct to hold the feerate.

When compacting the update logs we remove any fee updates when they remove height is passed. We do this since we'll assume fee updates are added and removed at the same commit height, as they will apply for all commitments following the fee update.

This commit makes the evaluateHTLCView method process any found FeeUpdates in the logs, by returning the last set feerate.

Instead of special casing the UpdateFee messages, we instead add them to the update logs like any other HTLC update message. This lets us avoid having to keep an extra set of variables to keep track of the fee updates, and instead reuse the commit/ack logic used for other updates. This fixes a bug where we would reset the pendingFeeUpdate variable after signing our next commitment, which would make us calculate the new fee incorrectly if the remote sent a commitment concurrently. When restoring state logs, we also make sure to re-add any fee updates.

This tests make sure we don't reset our expected fee upate after signing our next commitment. This test would fail without the previous set of commits.

Since the feerate is part of the computed view now, we don't have to pass the found feerate as a distinct parameter.

…ck-in

Roasbeef

LGTM 🐉

Moved onto testing the diff as is on the faucet as well now.

cfromknecht

LGTM 🎩

cfromknecht · 2019-01-12T02:48:06Z

lnwallet/channel.go

@@ -1229,6 +1232,16 @@ func compactLogs(ourLog, theirLog *updateLog,
 			if remoteChainTail >= htlc.removeCommitHeightRemote &&
 				localChainTail >= htlc.removeCommitHeightLocal {

+				// Fee updates have no parent htlcs, so we only


Roasbeef · 2019-01-15T01:06:12Z

Faucet testing went off without a hitch, ready to land!

Roasbeef added commitments Commitment transactions containing the state of the channel P2 should be fixed if one has time bug fix labels Jan 2, 2019

Roasbeef reviewed Jan 3, 2019

View reviewed changes

halseth force-pushed the reject-commitment-expected-fee-reset-bug branch 2 times, most recently from 71ce90b to 5e97cbb Compare January 7, 2019 21:38

Roasbeef reviewed Jan 8, 2019

View reviewed changes

lnwallet/channel.go Show resolved Hide resolved

halseth force-pushed the reject-commitment-expected-fee-reset-bug branch from 5e97cbb to 821cb23 Compare January 8, 2019 09:58

cfromknecht previously approved these changes Jan 10, 2019

View reviewed changes

Roasbeef reviewed Jan 10, 2019

View reviewed changes

halseth dismissed cfromknecht’s stale review via 233597c January 10, 2019 07:41

halseth force-pushed the reject-commitment-expected-fee-reset-bug branch from cb48abf to 233597c Compare January 10, 2019 07:41

halseth force-pushed the reject-commitment-expected-fee-reset-bug branch from 233597c to 567a41f Compare January 10, 2019 11:24

halseth added 2 commits January 10, 2019 12:26

lnwallet/channel: add lnwire<->PaymentDescriptor FeeUpdate conversion

480f43f

This commit adds conversion between the lnwire.UpdateFee message and the new FeeUpdate PaymentDescriptor. We re-purpose the existing Amount field in the PaymentDescriptor stuct to hold the feerate.

halseth force-pushed the reject-commitment-expected-fee-reset-bug branch from 567a41f to 3e59d39 Compare January 10, 2019 11:26

halseth added 5 commits January 10, 2019 12:31

lnwallet/channel: process FeeUpdate found in update log

810c56c

This commit makes the evaluateHTLCView method process any found FeeUpdates in the logs, by returning the last set feerate.

lnwallet/channel test: add TestUpdateFeeConcurrentSig

7e01525

This tests make sure we don't reset our expected fee upate after signing our next commitment. This test would fail without the previous set of commits.

lnwallet/channel: don't return feePerKw from computeView

dadfcef

Since the feerate is part of the computed view now, we don't have to pass the found feerate as a distinct parameter.

lnwallet/channel_test: assert FeeUpdate is removed from logs after lo…

e6ee835

…ck-in

halseth force-pushed the reject-commitment-expected-fee-reset-bug branch from 3e59d39 to e6ee835 Compare January 10, 2019 11:31

Roasbeef approved these changes Jan 12, 2019

View reviewed changes

cfromknecht approved these changes Jan 12, 2019

View reviewed changes

Roasbeef merged commit 9c59ac4 into lightningnetwork:master Jan 15, 2019

halseth mentioned this pull request Jan 29, 2019

Channel restoration of old format fee update #2558

Merged

Roasbeef mentioned this pull request Dec 13, 2019

LND sends an invalid sig on reconnection #3822

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[bugfix] Process fee updates as any other update message #2397

[bugfix] Process fee updates as any other update message #2397

halseth commented Jan 2, 2019 •

edited by Roasbeef

Loading

halseth commented Jan 2, 2019

Roasbeef left a comment

Roasbeef Jan 2, 2019

halseth Jan 4, 2019

Roasbeef Jan 2, 2019

halseth Jan 4, 2019

Roasbeef commented Jan 8, 2019

Roasbeef left a comment

Roasbeef Jan 8, 2019

Roasbeef Jan 8, 2019

halseth Jan 8, 2019

Roasbeef Jan 8, 2019

halseth Jan 8, 2019

halseth commented Jan 8, 2019

cfromknecht left a comment

cfromknecht Jan 10, 2019

cfromknecht Jan 10, 2019

Roasbeef Jan 10, 2019

halseth Jan 10, 2019

halseth Jan 10, 2019

halseth Jan 10, 2019

Roasbeef Jan 10, 2019

cfromknecht Jan 10, 2019

Roasbeef left a comment

cfromknecht left a comment

cfromknecht Jan 12, 2019

Roasbeef commented Jan 15, 2019

[bugfix] Process fee updates as any other update message #2397

[bugfix] Process fee updates as any other update message #2397

Conversation

halseth commented Jan 2, 2019 • edited by Roasbeef Loading

halseth commented Jan 2, 2019

Roasbeef left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Roasbeef commented Jan 8, 2019

Roasbeef left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

halseth commented Jan 8, 2019

cfromknecht left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Roasbeef left a comment

Choose a reason for hiding this comment

cfromknecht left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Roasbeef commented Jan 15, 2019

halseth commented Jan 2, 2019 •

edited by Roasbeef

Loading