Spent utxo still listed as unspent

[joostjager]

Background

We have a transaction X that is confirmed on-chain, but somehow its input is still listed by lncli listunspent. Transaction X also isn't listed by lncli listchaintxns. This causes that utxo to be coin-selected by FundPsbt, leading to invalid transactions ('input already spent').

Transaction X was created via FundPsbt and submitted via PublishTransaction. It is a one-in-one-out tx.

It seems that this problem started after a restart of lnd. We monitor the node via lndmon and record the minimum utxo size. The problematic utxo is small. After the restart, we saw the min utxo size dip down to the size of that utxo. As if the restart made lnd forget about the transaction.

Your environment

• lnd v0.13.1
• linux
• btcd

[joostjager]

Is there anything that we can do to obtain relevant data from this node to help pinpoint the problem?

[guggero]

The problem seems to be the single output that only sends to a channel funding TX, which the wallet doesn't recognize as its own. Wasn't aware this issue was already around, we're tracking this in #6434 now.

[guggero]

@joostjager I finally found some time to attempt to reproduce this. So far I wasn't able to. Did you only run into this a single time or do you know a pattern that produces the problem more often?

[joostjager]

We ran into this a single time. Unfortunately no pattern.

[guggero]

@joostjager I found the problem for #6434 today. Turned out that when sending funds to a P2WKH address that was created from the pubkey returned by walletrpc.DeriveKey or walletrpc.DeriveNextKey (so using the internal, m/1017' key scope) would cause the UTXO to not be detected as being spent.
Is it possible you were sending funds to such an address? If yes, then we can close this as a duplicate of #6434 which will be fixed with lnd v0.16.0-beta.

[joostjager]

I checked our code base, and nowhere we use DeriveKey or DeriveNextKey.

[guggero]

Hmm, okay. What's the state of this transaction at the moment? Did it eventually correct itself or is the UTXO still being listed as unspent?

[joostjager]

The transaction that spent the utxo is still not listed. The following command yields empty:

lncli listchaintxns | jq -r '.transactions | map(select(.tx_hash == "83f78b3ec18a10899ed654ee13fb61107f146b70005b7fde4518538f2ea2d875"))'

We've put a long utxo lease on the "utxo" which is not unspent to prevent it from being selected ever again...

[guggero]

So the UTXO shows for address bc1qa......vzjaq? Can you check if that is an address from the default scope please? lncli wallet addresses should help you there.

Was the wallet ever re-scanned with --reset-wallet-transactions?

[joostjager]

It's currently not showing because we've locked it manually.

The wallet was never re-scanned. If that would solve it, it remains a question why it happened in the first place.

lncli wallet addresses doesn't look like an existing command?

[guggero]

lncli wallet addresses doesn't look like an existing command?

Ah, right, that's only on master, sorry... Would've been useful here.

I do have a suspicion why it happened. But without knowing what type of address it is (default scope or non-default scope) it's hard to say.

[joostjager]

How can it be a non-default address if Derive(Next)Key was never called? It's most likely also an external address, not belonging to our wallet. Anything else we can do at this point?

[C-Otto]

On my node I experience this issue with transactions that are sweep transactions:

2023-09-07 14:20:01.411 [INF] SWPR: Creating sweep transaction 9ca548061daf109c5b90fdda09c67e47d2e7ffd95cc1dc8b5e088e2c507a1205 for 1 inputs (db20a2ddc9985204b8975cd074434887f5f98801db75a05502fa304fb8e8667b:2 (CommitmentToRemoteConfirmed)) using 1784 sat/kw, tx_weight=491, tx_fee=0.00000875 BTC, parents_count=0, parents_fee=0 BTC, parents_weight=0
2023-09-10 10:49:06.957 [INF] SWPR: Creating sweep transaction f9a1443bf812d55d366b10e34b5c5bef23a9951014575e8fac2d3f688fedf6f4 for 1 inputs (db20a2ddc9985204b8975cd074434887f5f98801db75a05502fa304fb8e8667b:2 (CommitmentToRemoteConfirmed)) using 2299 sat/kw, tx_weight=491, tx_fee=0.00001128 BTC, parents_count=0, parents_fee=0 BTC, parents_weight=0

Both 9ca... and f9a... spend from the same input. Both outputs are shown in lncli listunspent, even though only one can actually be used to fund new channels.

[guggero]

@C-Otto are either of these two transactions confirmed yet? I think one of them will go away if the other one confirms (making this a slightly different case than the original problem stated in the issue).

[C-Otto]

Here's the confirmed TX that successfully spent the output:
https://mempool.space/tx/4204cc4225def95105eb53643f1769a978ae92c187ab2802d522d24a15428049

The TXs I mentioned before are not shown in lncli listunspent anymore. I agree, as the confirmed/unconfirmed is a key difference, this is an unrelated issue. Sorry for the confusion.