-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[bug]: listunspent shows spent outputs #8786
Comments
Hi @kallerosenbaum, Thanks for the issue report! I've been trying to investigate the issue to see if I can spot the root cause. I have some initial questions and requests:
Finally, you're correct that restarting lnd with the |
@ViktorTigerstrom thanks! Correction: we didn't upgrade from 0.17.2 to 0.17.5, we upgraded from 0.17.3 to 0.17.5.
|
@ViktorTigerstrom I'm probably going to restart with |
Apologies for the late reply @kallerosenbaum! Thank you for the extended logs, we should hopefully have what we need for the further investigation with that. You can go ahead with the restart with |
I should probably mention that we sweep transactions in a maybe somewhat unusual way.
|
@ViktorTigerstrom ran --reset-wallet-transactions and it's back to normal. But I have a feeling the issue will resurface, since it happened on both our nodes. I think it might have to do with our onchain sweeping method (described in the comment above) which runs on both nodes. The method should be fine, but maybe it isn't handled well by LND for some reason. Or am I doing something that I really shouldn't be doing? |
Just curious, how long did it take to restart lnd with the
I think there's some investigation work needs to be done, think we should target this for 0.19.0? cc @saubyk
As for the sweeping method, I don't see anything wrong here - is this used for UTXO aggregation purpose? How did you get this set of inputs? |
Tagged for 0.19 |
Yes, we aggregate to an external wallet.
I think it might have to do with the relatively low fees we set on the transactions. The one described in this issue ( |
1 sat/vb is indeed pretty low - during a low-fee environment it should be enough to get the tx in the mempool, then it can be CPFPed. During a high-fee environment, however, peers may not relay the tx (you can check peers' minimal relay feerate via There's the
Meanwhile we will look into the |
We don't set 1 sat/vB. We set fee according to bitcoin core's
I opted not to use that since it doesn't give me enough control over the UTXOs to spend. |
FYI: We've been hit by this issue again on one of our nodes. I've kept the full logs for future investigation. |
@yyforyongyu I've been able to reproduce this on regtest. I use a docker-compose with bitcoin core 27.0 and lnd 0.17.5 to get the environment up, then I repeat the following until the bug triggers:
It can take up to 40 iterations before it happens. It varies. The sweep in step 2 fails with "output already spent". I'm sure this can be further simplified, for example use only 1 UTXO or reduce the number of confirmations, but at least this is a start. I run LND with the following conf:
|
@yyforyongyu When this issue gets fixed, is there a chance that the fix can, in addition to prevent future issues, also recover from ongoing issues? I mean, is it possible to heal from existing false UTXOs, for example by going through all UTXOs during upstart and check against bitcoin core if they are valid and if not, remove them? The reason for this is that a rescan took ~20h last time (#8786 (comment)). We currently have this issue on both our nodes and it causes lots of headache, especially with accounting. |
@kallerosenbaum Def - I'm investigating this issue this week and will come up with a fix to handle both existing and future cases. |
Background
This might be a duplicate of #5959, but not sure so I'll open a new ticket.
We have two LND nodes of the same version 0.17.2 (but later they got upgraded to 0.17.5). We call them lnd-main and lnd-routing.
Both these nodes recently reported and still report outputs they don't have. I'm focusing on lnd-routing here (leaving out all correct outputs):
These are already spent by tx
61f8f4b41196b01a3acb3eee16a81f890540d3b624ed2b0efa730520947f272f
, which was broadcast 2024-04-09 and confirmed on 2024-05-15.Your environment
Steps to reproduce
When searching through the logs I found this line right after a restart on 2024-05-21:
This transaction is
61f8f4b41196b01a3acb3eee16a81f890540d3b624ed2b0efa730520947f272f
.The same happened on lnd-main too when it restarted about 30 minutes prior to lnd-routing, but I won't list the details about that issue here. I'll provide whatever you need upon request.
Before we discovered this we upgraded both nodes to 0.17.5 on May 23.
I'm hesitant to restart with --reset-wallet-transactions because I don't know the consequences. The wallet is very old and I'm not sure how long the rescanning would take and what limitations apply on the node while doing it.
The text was updated successfully, but these errors were encountered: