lnd v0.4.2-beta
This release marks the second patch release to the recently released v0.4-beta! No new major features have been added in this release. Instead, this release packages a series of bug fixes, modifications to ensure better cross-implementation compatibility, the ability to perform on-chain seed rescans w/ full look ahead, and a series of important fixes related to the switch and state machine. No database level breaking changes have been made in this release, as a result users should be able to perform a clean update.
Verifying the Release
In order to verify the release, you'll need to have gpg or gpg2 installed on your system. Once you've obtained a copy (and hopefully verified that as well), you'll first need to import roasbeef's key if you haven't done so already:
curl https://keybase.io/roasbeef/pgp_keys.asc | gpg --import
The keybase page of roasbeef includes several attestations across distinct platforms in order to provide a degree of confidence that this release was really signed by "roasbeef".
Once you have his PGP key you can verify the release (assuming manifest-v0.4.2-beta.txt and manifest-v0.4.2-beta.txt.sig are in the current directory) with:
gpg --verify manifest-v0.4.2-beta.txt.sig
That will verify the signature on the main manifest page which ensures integrity and authenticity of the binaries you've downloaded locally. Next, depending on your operating system you should then re-calculate the sha256 sum of the binary, and compare that with the following hashes (which are included in the manifest file):
62c6d6df01d3adab63af0e25987e8567a9e1dec3b0d42c2df9a10e6cde6675d8 lnd-darwin-386-v0.4.2-beta.tar.gz
b0636c39fec61e9a4e9d19c026393b3f080d93c0c674edaf4156df2ed2b1c244 lnd-darwin-amd64-v0.4.2-beta.tar.gz
b3e8ad43ebaaa357c930020996cb7f8d0db175d8187d893a4dc02b6ab1f1bf43 lnd-dragonfly-amd64-v0.4.2-beta.tar.gz
199fe9adea0c91c1ba93eac17781234bf98e05d83780b394592b30d60c80b88d lnd-freebsd-386-v0.4.2-beta.tar.gz
db63789a8a94b13cf8dce041e81c0a403c9cda375efbd9a02dadb9670c791114 lnd-freebsd-amd64-v0.4.2-beta.tar.gz
a38a5c8930c12988cf241028c760f50d73e900b0fc578fd7b5d291779a186989 lnd-freebsd-arm-v0.4.2-beta.tar.gz
06aab3858f141d92e602097e64725292677b68f818914c026dfebde2676a38d8 lnd-linux-386-v0.4.2-beta.tar.gz
2b6b617d804bfee5352aefcabaae9e27e58013084f9c5654d3f1185222f604c8 lnd-linux-amd64-v0.4.2-beta.tar.gz
583afca9f4ebb53bc9a9ce2b643d686c28868c05e225b64c0694140e628f928b lnd-linux-arm-v0.4.2-beta.tar.gz
ba206e02ed589f3779500e6ab48089b8af6dba3a19526afb2263c298afc9f137 lnd-linux-arm64-v0.4.2-beta.tar.gz
2cacf4bb0e252ebea2a47cf873c198aaabe0172bd09e7ffccbc1024a4474ff34 lnd-linux-mips64-v0.4.2-beta.tar.gz
016d6e7a3482e7ca52bd6ccc76f3cb3577a8b0aaf11e62f81a8d701203ba5d9c lnd-linux-mips64le-v0.4.2-beta.tar.gz
e2788aa696ebb6dadf8d1ee9b3636ab27f6ca235c7132f9c00805b6b06fc9070 lnd-linux-ppc64-v0.4.2-beta.tar.gz
c3e4d58cf5f1f11b1e5e594d10c53399dc31e3c0f3b585cc12dbb2ef9a5a90c4 lnd-netbsd-386-v0.4.2-beta.tar.gz
5c582a393e4bd1eacff2490c90a81da7d70e6ade80d1ce83183df37cc813d516 lnd-netbsd-amd64-v0.4.2-beta.tar.gz
26f4d6be6bc73062034333e084d546ea524506200c92ca0168f9653c90225737 lnd-openbsd-386-v0.4.2-beta.tar.gz
7ab8e802029b878b849ec2d726834097211d54faecbd61f00610df90f70a75ef lnd-openbsd-amd64-v0.4.2-beta.tar.gz
b7ea54308ada52ab599009577f375e8ff561fda3683d62f990a0cda38a1209a3 lnd-windows-386-v0.4.2-beta.zip
807fe7edc02741e1f433110af748219158e1f47181e0aa0f9de1ce175594a520 lnd-windows-amd64-v0.4.2-beta.zip
One can use the shasum -a 256 <file name here> tool in order to re-compute the sha256 hash of the target binary for your operating system. The produced hash should be compared with the hashes listed above and they should match exactly.
Finally, you can also verify the tag itself with the following command:
git verify-tag v0.4.2-beta
You should see the following if the verification was successful:
gpg: Signature made Tue May 29 16:31:39 2018 PDT
gpg: using RSA key 964EA263DD637C21
gpg: Good signature from "Olaoluwa Osuntokun <[email protected]>" [ultimate]
This release can also be found in roasbeef's public keybase folder.
Notable Changes
Litecoin
The max payment size and channel size for Litecoin has been increased by ~60x. This is a stop gap measure before a feature bit is added to the protocol to enable arbitrarily sized payments and channel sizes.
lnwallet
Full on-chain seed recovery with configurable look aheads is now fully implemented!. As a result, users will now be able to use their aezeed with lncli create or lncli unlock to trigger a full rescan to recover any on-chain funds. The implementation is generic, so it works with: btcd, bitcoind, and neutrino. The look ahead value is configurable in order to give users more control over the thoroughness of the on-chain key search.
We'll now ensure that that any transaction broadcast have fee rates above the min relay fee of the node lnd is connected to.
Bitcoind and btcd Chain Backends
A bug in the bitcoind chain backend has been fixed that would cause lnd to stall on start up at times due to an internal bug when attempting to rescan to see if an output has been spent or not. As a result, startup using the bitcoind backend should be generally much snappier. Future versions of lnd will continue to increase the performance of the bitcoind backend. Particularly, once the BIP 158 implementation is merged and exposed over RPC, we'll be able to use those filters for rescans internally as we do for btcd.
The txindex is now no longer required for either btcd or bitcoind!. However, users will find that lnd is generally more performant if the index is enabled, as it saves us from performing manual rescans. v0.5-beta will contain an overhaul to the way we perform historical notifications dispatches which will eliminate manual rescanning all together.
Configuration and Documentation
Users can now configure log rotation to optional. By default lnd will maintain 3 compressed log files on disk, rotating them over each time we fill up a new log file. When running with the trace or debug logging levels, the logging can be quite verbose, which warrants disabling log rotation all together or tweaking parms concerning the max log size and also the total number of log files to maintain.
lnd will now properly recognize the BIP 173 hrp prefix for regtest.
contractcourt
A number of bugs have been fixed in the contract court ensuring that we don't play duplicate commitments, properly lay the remote party's full set of commitments, and also ensure that all related goroutines exit properly on shutdown (c5169a7).
The ChannelArbitrator sub-system has been modified to only act on confirmed commitments. This fixes a number of bugs encountered and ensures that we'll only attempt resolve contracts which are properly buried in the chain. As a result, new state has been added to the pendingchannels RPC: commitment broadcast. Channels will be in this state once we broadcast a commitment, but before a transaction spending the funding output has been confirmed. We do this as although we broadcasted a commitment, it's possible that another distinct transaction is confirmed instead. In either case, we'll play which ever spending transaction is confirmed, and proceed to resolve any active contracts.
The ChainWatcher has been modified to ensure it always plays all possible active commitments.
For cooperative channel closes, we'll now ensure that we play the transaction which ultimately enters the chain, rather than assuming the final signed closing transaction would be the one that wins over.
Channel Funding
A bug has been fixed that would at times cause a state desynchronization if both sides were lnd nodes and had selected custom values for the CSV delay.
BreachArbiter
The BreachArbiter (the sub-system tasked with enforcing justice against cheating channel peers) has been significantly simplified. Along the way, hand off between the breach arb and the contractcourt has been improved to ensure the hand off is atomic, even in the phase of a breach at the point of a daemon shutdown.
Docker
The docker image for lnd is now around 10% the size of the prior image! Additionally, it'll now pull in local changes rather than fetch the source from git. As a result, developers can now use the docker-compose set up for local simnet cluster management when doing development, or testing new pending PRs.
lncli and lnrpc
Several errors that result from incorrect usage of the rpc interface, or invalid arguments have now been made clearer.
The macaroon package now contains a set of unit tests, as well as integration tests on the lnd level. This paves the way to the more flexible "Macaroon Bakery", which will allow callers to generate a set of custom macaroons.
The GetInfo command now contains the version that lnd is running, and if compiled properly, will also display the commit hash that lnd was built off of.
It's now possible to turn off logging all together for all or a set of targeted sub-systems.
lncli unlock will now display a more helpful error message when a user attempts to unlock an already unlocked wallet.
Commands in lncli will now be grouped by category:
NAME:
lncli - control plane for your Lightning Network Daemon (lnd)
USAGE:
lncli [global options] command [command options] [arguments...]
VERSION:
0.4.1 commit=80852601dbd2b84c257a1b52a7f13518ed8a6091
COMMANDS:
getinfo Returns basic information related to the active daemon.
debuglevel Set the debug level.
stop Stop and shutdown the daemon.
help, h Shows a list of commands or help for one command
Channels:
openchannel Open a channel to a node or an existing peer.
closechannel Close an existing channel.
closeallchannels Close all existing channels.
channelbalance Returns the sum of the total available channel balance across all open channels.
pendingchannels Display information pertaining to pending channels.
listchannels List all open channels.
getchaninfo Get the state of a channel.
getnetworkinfo Get statistical information about the current state of the network.
feereport Display the current fee policies of all active channels.
updatechanpolicy Update the channel policy for all channels, or a single channel.
On-chain:
sendmany Send bitcoin on-chain to multiple addresses.
sendcoins Send bitcoin on-chain to an address.
listchaintxns List transactions from the wallet.
Payments:
sendpayment Send a payment over lightning.
payinvoice Pay an invoice over lightning.
addinvoice Add a new invoice.
lookupinvoice Lookup an existing invoice by its payment hash.
listinvoices List all invoices currently stored.
listpayments List all outgoing payments.
queryroutes Query a route to a destination.
decodepayreq Decode a payment request.
fwdinghistory Query the history of all forwarded HTLCs.
Peers:
connect Connect to a remote lnd peer.
disconnect Disconnect a remote lnd peer identified by public key.
listpeers List all active, currently connected peers.
describegraph Describe the network graph.
getnodeinfo Get information on a specific node.
Startup:
create Initialize a wallet when starting lnd for the first time.
unlock Unlock an encrypted wallet at startup.
Wallet:
newaddress Generates a new address.
walletbalance Compute and display the wallet's current balance.
signmessage Sign a message with the node's private key.
verifymessage Verify a message signed with the signature.
GLOBAL OPTIONS:
--rpcserver value host:port of ln daemon (default: "localhost:10009")
--lnddir value path to lnd's base directory (default: "/home/guggero/.lnd")
--tlscertpath value path to TLS certificate (default: "/home/guggero/.lnd/tls.cert")
--no-macaroons disable macaroon authentication
--macaroonpath value path to macaroon file (default: "/home/guggero/.lnd/admin.macaroon")
--macaroontimeout value anti-replay macaroon validity time in seconds (default: 60)
--macaroonip value if set, lock macaroon to specific IP address
--help, -h show help
--version, -v print the version
Channel State Machine
A bug has been fixed wherein we'd attempt to sweep breached dust outputs. Additionally, we now ensure that we can sweep breaches that contain both incoming and outgoing HTLCs.
A bug that prevented fees from being negotiated down has been fixed.
A bug has been fixed that would previously cause lnd to initiate a new state transition without any committed updates. This was a spec violation and would cause a compliant implementation to abandon the channel.
A bug has been fixed that would result in lnd restoring the incorrect log state after a restart with a pending commitment dangling. This bug would cause lnd to crash on start up with a nil pointer panic.
HTLC Routing
A number of improvements have been made within the ChannelRouter w.r.t how we respond to received onion htlc errors. As a result, users should generally find the routing hiccups that existed in prior versions of lnd have been resolved. Version 0.5-beta for lnd will contain a large overhaul in the MissionControl sub-system to address several drawbacks in the current naive implementation.
We've modified the way we handle FeeInsufficientErrors to more aggressively route around nodes that repeatedly return the same error to us. This will ensure we skip older nodes on the network which are running a buggier older version of lnd.
We'll now no longer prune vertexes in response to receiving an UnknownNextPeer error. Instead, we'll now prune the edges as otherwise faulty or malicious nodes could cause us to backlist a target node, rather than routing around the failure.
We'll now skip local channels unable to satisfies a potential payment flow during path finding. This should reduce payment latency a bit, and also result in less internal routing failures.
Private Channel Invoicing
When a user creates a invoice w/ active private channels, we'll now encode routing hints for each of these channels. These routing hints enable non-advertised channels to still receive incoming payments, and will be an important component of the network as it continues to grow. Additionally, our path finding logic has been updated to utilize any routing hints if populated.
HTLC Switch
When disconnecting peer due to an invalid commitment, we'll now ensure that the error to the remote peer is sent before we kill the socket. This ensures that our new log message with useful debugging information reaches the peer before we shut down the link.
A very old bug in the forwarding logic of the switch has been fixed. When accepting an HTLC we are meant to validate the fee against the constraints of the outgoing link. This is due to the fact that we're offering a payment transit service on our outgoing link. Before this commit, we would use the policies of the incoming link. This would at times lead to odd routing errors as we would go to route, get an error update and then route again, repeating the process.
We'll now properly use the incoming link for timelock related constraints, and the outgoing link for fee related constraints. We do this by introducing a new HtlcSatisfiesPolicy method in the link. This method should return a non-nil error if the link can carry the HTLC as it satisfies its current forwarding policy. We'll use this method now at forwarding time to ensure that we only forward to links that actually accept the policy. This fixes a number of bugs that existed before that could result in a link accepting an HTLC that actually violated its policy. In the case that the policy is violated for all links, we take care to return the error returned by the target link so the caller can update their sending accordingly.
The DecayedLog implementation to protect against Sphinx replays has been moved to the htlcswitch package.
A number of bugs surrounding forwarding to channels which haven't yet been confirmed, as well as premature HTLC trimming have been fixed.
A bug has been fixed that would cause a state machine that accepted a duplicate fail or settle to crash upon restarts. We now ensure that we'll reject any duplicate fails or settles.
A bug has been fixed that would cause us to forward a fail/settle too early internally, leading to inconsistencies within the switch, and the mailbox of each of the links.
A bug has been fixed that would cause us to add an HTLC to the log, but not fully evaluate it. We'll now ensure that we properly set the add/remove heights of internal PaymentDescriptors properly when restoring the update logs after a restart.
Peer to Peer and Server
An optimization has been made within the peer struct to no longer allocate a new buffer each time we to to write a message. Instead, we'll use a static write buf sized for the largest possible protocol message within each peer instance. As a result, our memory usage is much less bursty, and generally much lower.
We'll now on a best effort basis attempt to locate the advertised port of peers that have connected to us via inbound connections. Before this addition, lnd would at times be unable to automatically connect a certain class of peers which we first discovered via an inbound connection.
discovery
A bug has been fixed in the AuthenticatedGossiper that could at times cause a deadlock when a user attempted to update the set of channel policies.
A bug has been fixed that would cause partial goroutine leaks when the AuthenticatedGossiper is shutting down. The changes ensure faster and cleaner shutdowns of the ChannelRouter as well.
ChainNotifier
The ChainNotifier has been modified to primarily only dispatch spend notifications once the spending transaction has been confirmed within a block. Switching to this behavior fixes a bug wherein if both sides attempted to force close simultaneously, or a closing transaction we weren't expecting hits the chain, then we would fail to realize the contract has been resolved on chain.
aezeed
A set of test vectors has been added , and also the README now has additional details concerning the underlying cipher seed scheme.
Changelog
The full list of changes since 0.4.1-beta can be found here:
Contributors (Alphabetical Order):
- Adam Soltys
- Alex Akselrod
- Andrew Fuller
- Antonin Hildebrand
- Bretton Vine
- Conner Fromknecht
- Danny Paz
- Dimitris Tsapakidis
- Everton Melo
- Francisco Calderon
- Jack McCallum
- Jasper Raedisch
- Jim Posen
- Jimmy Song
- Johan T. Halseth
- Jens Heimbuerge
- Karlson Lee
- Kazato Sugimoto
- Kenneth Perry
- Max Vernon
- mecampbellsoup
- Olaoluwa Osuntokun
- Oliver Gugger
- practicalswift
- Richard Bondi
- Ron Gross
- Sean Erle Johnson
- Steven Roose
- t4sk
- Vadym Popov
- Valentine Wallace
- Vegard Engen
- Wilmer Paulino
- whythat
- Yohei Okada