lnd 0.4-beta
This release marks the 4th major release of lnd! All planned breaking changes have been implemented, and any breaking database changes in the future will utilize the built-in migration system. With this release, lnd has gained a considerable feature set, deeper cross implementation compatibility, a new specialize wallet seed, comprehensive fault-tolerance logic, has had a multitude of bugs fixed, and much more!
This is the first release that comes enabled with a flag to run on Bitcoin's mainnet, and also Litecoin's mainnet. For now, the neutrino backend mode has been disabled on both main chains as the developers of lnd feel that the required testing infrastructure to ensure correct operation in the face of all edge cases has not yet been implemented. However, with this release lnd now supports using bitcoind, and bcoin in addition to btcd as full-node backends!
The maintainers of lnd would like to thank all the dedicated testers, and 60+ contributors which have helped to make this release possible!
NOTE: It is important to note that this release of lnd contains several breaking changes. As a result, users will either need to upgrade using a completely fresh installation, or remove their existing channel.db database file before upgrading. As a courtesy, we recommend that users close out their prior channels (either cooperatively if the peer is online, or unilaterally (force close) otherwise) before upgrading. A new utility command on the cil, lncli closeallchannels has been added to streamline this process.
Verifying the Release
In order to verify the release, you'll need to have gpg or gpg2 installed on your system. Once you've obtained a copy (and hopefully verified that as well), you'll first need to import roasbeef's key if you haven't done so already:
curl https://keybase.io/roasbeef/pgp_keys.asc | gpg --import
The keybase page of roasbeef includes several attestations across distinct platforms in order to provoide a degree of confidence that this release was really signed by "roasbeef".
Once you have his PGP key you can verify the release (assuming manifest-v0.4-beta.txt and manifest-v0.4-beta.txt.sig are in the current directory) with:
gpg --verify manifest-v0.4-beta.txt.sig
That will verify the signature on the main manifest page which ensures integrity and authenticity of the binaries you've downloaded locally. Next, depending on your operating system you should then re-calculate the sha256 sum of the binary, and compare that with the following hashes (which are included in the manifest file):
db811b2c94288d50f1709508c72a35d1893b31fdfb54b2e19a4a65c92f32c581 lnd-darwin-386-v0.4-beta.tar.gz
bcf7813522d9461f27e47ee7879d00ebf45002ad497a1907c7a4312ee3800f0b lnd-darwin-amd64-v0.4-beta.tar.gz
d75a52a695fabee8c18fd8880120b86e891a1689c51fabd5a947584824223284 lnd-freebsd-386-v0.4-beta.tar.gz
86db36bf033e1237ea778f2ba6dbf82b750a2524796e3240803855db941ed8eb lnd-freebsd-amd64-v0.4-beta.tar.gz
43df7ec2eda8e754374b9065259afac360a57bf6c6698b5103aa137061616bbc lnd-freebsd-arm-v0.4-beta.tar.gz
fbda15e493fcf4e187a15852b3ae686cc0df2abdf91c846fcaef3e74df2d5c64 lnd-linux-386-v0.4-beta.tar.gz
dd781604c1f946414c0b053e6ae71b323556dbe4ff87f59408b6b13993e8d688 lnd-linux-amd64-v0.4-beta.tar.gz
be5d855d0ddb4db6db9861e86abd01c227ac892dd0cab8dd3610ff12287035a4 lnd-linux-arm-v0.4-beta.tar.gz
8e04d769c903805a20bafb79ae38ef50b0b98261b79e31ccc99edf5e41306446 lnd-linux-arm64-v0.4-beta.tar.gz
6e0d73c5f63a7500109945787121a479197b6c6af6593a9fabd4286f27d980af lnd-linux-mips64-v0.4-beta.tar.gz
f5e0eb6489346a649dbc85cf69872d6c1fe3c3e3c957d8487529734144b8f0bb lnd-linux-mips64le-v0.4-beta.tar.gz
ca6a99fce87afb9d542d299bd50d957846679840ddf00e1be137ce9bc281622a lnd-linux-ppc64-v0.4-beta.tar.gz
021814819002fb004cfe302c17fded9597567feaf38ee92ad8d7c07bebd8eb25 lnd-netbsd-386-v0.4-beta.tar.gz
e94aeb38b7d7124b29bb6628b28b752d650baa68e3247392294dc4700d208059 lnd-netbsd-amd64-v0.4-beta.tar.gz
a985e6ece98e6eb4d5170ca232517374625e29146946d36bc6f168f30f39c556 lnd-openbsd-386-v0.4-beta.tar.gz
3c7889892a205f7734be9b1c5fa832e0e9c14c29eec30019a026c1907e0a6112 lnd-openbsd-amd64-v0.4-beta.tar.gz
d039c371d01bf788d26cb2876ceafcb21f40f705c98bb0b0b9cf6558cac4ca23 lnd-windows-386-v0.4-beta.zip
1245abe9adeb2fab74fe57d62b6d8c09d30b9ada002cd95868a33406e5a14796 lnd-windows-amd64-v0.4-beta.zip
One can use the shasum -a 256 <file name here> tool in order to re-compute the sha256 hash of the target binary for your operating system. The produced hash should be compared with the hashes listed above and they should match exactly.
This release can also be found in roasbeef's public keybase folder.
⚡️⚡️⚡️ OK, now to the rest of the release notes! ⚡️⚡️⚡️
Notable Changes
Switch to dep for dependency management
In this release, we've switched to dep to handle all dependency management within lnd. In prior release we used a tool called glide. However, glide is no longer being actively developed, and dep provides a much more stream lined UX compared to glide. Most operations can be completed with a simple dep ensure. All installation guides have been updated to account for the new way of syncing the set of dependancies that lnd requires. We're also keeping an eye on vgo as it aligns well with our short-term goal of producing fully verifiable builds for future release.
New Directory Structure
The default directory structure of lnd has been fully revamped, and shouldn't change for the foreseeable feature. After we added dual-chain support, the directory structure was sitting in a bit of an awkward position, as it was only half way to supporting for multi-chain (simultaneous active chains) within lnd. The new structure is now future proof as it maintains a shared channel.db (where all the channel specific state lives), and segmented chain directories that will store the wallet specific information for each chain.
Example of new data directory structure
data/
├── admin.macaroon
├── chain
│ ├── bitcoin
│ │ └── testnet
│ │ ├── block_headers.bin
│ │ ├── ext_filter_headers.bin
│ │ ├── neutrino.db
│ │ ├── reg_filter_headers.bin
│ │ └── wallet.db
│ └── litecoin
│ └── testnet4
│ └── wallet.db
├── graph
│ └── testnet
│ └── channel.db
├── macaroons.db
└── readonly.macaroon
Example of new log directory structure
log/
└── bitcoin
└── testnet
└── lnd.log
Additionally, we'll no longer have a hard coded, unchangeable directory location for lnd's home directory. Instead, users can now use the --lnddir=X argument to set the home directory of lnd to an arbitrary location.
Automatic Peer Bootstrapping
With this release, lnd will now automatically seek out peers to connect with upon initial startup. In order to implement this functionality, we've added a new interface the to discovery package:
NetworkPeerBootstrapper. The NetworkPeerBootstrapper interface is meant to be used to bootstrap a new peer joining the network to the set of existing active peers within the network. Callers are encouraged to utilize several boostrappers in series as redundant sources of information. The MultiSourceBootstrap function will takes a set of boostrappers, and compose their outputs into a single unified set of addresses.
Two concrete implementations of the NetworkPeerBootstrapper interface have been added as a part of this release: the ChannelGraphBootstrapper and the DNSSeedBootstrapper. The former will utilize the authenticated node advertisements within the calling nodes view to boostrap new connections. The latter will use a set of BOLT-0010 compliant DNS seeds to query. This DNS seeding more will likely be used by nodes initial joining the network, as they may not yet have the channel graph as they haven’t connected to any peers. We've also extended our DNS seeder with support for queries over TCP. It was observed that users behind certain name resolvers would filter out the results of our SRV queries. As a result, if we detect this, we'll connect directly over TCP in order to complete the initial bootstrap.
As of this release, there are 3 active DNS seeders queryable:
node.lightning.directory (BTC mainnet)
ltc.nodes.lightning.directory (LTC mainnet)
test.node.lightning.directory (BTC testnet)
New Forwarding Log Timeseries and Query Interface
In this release, we add a much requested feature: the ability to track the fees successfully payment circuits have carried, and also examine the history of HTLCs successfully forwarded by the htlcswitch.
We've added a new storage namespace to channeldb: the ForwardingLog. This log will be used by higher level sub-systems to log each successfully completed HTLC. Each payment circuit will be
summarized as a ForwardingEvent. A series of events can then be queried via a time slice query. In a time slice query, the caller specifies a time range, a number of events to skip, and the max number of events to return. Each query will return the index of the final item. As we have a max number of events we’ll return in a response, callers may need to use this last offset index to seek further by skipping that number of entries. Combining these fields, callers are able to query the time series, skipping an arbitrary amount of events, and capping the max number of returned events.
We store the timestamp in nanoseconds since the unix epoch. Using this level of granularity, we assume that we'll never have a repeated time stamp (due to the monotonicity of recorded time in the Go runtime).
The FeeReport command has been extended to include the recorded fee revenue over the past day, week, and month. We've also added a ForwardingHistory command that lets callers query the entire time series, skip events in the time series, and specify a max number of events to return. A new lncli command fwdinghistory has been added to expose this query information on the command line.
Sample output from the testnet faucet:
$ lncli feereport
<snip>
],
"day_fee_sum": "2",
"week_fee_sum": "2",
"month_fee_sum": "2"
}
$ lncli fwdinghistory
{
"forwarding_events": [
{
"timestamp": "1519857535",
"chan_id_in": "1414915334303449089",
"chan_id_out": "1414950518670229504",
"amt_in": "1001",
"amt_out": "1000",
"fee": "1"
},
{
"timestamp": "1519857771",
"chan_id_in": "1414994499134226433",
"chan_id_out": "1414305105343741953",
"amt_in": "1211",
"amt_out": "1210",
"fee": "1"
}
],
"last_offset_index": 2
}
Improved connmgr Connection Reliability
Internally, in order ot handle automatically connecting out to any peers we have a channel with (and also one-off connections), lnd uses the connmgr package inherited from btcsuite. Our requirements for the behavior of the connmgr were a bit more involved that the functionality in inherited when initially drafted for usage within btcd. Since then we've added a number of useful features such as: the ability to cancel any pending persistent connection requests, and ability to cancel requests that hadn't yet been successful.
While we were at it, the server's peer handling code has been significantly simplified, and now only utilizes a central mutex for all synchronization. Additionally, we'll no longer attempt to tightly connect to our persistent peers in a tight loop in the case of a forced disconnect. Instead, we'll enter an exponential backoff phase in order to mitigate unnecessary flapping.
Deterministic Key Derivation and Cipher Seeds (aezeed)
We've revamped the internal key derivation of the wallet to be fully deterministic. Prior to this release, due to the way that we generated keys for use within channels, and for regular addresses, recovery would have been very difficult to implement in practice.
In this release, we modify the wallet's key derivation for channels and the like to be fully deterministic. We do so by adding a new package: keychain. Additionally, after this PR, the p2pkh addr type is no longer supported. We do this as the mix of segwit and non-segwit addresses has caused widespread confusion with new users in the last. A related change is that we no longer have a --witness_only option for the walletbalance RPC. Since the wallet will only produce segwit addresses.
new keychain package
The role of this package is to house all the interfaces that lnd will use to generate the various keys it needs to create contracts and operate within the network. Additionally, we also use this package to define a deterministic key derivation scheme, that can be implemented by any software/hardware that partially understands BIP43-like derivation.
The first version (v0) of the keychain schema is very simple. We re-use BIP43, with a slight twist. Re-using BIP43 lets us leverage all the existing libraries out there as far as compatibility. The purpose used is 1017, and the coin type, the particular coin type of the chain where the keys are meant to be used. Within our schema, we define multiple “key families”. Each key family is actually just manifested as a new “account” within the BIP44/43 family. With this schema, if we have static description of the keys used within a channel, given the seed, and the “KeyLocator”, we can re derive all keys necessary be able to re-sign for the channel.
New Seed Format for Backup+Recovery
In this release, we add a new package implementing the aezeed cipher seed scheme. This is a new scheme developed that aims to overcome the two major short comings of BIP39: a lack of a version, and a lack of a wallet birthday. A lack a version means that wallets may not necessarily know how to re-derive addresses during the recovery process. A lack of a birthday means that wallets don’t know how far back to look in the chain to ensure that they derive all the proper user addresses.
The aezeed cipher seed scheme has a few cool properties, notably:
- The mnemonic itself is a cipher text, meaning leaving it in plaintext is advisable if the user also set a passphrase. This is in contrast to BIP 39 where the mnemonic alone (without a passphrase) may be sufficient to steal funds.
- A cipherseed can be modified to change the passphrase. This means that if the users wants a stronger passphrase, they can decipher (with the old passphrase), then encipher (with a new passphrase). Compared to BIP 39, where if the users used a passphrase, since the mapping is one way, they can’t change the passphrase of their existing HD key chain.
- A cipher seed can be upgraded. Since we have an external version, offline tools can be provided to decipher using the old params, and encipher using the new params. In the future if we change ciphers, change scrypt, or just the parameters of scrypt, then users can easily upgrade their seed with an offline tool.
- We're able to verify that a user has input the incorrect passphrase, and that the user has input the incorrect mnemonic independently.
With the combination of the cipherseed and new key derivation, a user simply needs to back up their seed:
---------------BEGIN LND CIPHER SEED---------------
1. able 2. sound 3. almost 4. master
5. process 6. anxiety 7. organ 8. again
9. lesson 10. grid 11. write 12. rifle
13. lizard 14. kid 15. opinion 16. crop
17. crack 18. pottery 19. need 20. patrol
21. virus 22. wrestle 23. gallery 24. month
---------------END LND CIPHER SEED-----------------
And they'll be able to generate all keys and secrets generated by lnd for use in contracts. This is one component of our strategy for on-chain and off-chain state backup. Future release will implement full lookahead key scanning on chain for on-chain recovery, dynamic backups to private outsourcers for safety, and finally a static backup system for off-chain recovery in the case of partial data loss.
First time users should use the lncli create command in order to create an initial cipher seed mnemonic, add an optional passphrase to it, and finally set a wallet encryption passphrase that's used when unlocking the daemon with lncli unlock.
Bitcoin Core Support!
With this release of lnd, we've added a new backend interface implementation: bitcoind. Users of the Bitcoin Core full-node software can now utilize their node to provide lnd with a valid view of the Bitcoin chain. The implementation utilizes zeromq in order to get real-time notification of Bitcoin blocks. Additionally, if the rpcuser+rpcpass parameters aren't specified, then lnd will attempt to either parse our the parameters from the bitcoin.conf, or utilize the authentication cookie.
New lnd arguments
The set of arguments has been modified, and also expanded since the last release. Namely, users are now able to specify which precise network interface to listen on for p2p connections via the --listen flag. Users are now also able to specify a target interface for both the gRPC server (--rpclisten) and also the REST TLS proxy (--restlisten). Finally, it's also now possible to start up lnd in a mode that doesn't listen for any peer to peer connections at all with --nolisten.
Due to the addition of new chain backend within lnd (bitcoind, litecoind), we've split up the way that each chain and node backend is specified on the command line. First, users will need to specify which chain(and network) is active:
--bitcoin.active If the chain should be active or not.
--litecoin.active If the chain should be active or not.
--bitcoin.mainnet Use the main Bitcoin network
--bitcoin.testnet Use the test network
--bitcoin.simnet Use the simulation test network
Once the chain is specified, users need to specify their target node backend via:
--bitcoin.node=[btcd|bitcoind|neutrino] The blockchain interface to use. (default: btcd)
Finally, depending on the selected node backend, users will need to specify their selection of the (some optional) arguments:
neutrino:
-a, --neutrino.addpeer= Add a peer to connect with at startup
--neutrino.connect= Connect only to the specified peers at startup
--neutrino.maxpeers= Max number of inbound and outbound peers
--neutrino.banduration= How long to ban misbehaving peers. Valid time units are {s, m, h}. Minimum 1 second
--neutrino.banthreshold= Maximum allowed ban score before disconnecting and banning misbehaving peers.
btcd:
--btcd.dir= The base directory that contains the node's data, logs, configuration file, etc. (default: /Users/roasbeef/Library/Application Support/Btcd)
--btcd.rpchost= The daemon's rpc listening address. If a port is omitted, then the default port for the selected chain parameters will be used. (default: localhost)
--btcd.rpcuser= Username for RPC connections
--btcd.rpcpass= Password for RPC connections
--btcd.rpccert= File containing the daemon's certificate file (default: /Users/roasbeef/Library/Application Support/Btcd/rpc.cert)
--btcd.rawrpccert= The raw bytes of the daemon's PEM-encoded certificate chain which will be used to authenticate the RPC connection.
bitcoind:
--bitcoind.dir= The base directory that contains the node's data, logs, configuration file, etc. (default: /Users/roasbeef/Library/Application Support/Bitcoin)
--bitcoind.rpchost= The daemon's rpc listening address. If a port is omitted, then the default port for the selected chain parameters will be used. (default: localhost)
--bitcoind.rpcuser= Username for RPC connections
--bitcoind.rpcpass= Password for RPC connections
--bitcoind.zmqpath= The path to the ZMQ socket providing at least raw blocks. Raw transactions can be handled as well.
Finally, with this release of lnd, a flag for Bitcoin's mainnet --bitcoin.mainnet and Litecoin's mainnet has been added! At this point, we still only recommend users to experiment with small amounts as lnd is still young.
For further details, see INSTALL.MD.
All arguments to lnd are now also documented within sample-lnd.conf.
Dynamic Fee Estimation
In prior releases of lnd, we utilized a hard-coded fee in all locations as a stop gap until interfaces for dynamic fee estimation were erected. In this release, we'll now utilize dynamic fee estimation with the btcd, bitcoind, and litecoind node backends. This means that we'll query for a fee estimate whenever we want to: open a new channel, send coins on-chain, sweep any funds that were encumbered in a contract, and also close channel cooperatively. We'll now also dynamically sample the fee rate each block in order to propose a new fee rate for the commitment transaction (via the UpdateFee message) to ensure a timely confirmation in the case of a channel force closure.
If users wish to bypass this dynamic fee estimation, and specify their own fees, then the openchannel, closechannel, and sendcoins commands for lncli now have two optional parameters for fee control:
--conf_target value (optional) the number of blocks that the transaction *should* confirm in, will be used for fee estimation (default: 0)
--sat_per_byte value (optional) a manual fee expressed in sat/byte that should be used when crafting the transaction (default: 0)
During cooperative channel closures, we'll now properly negotiate fees with the remote node, rather than proposing our fee and never compromising. We use a conservative step-wise algorithm to ensure termination, and eventual satisficing for the fee of the cooperative closure transaction.
Increased ChainNotifier Robustness
Internally within lnd, we use an interface called the ChainNotifier in order to request notifications once a particular txid has reached a sufficient number of confirmations. This interface is used widely when: waiting for a channel to open, waiting for a channel to close, waiting for a transaction sweeping a time-locked output to confirm, etc. Before this release, the ChainNotifier wasn't able to properly handle re-orgs, and would return incorrect information in the face of excessive re-orgs. With this release, that's been remedied as the ChainNotifier is now fully aware of disconnected blocks, will send negative confirmations for transactions that have been re-org'd out, and has been significantly refactored in order to remove code duplication.
New contractcourt package.
In this release, of lnd, we've added a new fully-featured contractcourt package. This package houses all the logic to ensure that in the event of an on-chain event, we're able to effectively go to chain in order to redeem any outputs encumbered by a contract. This includes all incoming and outgoing HTLC's, time-locked second-level HTLC transactions, and all forms of commitment outputs. The role of this package is to examine the chain, and the internal state of each channel, then decide if it needs to go to chain (court) in order to ensure the ultimate resolution of user funds.
The package is split up into distinct sub-systems with isolated responsibilities, the first of which being the ChannelArbitrator. The duty of the ChannelArbitrator is to watch the set of active contracts on a comment transaction and act accordingly if any of their redemption criteria have been met. Potential criteria include: an HTLC about to time out, and HTLC about to time out that we know the preimage to, or the remote party going to chain (forcing us to resolve all pending contracts on chain).
The primary goroutine of this struct implements a persistent state machine in order to ensure that mid contract resolution, we’re able to properly survive restarts without losing our place, or forgetting about a pending contract.
A ChannelArbitrator will stay alive until all contracts have been fully resolved. This means that outside sub-systems no longer need to worry about remembering to mark a channel as fully resolved, as it’s the job of the ChannelArbitrator to do this task.
In the event that the ChannelArbitrator deems it needs to go to chain in order to preserve funds, it launches a series of [ContractResolvers](https://github.com/lightningnetwork/lnd/blob/master/contractcourt/contract_resolvers.go#L23). In this release, we introduce a new interface, the ContractResolver. The duty of a ContractResolver` is to watch a contract on-chain, for all possible transitions, and exit finally when the contract has been fully resolved. Resolvers themselves can be recursive: meaning producing another resolver to hand off the duties require to fully resolve a contract.
Each resolver also has a ResolverKit which contains all the function closures and interfaces that the resolver need to properly do its job.
The 5 types of resolvers are:
- outgoing HTLC timeout
- outgoing HTLC contested
- incoming HTLC know presage
- incoming HTLC contested (don’t yet know)
- commitment sweep
Wrapping it up, the The ChainArbitrator is a special sub-system that will oversee the on-chain resolution of all active channels, and also channels that are in the pending close state. The ChainArbitrator maintains a set of ChannelArbitrators, one for each channel that hasn’t yet been fully resolved.
Outside sub-systems should send new channels to the arbitrator once they’ve opened. Additionally, they can also trigger manual interventions to close out a channel on chain forcibly, or just to signal that a channel has been closed cooperatively.The ChainArbitrator should be notified once a fresh set of signals for a channel becomes available. The ChannelArbitrator for the channel will use these set of signals to be notified when an on-chain event happens.
Finally, this package also houses the chainWatcher. The duty of this struct is to replace the functionality that was previously implemented by the closeObserver of each channel. Rather than the source of notification being tied to the lifetime of a particular object, it’s now delegated to a persistent object that will be around for the entire lifetime of the channel (until it’s closed). This will serve to greatly simplify the code, and eliminate a large class of bugs.
UtxoNursery Persistence + Hardened BreachArbiter
The UtxoNursery (which handles overseeing the maturity of time-locked outputs) has been revamped in order to be fully fault tolerant. Before this release, if the nursery watch watching over an immature output, and restart occurred, then the nursery would forget about the output, leaving user funds on the chain. With this release, the nursery has been modified to be fully fault tolerant via staged check points throughout the various stages of output maturity.
Additionally, the BreachArbiter has also gone through a similar persistence hardening. It'll now remember in-progress breaches, in order to finish bringing the counter-party to justice. Within this release, we'll now properly handle attempted contract breaches that take HTLCs to the second level. We do this by detecting if any of the HTLC outputs has been spent on the commitment transaction, and modifying our spend of those outputs to target the second level with updated witnesses.
Full Validation of Commitment Constraints
In this release, we'll now adhere to all the current commitment constraints during the channel update procedure. This includes the: channel reserve, min HTLC, max value in flight, and max number of active HLTCs for both side. If either of these constraints are violated, then the channel will be failed accordingly.
HtlcSwitch Fault Tolerance
In this release, we've upgraded the htlcswitch (the forwarding fabric for HTLCs, you know, that whole "payments as packets", thing?) to be fully fault tolerant. This means that in all cases of partial HTLC adds/fails/settles, upon restart, the switch will ensure that the payment circuit creation or completion is satisfied properly. For further details, refer to the bodies of the primary PR's that implemented this new functionality:
As set of detailed integration tests have been added to excercise new added persistence of the switch.
Additionally, lnd is now safe against Sphinx packet replays. We'll maintain a decaying log of all the shared secrets we've derived, and reject any HTLC's that attempt to re-use the same shared secret. This defends against a class of attacks against payment privacy launched by attempting to inject stale HTLCs into the network observing their propagation path. With each block we'll then garbage collect any shared secrets for HTLCs that have been expired, freeing up space on the log. As the onion blob commits to the HTLCs, they can't be decoupled to replay an expired HTLC.
Linear Speedup in Sphinx Shared-Secret Construction
A long lived TODO was resolved in this release as we modified the algorithm used to derive the shared secrets and ephemeral public keys during sphinx packet construction to achieve complexity that is linear in the number of hops. The current version uses a quadratic number of scalar multiplications during the derivation, which is potentially noticeable when sending payments, particularly on resource-constrained devices.
The primary optimization offered by this release is to cache the intermediate product of the blinding factors, such that the i-th blinded pubkey can be generated via a single scalar multiplication, instead of iteratively applying the previous i-1 blinding factors to a given field element. This alone reduces the number of scalar multiplications to be linear in the number of hops. Note that the product is taken modulo the curve order |F(G)|, not the prime P.
We also include a second optimization, that exploits the similarities in deriving both the ephermeral and blinded pubkeys.
From Section 3.2 of the Sphinx paper, the equations for deriving the i-th ephemeral pubkey is given:
a_0 = g^x
a_i = g^(x * b_0 * ... * b_{i-1})
and the i-th blinded pubkey as:
s_0 = Y_0^x
s_i = Y_i^(x * b_0 * ... * b_{i-1}),
where x is the session private key.
Now, instead of only memoizing the product of the blinding factors, we modify the value to also include a multiplicative factor of x which can be shared across both exponentiations. This does not result in any improvement asymptotically. However, it allows us to replace the usage of scalar multiplication with more-efficient scalar base multiplication when computing the ephemeral pubkeys (the a_i's), in addition to shaving (N-1) scalar multiplications in raising each Y_i to x before applying the blinding factors separately.
Macaroon Authentication Changes
In this release, we've switch to a new macaroon format which is much more compact as it uses a protobuf like encoding. Additionally, we'll now encrypt the root macaroon key on disk, using the daemon's encryption passphrase. Docs have been updated to detail how to pass macaroons over the REST interface:
GET https://localhost:8080/v1/getinfo
Grpc-Metadata-macaroon: <macaroon>
Outbound Tor Support
In this release, we've added outbound Tor support. This means that if specified, lnd will attempt to tunnel all outbound connection traffic over Tor, and disallow incoming connections. Care has been taken to ensure that no DNS leaks occur during normal operation in conjunction with our Tor usage. A new section of documentation has been added to document usage of Tor with lnd`:
Revamping HTLC Routing
In this release, we've introduced a sub-system called missionControl. missionControl` contains state which summarizes the past attempts of HTLC routing by external callers when sending payments throughout the network. missionControl remembers the outcome of these past routing attempts (success and failure), and is able to provide hints/guidance to future HTLC routing attempts. missionControl maintains a decaying network view of the edges/vertexes that should be marked as "pruned" during path finding. This graph view acts as a shared memory during HTLC payment routing attempts. With each execution, if an error is encountered, based on the type of error and the location of the error within the route, an edge or vertex is added to the view. Later sending attempts will then query the view for all the vertexes/edges that should be ignored. Items in the view decay after a set period of time, allowing the view to be dynamic w.r.t network changes.
Additionally, we'll now take an iterative approach to path finding in order to reduce payment latency, and also reactively modify our available channel graph in response to routing failures.
Memory Optimization
This new release of lnd should now be much slimmer. A number of GC related optimizations have landed to reduce the memory usage of lnd at peak capacity. Further work will be done in future release in the form of adding reusable buffer pools to reduce the working set of the daemon.
Changelog
The full list of changes since 0.3-alpha can be found here:
Contributors (Alphabetical Order):
- 34ro
- Adam Gibson
- Alessio
- Alex Akselrod
- Alex Bosworth
- Alex Gessner
- Alexandre Viau
- Andreas Tasch
- Andrew Naoum
- Andrey Samokhvalov
- Benjamin Congdon
- Brandon Lico
- Braydon Fuller
- Brenden Matthews
- Brian KimJohnson
- Chris Moore (dooglus)
- Conner Fromknecht
- Cristobal Griffero
- Daniel Coonce
- Daniel McNally
- Dave Collins
- Dave Kerr
- Deelight-fr
- Dimitris Tsapakidis
- Dustin Dettmer
- Eugene Lziege
- funyug
- gshub77
- hackerrdave
- Hector Jusforgues
- Jason Dufair
- Jim Posen
- Johan T. Halseth
- John Griffith
- Juan Ignacio Donoso
- Justus Kandzi
- Laura Cressman
- Liu-Cheng Xu
- Maarten Tutak
- Matt Drollette
- Micah Lerner
- Mrmaxmeier
- Nabil Boag
- Nalin Bhardwaj
- Naveen
- Olaoluwa Osuntokun
- Oliver Gugger
- PaddyQuinn
- Pat White
- Philip Hayes
- Philipp Gillé
- practicalswift
- rajeshnair2k
- Ronald Vandermeer
- Salman Ansari
- Sam Lewis
- Samuel Dobson
- Simon Horlick
- Steven Roose
- Thijs Triemstra
- Timo Schmid
- Tomas Susanka
- Tasuku Nakamura
- Valentine Wallace
- vegardengen
- Walt Seymour
- Wilmer Paulino
- Yancy Ribbens
- Yuri Zhykin