lnd v0.2-alpha

lnd-v0.2-alpha marks the latest minor (but in a sense major) release of lnd! This release represents a substantial step forward in the stability, feature set, BOLT specification compliance, performance, reliability and maturity of lnd and also Lightning as a whole.

As the lead maintainer of the lnd software, I would like to express my gratitude to the extremely dedicated community of software testers and application developers that have started to emerge around lnd. Your swift bug reports, constant experimentation, and passion for the system you're helping to create have significantly accelerated the development progress of lnd, and for that I deeply thank you. In no particular order, I would like to recognize the efforts of (also bare in mind this isn't an exhaustive list):

• moli
• juscamarena
• takinbo
• mably
• weedcoder
• dts

NOTE: It is important to note that this release of lnd contains several breaking changes. As a result, users will either need to upgrade using a completely fresh installation, or remove their existing channel.db database file before upgrading. As a courtesy, we recommend that users close out their prior channels (either cooperatively if the peer is online, or unilaterally (force close) otherwise) before upgrading.

⚡️⚡️⚡️ OK, now to the rest of the release notes! ⚡️⚡️⚡️

Notable Changes

Sphinx Packet Shared Secret Derivation Change

The method we use to generate the shared secret which is used to derive several keys used in the generation and processing of our Sphinx packets used in multi-hop routing has been modified. Previously we directly used the btcec.GenerateSharedSecret method which returned the raw bytes of the x-coordinate of the derived shared point. After our last release the BOLT#4 specification was modified to instead utilize the SHA-256 of the entire coordinate serialized in compressed format. Within the specification this derivation method was modified in order to align ourselves with the ECDH method used by libsecp256k1, a widely used and extremely well tested library within the Bitcoin ecosystem.

lncli Support for Positional Arguments

The lncli command line tool has now gained support for positional arguments, keyword arguments, and mixing the two within most command line commands. This change was made in order to minimize the required typing for users on the command lone. For example, commands like:

lncli openchannel --node_key=7a379c554dd680083432648e4416255deaf113d655d002baea62652cd9e5b95f --local_amt=100000000 --push_amt=50000 --num_confs=5

Can now optionally be entered as:

lncli openchannel 7a379c554dd680083432648e4416255deaf113d655d002baea62652cd9e5b95f 100000000 50000 5

Multi-Path Payment Path Finding

In this release of lnd, when attempting to select a candidate path for a payment requests we now utilize a modified version of Yen's algorithm to select a set of candidate paths which themselves each have the necessary capacity to carry a target payment. Within our modified version of Yen's algorithm, rather than removing conflicting edges in the current root path and non-spur node edges from the root path, we instead utilize a black-list of edges and vertexes which are to be ignored within the K_ith iteration. This allows us to avoid loading the entire graph into memory during our repeated graph traversals.

Once we obtain a set of candidate payments paths, we then rank the set of candidate paths according to the cumulative fee-rate, and then cumulative absolute time-lock value. As a result, the dispatch of payments within the daemon is now much more reliable as we're able to serially fall-back to lower ranked paths in the case that a payment attempt to the highest ranked path fails.

Additionally, the version of Dijkstra's algorithm we employ (which is used as a subroutine within Yen's) has been modified to: include the capacity of a target edge within the relaxation condition, utilize a binary heap when selecting the next edge to greedily explore and gains white-box knowledge of its role in Yen's algorithm. Finally, we've added an additional layer of caching/memoization of the map: {paymentAmt, targetVertex} -> [path_1, path_2, ....path_n]. Such caching serves as a significant optimization as we are able to skip any dis access when querying repeated payment tuple if no new edges have been added to the graph or modified since the last invocation.

Due to the changes above, the queryroute within the lncli tool and the QueryRoute RPC command of the gRPC Lightning service is no more. Instead they've been replaced with queryroutes and QueryRoutes which are identical to their predecessor commands aside from the fact that they now return a list of all eligible candidate paths rather than simply returning the first available path and exiting.

Complete Authenticated Channel Graph Verification

In the prior release(s) of lnd, the daemon didn't fully verify the authenticity of all vertex and edge announcements within the graph. This meant that lnd would gladly accept an invalid or false channel to it's channel graph, or a forged node announcement as a vertex within the graph. With this release, we now both fully validate all edge+vertex announcements seen within the network, and also generate valid channel authentication and node announcement proofs.

These changes eliminate a class of error encountered by our alpha testers wherein an invalid or non-existent channel would be accepted to the channel graph leading to egregious payment routing failures. Furthermore, with this change, lnd is now fully compliant with BOLT#7. Due to this change, the prior channel graph generate and accepted by prior version of lnd has now been invalidated.

BOLT Commitment State Machine Compliance

In the prior releases of lnd we utilized a channel commitment update state machine which pre-dated the one currently specified within the current BOLT drafts. This statement used an "initial revocation window" which was then dynamically adjusted using a TCP sliding window like commitment update procedure with explicit log indexes used in state transitions. In this new release of lnd, we've now restricted the functionality of lnd's state machine in order to comply with the current BOLT drafts. One minor deviation still exists: we use an initial revocation window of 1, as our current funding state machine still requires such as construct.

Independent of the modification of lnd's state machine a number of minor bugs within the state machine itself have been fixed which include: an incorrect reporting of the number of HTLC/s within the htlcSwitch logs, a fix for a HTLC commitment transaction overflow bug which could results in state desynchronization, and an omitted HTLC cancellation error back propagation within the state machine.

Graph Topology Notification Client

With this release of lnd we've added a new serving streaming RPC command. This RPC command, tilted: SubscribeChannelGraph allows callers to receive streaming asynchronous notification whenever the channel graph has been updated from the PoV of the serving lnd node. Within our integration tests usage of this new streaming RPC has allowed us tighten the running time of our integration test and eliminate several lingering flakes. Outside of our integration tests the addition of this new streaming RPC call opens up the door for client side applications to dynamically render the known channel graph in real-time.

The added gRPC stub service and new proto definitions follow:

rpc SubscribeChannelGraph(GraphTopologySubscription) returns (stream GraphTopologyUpdate);
message GraphTopologySubscription {}
message GraphTopologyUpdate {
    repeated NodeUpdate node_updates = 1; 
    repeated ChannelEdgeUpdate channel_updates = 2;
    repeated ClosedChannelUpdate closed_chans = 3;
}
message NodeUpdate {
    repeated string addresses = 1;
    string identity_key = 2;
    bytes global_features = 3;
    string alias = 4;
}
message ChannelEdgeUpdate {
    uint64 chan_id = 1;
    ChannelPoint chan_point = 2;
    int64 capacity = 3;
    RoutingPolicy routing_policy  = 4;
    string advertising_node  = 5;
    string connecting_node = 6;
}
message ClosedChannelUpdate {
    uint64 chan_id = 1;
    int64 capacity = 2;
    uint32 closed_height = 3;
    ChannelPoint chan_point = 4;
}

--externalip observance by NodeAnnouncement

In the prior releases of lnd, the externalip command-line/configuration parameter was present, yet unobserved and unused. With this new release of lnd, users that wish to advertise their willingness to accept inbound connection to establish channels from participants on the network should set their --externalip parameter to their publicly reachable IP address. Along with a set of global features, if set, this value will now be advertised to the network.

A future minor release of lnd will utilize this new reachability information to optionally automatically bootstrap a persistent channel balance by automatically maintaining a set of channels with peers selected via various heuristics.

Near Uniform Usage of Satoshis within the RPC Interface

In prior releases of lnd, we mixed the usage of BTC and Satoshis within he RPC interface, both when accepting input and displaying results to the user. In order to be more consistent all RPC's that deal with monetary amounts now use satoshis directly, other than the WalletBalance RPC>

Switch to Golang Standard Library For Cryptographic Operations

Within lnd our two most commonly used cryptographic operations were invocations of sha256 several areas of the codebase, and also chacha20 and poly1305 within both brontide and lightning-onion. Previously we ere using two non-stdlib libraries fastsha256 and [aead/chacha20](https://github.com/aead/chacha20). The latter was used as at the time of our prior releases, the stdlib hadn't yet merged in an implementation of chacha20 so we were forced to select an alternative library temporarily until the stdlib gained support.

In this latest release, we've switched to using crypto/sha256 and x/crypto/chachapoly1305 within the codebase. With this change we gain a bit of a performance increase as both of these libraries use highly optimized assembly operations which utilization extensions such as SSE2 and AVX2.

Basic RPC Documentation for Python+Node.JS

Thanks to some new contributors, we've now gained two brief tutorials on driving lnd natively from both Python and Node.js. In future releases these brief tutorials will be expanded to cover all other languages supported by gRPC, include complete example scripts and also partially complete applications.

Switch from Elkrem to Shachain

In prior releases of lnd, for deriving the hashes used in the revocation process for commitment transactions we used a method constructed by Tadge Dryja called elkrem for allow the receiver of the revocation hashes to collapse N hashes into a tree of size log N. When drafting the current BOLT specification, we opted to instead go with shachain (invented by Rusty Russell) instead, as it more efficient when deriving child hashes. In this new release of lnd, we now use shachain throughout the codebase for handling commitment revocation generation, and storage.

48-bit Commitment State Hints

lnd now uses 48-bit total bits from the sequence of the input spending the funding output, and the locktime on the commitment transaction to encode an obsfucated version of the commitment number. In prior release of lnd, we only encoded 24-bits of the current state number using the lower 24-bits of the commitment transaction's sole input. By switching to using 48-bits, we can now encode 281,474,976,710,656 state updates within the commitment transaction. We call this encoding "obsfucated state hints" and it is defined within BOLT#3 of the Lightning Network protocol specifications.

Local+Global Feature Signaling

lnd now implements the initial versioning handshake detailed in BOLT#1. Upon initial connect, the first message send is the init message which allows peers to exchange their local and global features. This exchange allows peers to negotiate the features than are to be used within the session which may for example govern: the type of channels used, the format of the HTLC outputs, and the visibility of transactions created.

Graph Rendering Capability

The lncli describegraph command has been extended to support generating an svg rendering of the known channel graph from the PoV of the generating lnd node. This addition was inspirited by an early tweet by Andreas Antonopoulos shortly after the initial release of lnd, and can be activated by taking on the --render flag to the end of the command.

Change Log

For the specific change log, see the commits referenced within this tag since the prior release.

Contributors (Alphabetical Order):

• Alex Akselrod
• Andreas M. Antonopoulos
• Andrey Samokhvalov
• Bryan Vu
• Christopher Jämthagen
• Cris Perez
• Faris Amali Alis
• François "mably" Masurel
• Justin Camarena
• Marty Jones
• Michael Lynch
• Michiel de Jong
• Olaoluwa Osuntokun
• Thomas Preindl
• Tim Akinbo
• Travis Cline