Lightning Network BOLT by BOLT
Speakers: Jim Posen
Date: July 24, 2018
Transcript By: Gloria Zhao
Tags: Lightning
Category: Meetup
Media: https://www.youtube.com/watch?v=Ysj2yobFMF4
Topic: Lightning Network BOLT by BOLT
Location: LA Blockchain Meetup
Transcript by: glozow
Introduction
I’m Jim from the protocol team at Coinbase. We work on open source stuff in the crypto space. We just contribute to projects like Bitcoin core. The reason I’m talking about this is I’ve been contributing on and off to one of the major lightning implementations lnd for the past eight months or so.
This is gonna be a fairly technical talk. We’re going to break down the protocol specification that specifies how Lightning Network nodes communicate with each other. It’s an 11 part spec. There’s 11 BOLTs and so we’re gonna go, piece by piece, BOLT by BOLT. Some of them will be more in depth than others. I know people are probably at different levels of understanding with this stuff so can I just kind of poll… who knows what a UTXO is? Who knows what an HTLC is? Okay cool, that’s helpful for me to know where to spend time and stuff.
Very high level: what is a blockchain? It’s an ordered transaction ledger. The defining properties in my opinion are: it’s globally replicated computers around the world are constantly synchronizing these transactions with one another. You have this notion of probabilistic finality which means that a single computer when it’s downloading the ledger at some point in time can make an assessment and say I am almost 100% certain that this transaction will never be reverted from the global state. This idea of probabilistic finality is really important, and it’s censorship resistant which means that if you are trying to make a transaction and you’re willing to pay the fees, it doesn’t matter what the contents of the transaction are. It will make it into the ledger eventually. In this talk, we’ll be talking about the Bitcoin blockchain, but these are kind of general properties of blockchains.
Pros and Cons
What this gives you on the plus side is: zero counterparty risk. So if I send you a million dollars today you know and you wait 30 minutes you know absolutely that you have this million dollars and there’s no way for it to be reversed even if regardless of how much you trust me or don’t trust me and that’s kind of all the good of things about blockchains. They have a lot of downsides because they’re globally replicated. They’re really slow; it’s expensive for everyone to run these things and you have pretty poor privacy by and large because everything’s public. Everyone can see everyone else’s transaction.
So when we think about how people have solved transaction efficiency in the past, if you think about how our financial system works, there’s kind of batching at every layer. If I make some bank transfers, those don’t get the Fed’s account balances immediately. All the banks batch up all of the transactions they’re gonna do between one and another. All the transactions between banks get batched once a day, and you don’t even make bank transfers for most things. You pay with a credit card and then settle your credit card bill once a month. And then there are certain things like opening a tab at a bar where you don’t even want to do a credit card charge for every transaction. So at every layer we have batching, and that’s what the Lightning Network tries to provide. We’re gonna drill into that last example of the bar tab that’s gonna be kind of this constant analogy.
Buying beer at Bob’s bar
Alice is at Bob’s bar. They don’t know each other, and they’re both pretty paranoid people so they want every assurance at every step of no default. Alice opens this tab, and she gives Bob an IOU. She’s gonna sign this and say “I owe you $5.00” and Bob gives her back a beer. And then they do another transaction, and instead of giving him another IOU, she’s gonna replace the old IOU. So she writes a new IOU, signs that one for $10, and says “this one replaces the old one” and now Bob’s willing to give her another beer. And then hopefully, at the end of this interaction, they will settle up, and Alice is going to pay Bob the total balance of the IOU.
But as I said, Bob’s pretty paranoid. What if Alice doesn’t pay? He could ask for the money up front… but Alice doesn’t want to give him the money up front, because what if he doesn’t refund? So they try to find some mutually trusted party. We’re gonna go with this guy Dorian. They both trust this guy and he’s gonna hold the money in escrow. The very first thing that happens when Alice opens the bar tab is she gives $20 to Dorian. Now, she can do up to $20 worth of beer. So we do the exact same thing. She buys a few beers, she gives some IOUs, and then Bob at the end of this can just give the IOU to Dorian. Dorian sees that it’s signed by Alice and he can divvy up the original settlement. So he gives $10 to each of them. This is pretty good, but we don’t get any guarantee that Bob is actually gonna give Alice a beer after she pays, or that she’ll give him an IOU for the beer. It isn’t really atomic and it never really can be unless you do some physical thing.
But we can pretty surprisingly do atomic swaps of payments for signed receipts that a payment was delivered. The way we do this is Bob’s gonna sign this message that says “if Alice knows this SHA256 preimage, then she must have paid me five dollars.” I know this preimage and I’m only gonna tell her if she pays me.
So Bob’s gonna sign this notice, then Alice is gonna construct this contract that says “if Bob reveals the preimage by 1:00 p.m. I’m gonna pay him five dollars.” This contract on the right is super important. It’s called an HTLC. It’s the building block of a lot of this stuff and this is how we’re gonna do these atomic swaps of payments for receipts. So instead of giving Bob an IOU, she’s gonna give him an HTLC. And this has to be parameterised with the hash amounts and the expiration. Those are the parameters to this contract. It’s a binding contract, and now Bob can give Alice the beer because he knows the preimage. He knows that, at any time, he can go and take the signed HTLC - the signed contract - and give it to Dorian, along with the preimage. This, together, constitutes proof of… you know, it fulfills the contract. Oh, I put ten dollars - it should be five and fifteen. Dorian will pay out. But this is kind of inconvenient because if Bob presents the HTLC right now he only has until 1:00 p.m. to do that and so that means Alice’s bar tab has to close out by 1:00 p.m.
There’s another thing they can do, since Bob knows he can do this at any time. What he’s gonna do is: when Alice gives him the HTLC, he’s just gonna tell her the preimage right away. And now that Alice knows the preimage she has this signed receipt from Bob that says “I’ve been paid for beer” so she’s gonna replace the HTLC with an IOU because at this point he has the money and she has the proof. So now the tab remains open past 1:00 p.m. and they can continue transacting.
Chaining Channels
Okay so the analogy’s gonna break down at this point. We have this abstraction: this way for people to pay each other back and forth once they’ve established this initial payment to some asked escrow person. And so what we want to figure out is “how can Alice pay Bob if they’ve never opened a channel with one another?”
Everything that I called a tab before is now going to be a channel. So she wants to establish this payment system. “I, Alice, will pay you $5.50 if and only if you pay Bob five dollars. So I’m going to give you a 50 cent fee if you can facilitate this payment to Bob. You don’t receive money unless you pay Bob.” So Alice gives an HTLC to Carol for $5.50 and she’s gonna delay it a little bit. The expiration is gonna be 1:15. If Carol gives an HTLC to Bob, she knows that if she has to pay Bob, she has the preimage that she can turn around and give back to Alice. Bob in this case has received payment for his beer, so he’s willing to give the beer to Alice. The whole interaction here is gonna play in reverse where Bob gives the preimage to Carol, Carol now has a nice 15-minute buffer to turn around and give that preimage back to Alice if she needs to go to Dorian and settle and close out that tab. So this is how we’re gonna chain these channels together and this is the fundamental insight behind Lightning Network. And this whole interaction that I’ve sort of described is, for the remainder of the talk, what we’re gonna break down.
Payment Channel History
Just a little bit of history; a lot of these ideas are really old. In fact satoshi, in early emails with devs like Mike Hearn, talked about the idea of batching and using payment channels. But we didn’t have the right features in the scripting language for a while to do it. But in 2013, Core devs Matt Corallo and Mike Hearn actually wrote a payment channel implementation. It was broken and they knew it, but it was there. And then in 2015 the Lightning Network paper got released. This had a few major kind of landmark findings. One was the ability to do bi-directional channels, so if Alice pays Bob some money, she doesn’t need to open another channel for him to pay her back. They can use the same channel to pay back and forth. with this lightning channel flattening network construction and it kind of came up with this idea of using h TL CS to chain these payments together.
In 2016, these three teams that were kind of separately working on lightning at the time decided to get together and come up with an interoperable specification so that all their implementations could work with one another. And that’s called the BOLT Spec and that’s what we’re talking about. We’re kind of just at the point where these things are getting launched into beta. Two of the three are approved for mainnet usage. They’re all getting used on mainnet at this point. I wanted to put the timeline up because it takes a really long time to get this stuff right. And we’re kind of just at the cusp of this stuff being usable.
BOLT 3: Bitcoin Transaction and Script Formats
So where do we’re gonna start right at the beginning at bolt three so this is how you actually construct the smart contracts out of curiosity how interested in people how interested are people in Bitcoin minutiae? I plan to skip it but… I feel like this stuff is pretty Bitcoiny. Here’s the basic ideas.
Funding Transaction
The very first thing that happens when you open a channel is one participant creates any transaction paying to a 2-of-2 multisig address so you have two endpoints they each they swap public keys with each other and one of them pays to a contract that says “the only way to spend this money is with explicit authorization from both endpoints” and that’s it this goes on blockchain. this gets confirmed this is how the channel opens it’s non malleable for reasons that are important - and we’ll talk about in a second. In the optimistic case, they do a whole bunch of back-and-forth payments. The whole you know bar thing with the HTLCS and then in the optimistic case, all that happens is they mutually agree “okay the division is now Alice gets 75% of the money and Bob gets the other 25, and we’ll make a transaction that pays out in those proportions.” The two of them we will both sign this because we agree that’s how it should be, and this goes on chain. So in the the mutual consent case, this is the only interaction that anyone sees. It’s a multisig spend but this is the very last transaction to be agreed upon between the two of them.
Why would Alice pay to a 2-of-2 multisig if she knows that maybe Bob is gonna go offline and there’s no way for her to ever spend these funds? That would be really bad. She needs some way to ensure that she can get reimbursed.
The whole the entire time, both endpoints Alice and Bob are going to keep what’s called a commitment transaction. The commitment transaction is something that they store on their own and if they need to (because the other party stops cooperating or they they come into disagreement or whatever), they always have the option of publishing this transaction on chain. Each node has a separate commitment transaction that’s signed by the other node before anything happens. You’re probably confused because this transaction looks just like the closing transaction, but I’m telling you they’re different. The reason is because these one of these outputs is encumbered by an additional condition that makes it safe so that there’s this challenge period. So what we want to do is if Alice publishes this commitment transaction, we want to give Bob a time buffer - let’s say a day - to see that this happened on chain. He says “ok actually was the last state that we agreed upon and not some prior commitment transaction that I gave her a long time ago and was reverted.” So if in some state Alice has 75% of the money and then she pays him a little bit more now she only has 50% we need a time buffer so that if she publishes the transaction where she has more money, Bob can go and and challenge it and say that’s that’s invalid this was actually a replaced state that Alice said she was never gonna broadcast.
What we do essentially is: the direct payment back to the person who published the commitment transaction has two different ways to spend it one is after a time delay if you have signatures from both parties then it goes to the person it’s intended for so if I publish and I wait ten minutes from the time sorry like a day from the time that I publish then I can get my money back within that day though there’s a non-time-restricted clause that says “if bob has this sort of trapdoor key the secret revocation key then he can spend at any time and so the way we’re gonna enforce that people don’t publish past states that they only publish the current state of our negotiation” is by saying “if you ever publish a last state I’m going to you’re gonna have to give me a trap door key upfront that allows me to spend everything.” It allows me to take the entire channel that once, so it’s really never in your interests to publish a past date because I’m gonna punish you by taking all your money.
These outputs are encumbered by this time delay. So the commitment transaction doesn’t just have the two outputs. It has all these HTLC’s so we can encode HTLCS in Bitcoin script and some of the HTLCS are from one person the other some are from the other person back to you what Ann HTLC typically looks like is it has to redeem clauses but if you’re curious this is a language called Ivy which compiles to Bitcoin script and I know you can’t read it but I think it’s more readable than Bitcoin script there HTLC has two clauses one says if you have the pre image so if Alice is giving in HTLC to Bob then Bob can spend it with the pre image or Alice can spend it after the time delay those are the two clauses the timeout and redeem so we have a problem if we just put these right in the commitment transaction directly the problem is that we have parameterised the HTLC by one timeout this is the expiration time on the HTLC agreement but we also have another timeout which this challenge period and so we can’t really put both of these timeouts into the same contract it’s the same Bitcoin script.
Regular HTLC
What we do is we have to break it out into two separate layers of transactions. So if Alice publishes this commitment transaction and one of the clauses is “Bob’s gonna pay me if I have a hash preimage.” On the one hand, she shouldn’t be able to spend that immediately and get the money, because it could be an old state and that would be bad if she could get the money immediately. But she also needs to immediately lock in the fact that she has the preimage before that HTLC times out. So when she publishes the commitment transactions she’s going to immediately spend the HTLC, but it can only be spent to one place. It can only be spent to these pre-signed transactions called the HTLC success timeouts. and this is where the challenge delay happens. So the commitment transaction is actually a whole suite of commitment transactions. You have one base transaction and then, for every HTLC, you have a separate second layer of transactions.
Yeah the trapdoors are on all of these outputs but after the challenge period expires, the trapdoor is still there, but the other person should have swept the funds at that point. So it’s kind of a race; as soon as the challenge period ends, they’re racing. but like really there should be no race, because you should have just done it before.
That brings us to a good point about these trap door keys. They’re called the revocation keys. We’re gonna have a series of these commitments right every time there’s a new state there’s a new commitment and it would be bad if they all have the same trapdoor key obviously because if you got it once then you would have it for all future transaction so the revocation key needs to be different for every commitment transaction and actually we want all of the keys like there were like five different keys up here and this is like a simplified version of it we want all the keys and all the scripts to be different on every transaction there’s a reason it’s better for privacy if you want to use a watchtower… When you open a channel you don’t tell the other person a key, you tell them in an elliptic curve point that they use to derive all the keys. Who here knows BIP32? Hierarchical wallets. In a similar sense, you can use one private key to derive a whole bunch of future private keys. And so upfront, they just exchange something that they use to derive all of the future keys for all the future transactions and that’s how you get different revocation keys.
BOLT 5: on-chain handling
BOLT 5 specifies on-chain handling. It’s pretty simple. Both of these endpoints have this channel open and they’re both locked. They’re both watching the blockchain at all times - if someone spends the funding output, they know immediately to check if this is the latest state then okay we’ll let it happen we’ll just close the channel. If it’s a past state then go use your revocation key and steal all the money. The other case where you want to take your commitment transaction and publish it on-chain immediately is if maybe the other person’s just totally negligent and you need your money now. But also if you have an HTLC you see that’s about to expire and you have the preimage you should definitely publish that commitment transaction before the HTLC expires so that you can spend it with the preimage before they time it out. Conversely, if your HTLC just timed out you want to publish the transaction so you can spend the time out. So if any of your HTLCS are getting close to the expiration you publish the commitment transaction now and that means the other party just like was misbehaving because they should have done something to them at that point. BOLT 5 has a bunch of other rules but it all just boils down to: anything you can spend, (like if you have a pre-image), spend. If your output timed out, spend it. Just sweep everything all the time.
BOLT #1: Base Protocol
This brings us to the protocol itself. so every Lightning Network node is connected to the Bitcoin network; it’s monitoring the blockchain. But it’s also connected to the Lightning Network which is a totally separate peer-to-peer network and so there’s a bunch of just typical peer-to-peer stuff that you need to know how they are communicating.
well they’re communicating over TCP. That’s the serialization format. custom binary serialization. There’s a few other cool things about the handshake. The protocol is pretty extensible. when you connect to someone you give them a bitfield of all of the features that you support. So let’s say for example let’s say we wanted to upgrade one of those transaction formats and we decided “oh there’s a better way of writing this script, do you support this new way of writing the script?” you advertise that when you connect to someone “there’s another cool rule that I want to talk about that I just think is clever.”
There’s this general, over the entire protocol, “it’s ok to be odd” rule, which means that in many situations where things are numbered… every protocol message has a number like the hello would be 1. And then some of the messages we’ll see in a sec might be 5 or 7 or whatever. Any protocol message that’s odd you will ignore from your peer if you don’t know what it is. Any protocol message that’s even that you don’t know what it is you’re gonna disconnect immediately. So it kind of bakes in extensibility in kind of a neat way where in this bit field, if you if your peer has a feature that’s odd that you don’t know you’ll kind of let it be.
Okay and then we do encryption and authentication over this TCP connection with this protocol called Noise_XK_secp255k1_ChaChaPoly_SHA256. That’s probably enough right? So what this actually means is like instead of using TLS or something like that this is kind of a more lightweight way of doing what TLS does but like with far fewer dependencies. With TLS you basically need open SSL which is something that people don’t really like to include if avoidable and also it brings in some cryptography that is sort of non-native. What lightning does is it uses the same exact elliptic curve that Bitcoin uses so you use the same dependency. And it has a really fast stream cipher. This is all basically to say that it’s got a really kind of lightweight and modern way of doing encryption and authentication. Importantly this means that every connection isn’t authenticated. So when you connect to a node you have their public key and there’s no way that you could be a man in the middle. You know you’re gonna authenticate that they have the private key. So every node has a cryptographic identity on the network, unlike in Bitcoin, where you just connect to a host and hope. In fact you don’t even care if they’re who they say they are.
BOLT#2: Peer Protocol for Channel Management
Now we can get into kind of what I think is the most interesting part of this and what I spend the most time on, which is how do we take that whole bar protocol and turn that into like Network messages. This is basically what we want to replicate. so Alice and Bob dotted lines are TCP connections they have a TCP connection and Alice wants to open a channel and right now lightning only supports single thunder channels. so Alice is basically gonna a tell Bob “hey I want to open a channel to you. I’m gonna pay for everything so it’s at no cost to you. Do you want to accept this channel?” And she’s gonna give him some parameters like “this is the most I want to have in flight in each HTLC at anytime. in terms of money, this is the amount of time that you need to wait to get your money back.” There is that challenge period. There’s a bunch of parameters to these channels that they need to negotiate upfront.
So Alice asked Bob “Do you want to accept this channel?” Bob says “yeah I want to accept the channel. Here are my parameters.” Alice is gonna locally create the funding transaction herself and she’s not going to tell that transaction to Bob but she’s going to tell him what to sign. So that the initial commitment transaction that just does a full refund back to Alice of the entire amount that she’s putting into the channel he’s gonna sign that because there’s no reason not to. He doesn’t need to put up any money at this point. Alice creates this transaction; she says “here’s my signature and give me yours” so he signs back and then she can fund this transaction and publish it to the blockchain and they’re both watching the chain when it confirms. They send this funding lock message which means like our channels open. Like, let’s go. The lines are channels.
Ok so this is where it gets interesting. The first thing Alice is going to do is send HTLCs to Bob. These are numerically ordered sequentially from 1, so every time, Alice gives Bob a new HTLC. These are all parameterised with the HTLC parameters. And then at the end of like a bunch of updates, she gives this commitment, a signed message to Bob. So she doesn’t actually give him the commitment transaction, she just gives him these signatures because Bob can locally recreate the commitment transaction himself based on the history of their messages. So they’re not actually exchanging transactions, they’re just exchanging sufficient information to recreate these transactions. So Bob knows “okay I know how much goes into the channel and I know these HTLC’s, so this is what my transactions gonna look like. And do Alice’s signatures actually sign this?” so he’s gonna check the signatures and if this is all good then he’s gonna send back a revoke an ACK message which essentially it acts as “I’ve received your signatures and I think they’re good” and it revokes the prior state saying “okay not only have I ACKed that your signatures are good, but I will never publish anything before the state ever again. and to ensure that, here’s my revocation key for the past state.” So he actually gives back that revocation key So Alice can go and spend it if he ever were to publish that.
So at this point Bob still can’t go and give Alice the preimage. Why? Because he’s revoked his past state. But Alice hasn’t revoked her past state. So she actually still has a valid state in which these HTLC’s didn’t exist. So what has to happen now: Bob has to send back a commitment signed and say “here you go and update your view of the state. here’s my signatures on it” and alice is going to send back revoke_and_ack.
So these two changes, add_htlc, have been signed by alice and ACKed by Bob and signed by Bob and by Alice. When we’re in this state where this change has been applied in both people’s states, and all prior states revoked, these updates have been irrevocably committed. This is important because once the HTLC is irrevocably committed, Bob can respond. He can say “I’m gonna take the money on HTLC one here’s the preimage, so like I’m gonna fulfill this HTLC.” I don’t know maybe the amount was wrong or actually like you got the destination wrong and I don’t know how to fulfill this or expired or whatever I’m gonna fail this one so this basically says I’m gonna claim the first one and I’m gonna reject the second one and these are new updates that have to happen. So Bob’s gonna go ahead and apply these to his commitment transaction, sign that new transaction, and then Alice is gonna ACK it, and then Alice is gonna take those two updates and she’s gonna sign the new commitment state with those applied. Bob’s gonna ACK it and now the fail_htlc and the fulfill_htlc have been irrevocably committed. The channel state I want to say here.
Yeah I mean this is just as fast as you can send messages over a TCP connection. It’s as fast as you can send stuff over TCP connection but you need to persist the signatures so that slows things down. That’s the slowest part of this because if Alice revokes a thing and gives him the revocation key and he doesn’t save that to disk like he’s in a dangerous position. so you have disk I/O happening so that’s kind of what slows some of these interactions down. But it still happens in like way under a second.
The other thing I meant to say here was this sender of the HTLC always sends the HTLC and the recipient of the HTLC always removes it. So let’s say Alice sends Bob an HTLC that that times out after a little bit. Alice has no way of telling Bob like “hey look let’s clear this failed HTLC.” it’s Bob’s responsibility. that just simplifies the protocol so like the sender of the HTLC adds it to the state and then the recipient removes it from the state.
The other thing about this whole interaction has been that it’s been one direction, Alice to Bob the whole time but actually this is bi-directional and it’s happening bi-directionally simultaneously. Alice sends 2 HTLC’s to Bob. Bob sends 2 HTLC’s to Alice. You can see like the the IDs the incrementing IDs are in different namespaces for the two different directions and increment on different counters and when we get down here like what what is Alice signing because we don’t really know if she’s even received these from Bob yet like maybe she has maybe she hasn’t so how do we make sure that it’s like unambiguous what she’s signing right so there’s this essentially hopefully this example shows us but when you sign when you send us a commitment signature you’re only signing the updates that you sent and then the revoke is how you sort of link these two logs together so this stuff gets really confusing this is like why I wanted to put it up because it’s like a total mind bender but uh it’s gonna be fun
Alice has a view of the updates that she’s sent to Bob and she has a view of the updates she’s received from Bob. Bob also has a view of the updates he’s received from Alice and he has a view of the updates he’s sent to Alice. And these are kind of these can be out of sync because it’s an asynchronous TCP connection but they will happen in the same order. Alice creates these two adds and then she sends them to Bob. and then Bob gets them. so we’ve now they both have the same sort of like view of their interaction. and then Bob’s gonna do an htlc_add and send that to Alice. and simultaneously, Alice is gonna sign her two previous adds and then she’s gonna send the sig to Bob. so she is only signing the stuff in her column. Oh man you can’t see the colors this okay well these are yellow for now.
So she sends the commits sigs to Bob but she doesn’t these haven’t been ACKed so these are kind of an appending state where Alice has signed it but it’s not irrevocably committed yet right. and then she receives the htlc_add from Bob. Bob gets the commit sig. He’s gonna send this ACK. so he sends the ACK. It’s still not irrevocably committed on his side because Alice has an Alice hasn’t revoked the prior state where these HTCLs didn’t exist and then simultaneously Alice is going to add another HTLC.
These things are all happening simultaneously and they will come into consensus with each other due to the way these messages are constructed. What happens as soon as Alice receives Bob’s ACK and as soon as she receives Bob’s ACK let’s see she’s going to immediately what happened there Bob got the the add and he got the sig. she gets the sig from Bob nothing really changes except that she’s gonna ACK the commit.
This commits to the add but it also commits to the revoke. so this you can kind of follow this up and down this line so this commit means I have added this HTLC for myself but I’ve also received your two adds and so that means it’s irrevocably committed. Bob’s view because he’s committed he’s signed this state and then so he sends that over and then as soon as Alice acts this commit and then it’s irrevocable sorry it wasn’t irrevocably committed yet now it is in Alice’s view because she has revoked the state before these two adds were were added. As soon as Bob gets it those went green which means that this was irrevocably committed.
The only thing that’s been irrevocably committed at this point is these two and these two and then those two are in a pending state. And so as soon as Alice sends the revoke she also needs to send a sig. So this is now signing the fact that I have revoked. or like this kind of signs this stuff and it signs this and what happened there oh now because Alice had irrevocably committed this one Bob’s gonna go in fulfill HTLC one and then he’s going to sign that so again still. the only thing that’s been irrevocably committed so far is the two adds from Alice he gets the commit sig and they both revoke so what’s been committed. Now let’s see. So Alice’s updates. She’s ACKed this stuf. This is still incomplete because this needs to be ACKed by Bob as well. It’s got really confusing without the colors so anyway this stuff works.
BOLT #4: Onion Routing Protocol
Ok so we’ve got these adds and then we need a way of chaining this stuff together. So every add packet has the amount, the ID, the expiration and the hash. it also has another field which is a routing packet. so the routing packet actually has the entire information for the whole circuit. if Alice sends a packet to Carol that packet says here at this HTLC but also here’s Bob and like here’s maybe David behind Bob as well. and so for every hop you need to tell them what’s the next channel in the hop, what is the amount that they’re supposed to get, because each one has a fee. and then what’s the expiration time on the HTLC so that everyone unambiguously knows how to construct the the packet that they’re supposed to send out.
But this kind of leaks a lot of information right, because now Carol knows not only how much money Bob’s getting but that he’s the last person in the entire channel that Alice is the first person in the whole chat the whole circuit that Alice is paying Bob. so it turns out we can encrypt this stuff using Onion Routing which is the same technique that Tor uses where every hop only sees the next hop, and every hop behind that is encrypted. So if you look at what the packet looks like it looks like you know. I have my information and the next person but everything after that is encrypted to the next hop. So when the person at hop one gets this packet he’s gonna decrypt this and then he sees hop 2. And he can forward it on.
What’s really cool about onion encryption is not only do you kind of hide the amount that is ultimately transacted down the line but you hide who the ultimate sender is and who the ultimate recipient is. so if you just see the next hop you don’t know if that next hop is the ultimate destination or whether they’re forwarding it to someone else. so you’ve kind of added a really interesting layer of privacy to these payments. you can glean certain information like if you know roughly in the range of what people are charging for fees and you know the network graph you can try to de-obfuscate some stuff, but it’s a privacy enhancing technique. And we can do this because every node has this public key bound to their identity - that was the authentication key when we talked about with Noise protocol. Not only is routing information encrypted but if there’s a problem like the recipient has to fail this for some reason, not everyone back upstream sees that error. That error is encrypted back to the original sender so we get some pretty good privacy properties from this.
BOLT 7: Channel Gossip
Ok so now we have a way that if the sender can create this routing packet for the entire circuit then we have a way for everyone to pay each other and for people to forward payments. So now all we need is a way for people to discover routes. If we have this network where all these people are connected by channels at all times, everyone’s just gossiping about who they have channels with as long as both of them opt into that channel being public. And so they’re all constantly synchronizing this.Not everyone has a global view in a consistent way of the topology. like this node might only know about these four other nodes and these channels between them then eventually hopefully as these nodes come online and they gossip, this person tells this person he’s gonna relay the information about this channel over to the original person. So eventually this stuff should synchronize and ideally everyone does have a complete view of the entire network graph so with the entire view of the network graph you can construct your own routes. That does put a pretty large burden in terms of bandwidth and storage and stuff and memory usage on every node as the network grows. That’s just how the routing works right now. So there’s two kinds of main packets that are interesting in this Gossip protocol.
First, you announce a channel. This happens once and you say “here’s the output on the blockchain.” This is really important because: one, it allows people to tell whether your channel is still open and whether they should continue. like if that funding output didn’t exist then that that channel doesn’t exist and there’s no reason you can stop sending payments through it but it also tells you the amount the total capacity of the channel. That doesn’t need to be explicitly put in there and the channel announcement is signed by both endpoints.
Every side of the channel sends these channel_updates at all times. This is one side of the channel signs this and says “hey like maybe the other person went offline so I just want to tell the world I’m not gonna be able to send payments right now so I’m gonna disable this channel.” And then when they come back online I’ll send a new update that says okay they’re back online channel, re-enabled. You can also say “if you’re gonna route through me like this is the fee I’m charging” and maybe you want to change that dynamically over time. Every time you change the fee, you propagate out a new channel_update message that gets propagated through the network. So if people were doing that all the time it would you know cause a lot of bandwidth consumption. Every channel_update has a timestamp so you know what the latest one is.
You can prune all channels so if you don’t get a newer update for a channel within some period (I think it’s like a few days by default) you’ll just kind of drop that channel. So if you have a channel you’re supposed to rebroadcast a channel update every so often.
BOLT #11: Invoice Protocol
Okay, this is the last one I promise. BOLT eleven ties everything together. What does an end to end payment look like? Well it looks like this I guess. So if you want to pay someone on lightning they’re gonna give you a blob that looks like this and you can tell because it says lnbc. see this is the amount it’s a twenty five hundred micro Bitcoin. and then there’s a whole bunch of other data and you can take this it’s bech32 encoded and you can decode it. Your lightning software should be able to tell you okay the recipient has this pub key, so go find them on the network. The amount is 2500 microBitcoin. It expires in 60 seconds so pay it before then. Here’s the hash and you can put a description in.
So this is really cool because it does kind of this half of the receipt thing right so if you have a signed invoice by the recipient that says if you have this preimage then like you have for sure paid for this item. That’s a really powerful thing, the preimage combined with the lightning endpoint. The lightning invoice together forms irrefutable cryptographic proof of payment.
[Demo]