Day 2 of the Chaincode Labs Lightning Residency
Lightning ≈ Bitcoin by Dr. Christian Decker
Explanation of why Lightning is Bitcoin and why it’s not exactly Bitcoin. There are tradeoffs, when to use which. Some of these tradeoffs are the reason why we use Lightning, other tradeoffs make coding a Lightning app very annoying.
Lightning = Bitcoin
- Payments on LN are denominated in bitcoins. No need for an ICO!
- Bitcoins on LN are fungible with on-chain bitcoins
- LN is complementary to on-chain payments. There’s different levels at which on-chain payment makes more sense than off-chain.
- On-chain: pay per byte sent, fixed cost.
- Off-chain: fixed base fee + variable proportional payment, pay per satoshi sent.
- At some level, large payments become less expensive on-chain than off-chain.
Lightning > Bitcoin
- More private, not perfect but better than on-chain
- More scalable, unicast vs broadcast, more efficient
- Fewer fees (only true for a certain size of payments, determined by network liquidity dynamics)
- Real-time payments, source of most of the new use cases. Don’t sit around waiting for confirmations.
- Eventually will have streaming payments.
- Invoices that actually work! Invoice standard is expressive and easy to use. Has on-chain fallback address.
Lightning < Bitcoin
Tradeoffs that we don’t have a good solution for…
- How do we allocate funds? Fund allocation is not trivial, hard to automatically anticipate liquidity needs of user. Mitigation: open channels at random to create as cohesive as possible network. Well connected network can route rather well, versus everyone being connected to Starbucks and it is an island. Bridges will emerge to have well connected network, incentivized as fee pressure on Bitcoin increases. Routing may not be a profit center, but can be used to offset your fees.
- Routing can be difficult
- Relies on remote channels and their capacities. You have to try to make a payment to find out if there is capacity, can’t know ahead of time. Too much information to broadcast.
- Relies on peers being online
- Need to be online to receive. Interactive process, you need to acknowledge you received a payment.
- Funds are in hot wallets. Don’t keep your retirement pension in LN channels. This is for coffee payments.
- Payments may drop on-chain. Stuck payments are slower than on-chain due to timeouts. Nice thing for merchants to do is send a refund for stuck payments for immediate retry, this isn’t supported by the clients yet but will happen automatically eventually.
- Not all funds are spendable (reserve). Nasty detail where the merchant has to keep a reserve for a new channel, while the consumer is trying to get a full refund.
Routing is Hard
- Nodes connected, but no channels. User surprised they can’t send a payment!
- Imbalanced channels
- Bottleneck issue when routing. Try to make the channels of reasonable size, too-small channels are bad.
In Progress Improvements
- Splice-in / splice-out: add and remove funds from an existing channel
- Multipath routing: split larger payments onto multiple paths
- Spontaneous / streamed payments: perform multiple payments based on a single invoice
- Payment decorrelation: make it harder to correlate payment hops
- Dual-funded channels: both parties contribute funds to a channel
- Fee hooks: decide fees at the time we broadcast
- Watchtower protocol: have third-parties monitor the blockchain for you. Still unclear how to reward them for this service.
- Bitcoin information relay: notify peers about block headers and channel closes. Instead of having bitcoind be your only source of truth.
Building Satoshi’s Place with LightningK0ala
Idea: make it rain satoshis, with a web page animation
Idea: Street fighter type game, where the relative health is the channel capacity
In 2005 Million Dollar Homepage to raise money for creator’s college costs. $1 per pixel. April Fool’s day 2017, Reddit Place was a 1 million pixel square canvas.
The game became very popular, Reddit engineers had to implement rate-limiting to reduce load on their servers.
Satoshi’s place: 1 satoshi per pixel. Roughly 1 satoshi = $0.00006498. Dynamic pricing would add complexity. Goal was to provide cheap, simple way to use LN for first time users.
Backend: connecting the clightning node, database, websocket server.
Frontend: did performance testing to render pixels, settled on PixiJS
Worked on zooming in/out for drawing. Lots of ideas came to mind, write those down but focus on getting the app MVP working, iterate later.
Took inspiration from yalls.org for how to do payment.
Friend wrote Python script for uploading images.
Hardware: Ordoid-XU4 + active cooling. 2TB SSD.
Lightning Applications on the Cheap with Elaine Ou
Not best security practices, not optimized for that.
Unfairly cheap hardware potential challenges:
- Unreliable, insecure connection
- Non-static IP address
- Low-bandwidth connection, no full node
- Computational resource constraints, no full node
Separation of Concerns
Bitcoin Node -> Lightning Node -> Web Server running on hardware, maybe connected to controls like a vending machine -> Customer.
How to best put these together?
Put it all on one machine
You can put a full node, LN node, Lapp on one single Rpi.
- Storage requirements
- Network challenges remain, unreliable LN channels
Basic Setup: Raspberry Pi
OS: Ubuntu MATE or Raspbian
External HDD: 200+ GB (less with pruning)
- Power adapter needed
- Can use microSD but flash degrades over time
- Bitcoind will need HD swap space
Remaining problem, port forwarding? How to accept inbound connections?
Light Client Mode
- Hardware constraints relaxed
- Don’t need a full Bitcoin node to run a LN node
sPRUNED: pseudonode, bitcoind emulator using electrum network, can be used with clightning and bitcoin-cli, sup-obtimal: has to download full blocks, high bandwidth cost
Neutrino: privacy-preserving light client
- BIP 157: Full nodes generate filters on block data
- Client downloads and validates chain of block headers
- Client downloads and validates chain of filter headers
- Client determines whether a block contains relevant info
- Can download block from any source
- Less likely to miss relevant transactions
- Filter-matching done by client, not the remote node
- Filters don’t need to be stored
- Filter bandwidth ~70MB per month
LN node’s port needs to be forwarded if you want people opening channels to you.
The webserver / lapp needs to port forward too.
Your home internet router needs to be configured too.
Minimize Trust while Maximizing Mobility
Separate the different pieces of the stack.
You can have a Bitcoin node running elsewhere that the LN node connects to remotely.
C-Lightning depends on the bitcoin-cli utility so you still need to install that.
LND can be remote, for security a tls certificate is generated, give the certificate to the app server to access the LND node.
Example: Lightning Peep Show
Users can pay with LN and view a live stream from the camera.
Hardware is a raspberry pi, raspistill -> mjpg_streamer. Converts images into a video stream
Webapp should have a stable connection
The Lapp creates the invoice, sends it to the customer. Customer pays invoice with mobile LN wallet. When the invoice is paid, a websocket is opened, camera streams to the device.
Separating all the different pieces (bitcoind, LND, lapp, web server, clients) gives you flexibility, many different clients can connect to the webserver and not overwhelm the rpi LND node for example. You can have many LN nodes getting info from one bitcoind node.
Building Applications on LND by Alex Bosworth
Design Philosophy of LND
- Works end-to-end: you can just pay and receive, done.
- Community developed, releases on the cutting edge.
- Soup to nuts APIs: signing, chain, logging, path finding. Not just LN APIs endpoints, comprehensive solution.
- Current best option: gRPC. Upgraded version of RPC. Superfast, small data, streaming. Downside is there is less support across platforms, not as easy as just a REST request.
- In development: embed LND for mobile, etc
- Future: WASD support, install as a library
- Use Bitcoin Core (C++) in production, most mature. btcd (golang) is very useful for testing.
- Neutrino: validating, more private SPV, sync in minutes. The solution for mobile phones. Need to solve incentivizing filter providers.
- Future: full node + neutrino to double-check, pruned mode with a full node, authenticated Neutrino.
- New seed format, AEZeed Format: BIP 39 + birthday, version (upgradable), KDF for more secure encryption. Customized BIP 39 to take into account feedback.
- LND is 2 wallets in 1: chain and channels. There are no good chain wallets. Massive challenge to make a good LN wallet and a good chain wallet. LND is trying to do both.
- Chain aspects of LN are tricky and multi-stage.
- To receive, you need to get in-bound channel capacity. Problem, working on ideas on how to improve that.
- Limit min-chan size, you want to have just a few big channels, no many small channels, that’s the whole point of LN!
- To send, choose stable channels. Not the biggest ones! Currently, you need to look at the graph to manually optimize.
- The future is autopilot, outsourced channel hassles. Better UX.
- Payment requests vs invoices. Domain-specific vocabulary. Payment request is generated from an invoice. The invoice includes the payment preimage, but not included in payment request! Invoice is your personal local version, payment request is “public”.
- Normal features: expiry… (missing stuff here sorry)
- Listen to the invoices subscription: internally, externally. You can notify payment success faster than the user would know from their node, because of direct connection.
- Move received payments through external db stages, business logic. Full invoice lifecycle
- Guard pre-images until payment, then push them out
- Query routs (local graph) to see if a payment is possible
- You can send to specific routes, implement mission control, maybe you want to reward your friends with fees :). In the future LND will store route success statistics, manual control misses out on this learning.
- Chain sends are supported, future is splice-out, submarine swaps
- Chain balance: vanilla, sweeping HTLCs
- Limbo balance: timelocked, limbo, pending resolution / stuck
- Channel balances: reserve, minus commit fees. Your effective balance goes down or up due to fee estimation changes.
- Everyone sees everyone else with public channels
- Channels have dual sided relationships: outgoing edges. The two parts each have their own policies, like fees, minimum HTLC, % fee that’s allowed, max HTLC size. Disabling channels is possible.
- The future of apps is off the public graph. App developers would keep all their channels private, they select their routing nodes.
- Hot wallet locks and encrypts, so you don’t leave an unencrypted on disk
- Macaroons scope run-time permissions, for example read-only access, only create invoice — great for app. Put invoice-creator macaroon on app server.
- Stay online, commit smaller amounts.
- No formal backup system at the moment
- Limit funds, shard across wallets, use iptables
- Use RAID mirroring to back up, but the future is a distributed db
- Static backups coming soon
- LTC support is included, more contributors needed for Litecoin though
- It’s fairly straightforward to swap in and out
- In the future the routing network will take care of it
- Don’t make a closed source wallet
- LND has a wallet project you can fork, but it’s GPL
- Try to be a good citizen, don’t write a wallet that destroys privacy by sending data to your server