Bitcoin Block Reorgs Explained with Adam Back and Bryan Bishop

WBD106 - Interview with Adam Back & Bryan Bishop (Banner).png

Where to find the show

iTunes | Google | Spotify | Stitcher | SoundCloud | YouTube | Deezer | TuneIn | RSS Feed

Download Episode MP3 File
The file will open in a new window. Click down arrow to download the file.

Interview location: New York
Interview date: Sunday 12th May, 2019
Company: Adam - Blockstream | Bryan - Independent
Role: Adam - Cofounder | Bryan - Core Developer

Following the recent hack of Binance for over 7,000 Bitcoin, a debate was sparked on Twitter regarding returning the funds using a block reorg. While this topic may have been discussed in the past, being new to Bitcoin, I was unaware of the history.

There are many technical aspects to how Bitcoin functions and to the less technical it can be challenging to grasp. I was unaware that block reorgs happen regularly in Bitcoin and why.

I caught up with Adam Back from Blockstream and Bitcoin Core Developer, Bryan Bishop to understand more about what block reorgs are and why large reorgs to reverse transactions are bad for Bitcoin.

TIMESTAMPS

00:03:25: Introductions 
00:04:09: Delving into the background of Bitcoin re-orgs, why they exist and have they occurred before
00:08:42: Exploring the recent Binance hack and some of the issues with re-orgs after hacks 
00:12:43: Looking into the deeper implications of re-orgs and refuting some of the suggested solutions
00:18:31: Discussing the role of miners in re-orgs and the similarities to a 51% attack
00:26:33: Exploring key learnings, including confirmation thresholds, game theory and Bitcoin history
00:33:08: Delving into the geopolitical risks associated with Bitcoin re-orgs
00:37:10: Exploring potential solutions where mining equipment could automatically reject re-orgs
00:42:00: Discussing if wider adoption of BetterHash or other software would help secure the network
00:46:44: Considering whether changes to mining or proof of work are needed in the future 
00:49:22: Summarising why block re-orgs are bad for Bitcoin
00:51:00: Discussing how many users should be running full nodes in the Bitcoin ecosystem
00:53:31: Touching on future technology of running full nodes on smartphones
00:56:04: Final comments

 

SUPPORT THE SHOW

If you enjoy The What Bitcoin Did Podcast you can help support the show my doing the following:

If you are interested in sponsoring the show, you can read more about that here or please feel free to drop me an email to discuss options.

SPONSORS

Patron Sponsors

And a big thank you to Rise Wallet

THANKS

A big thanks to my WBD Maximalist Patrons for helping support the show: JP Petit, Logan Shultz, Seb Walhain, Steve Foster, Tony, Gordon Gould, David Burlington, Jesse Powell and Wiel Menger.

TRANSCRIPTION

Peter McCormack: Adam, how are you?

Adam Back: Good!

Peter McCormack: How you doing Brian?

Bryan Bishop: Doing good!

Peter McCormack: So we're here to talk about re-orgs. Something I'd heard about before, because didn't the BSV chain have a couple of re-orgs recently?

Bryan Bishop: I actually don't follow Satoshi's Vision that well!

Adam Back: I think that was accidental. The side effect of having an extremely large blocks, kind of analogies to having a very fast block interval. You have a very fast block interval, you get re-org, some of the alt coins did this. So if you have an extremely large block, it has the same effects, which is the transfer time for the block becomes more than 10 minutes and then you get random re-orgs. So they had an effect like that basically. It wasn't a policy thing, it was just a technical issue.

Peter McCormack: It's a new thing for me to learn about. In the last week or so, it lit up Twitter for a day. So probably the starting point is to just explain what a block re-org and also has there ever been one on Bitcoin?

Bryan Bishop: Oh, of course there's been many Bitcoin re-orgs. In fact, it's part of the design of Bitcoin that allows for re-orgs to occur in the first place. A re-org is quite simply a situation where a new best chain tip has been discovered and client nodes on the network switch to that chain, which might have different transactions.

Although it's not necessarily true that a re-org will not include transactions or or will always have the same transactions. It's up to the miners that make those blocks to choose which transactions go into the blocks that appear in a re-org.

Peter McCormack: Why do re-orgs exists? You said there's been a few in the past. What's the reason?

Bryan Bishop: The reason why re-orgs are a thing in a system like Bitcoin, which is a decentralized consensus system, is because there's actually no formal authority to say what the correct chain really is. These alternative histories get proposed all the time in Bitcoin and many are checked and then most of them are invalid and that's why the proof of work rules, do not allow it to be accepted.

Adam Back: Yeah, I mean it's a side effect of a Bitcoin being a [Inaudible] process. So you're sort of tossing coins and trying to find a solution. The other side effect that causes it, is it's a decentralized system, so there is no logical clock, there's no central correct time and so if two miners... The premise of the system are tuned such the blocks are mined on average every 10 minutes, but occasionally they will happen within a few seconds of each other.

It can happen that they're found close enough together, that different parts of the network see different blocks first and then it's just an accident. You have different views about what the current time is and so there has to be a tiebreaker. The tiebreaker is that you build on whatever you think came first and we'll see who wins! And that's how Bitcoin works. So it's a bit random at the edges and so you get small reorganizations as part of that.

As Bryan was saying, typically if they're running normal software, they're just going to put the same transactions back into it. But occasionally somebody will pay a higher fee in the meantime and that would actually displace the original transaction, that was like one confirm and replace it with a different higher fee paying one, that becomes one and then two confirm.

Bryan Bishop: In fact, it gets really interesting because in certain situations people have broadcast different versions of "the same Bitcoin transaction". But realistically in event of a re-org, a miner can actually choose to include any version of any of the value transactions they've seen in the past.

Peter McCormack: Does that mean transactions can get lost?

Bryan Bishop: Transactions could get lost in a re-org, although it's somewhat unlikely. The way that the software is written is that re-org, is by default in Bitcoin core, if you're mining, you're including something from your mempool. Only if you have custom software, is it a situation where you're likely to exclude a large majority of the data that people have already seen. That's the default.

Peter McCormack: So you do transactions then go back into the mempool?

Adam Back: I mean generally speaking there is a mempool and things are taken from the mempool when they're inserted into a block. If a block gets reorganized because there was a race, where two are found at the same time, logically they get extracted back from the block that's now undone and put back into the mempool and you see them come in, in a new block. But I think you get that by side effect. But the overlap, the transactions that are not in the new block, that were in the old block, get reinserted into the mempool.

Bryan Bishop: So re-orgs are a design aspect of the Bitcoin system and it's actually really important that merchants and other companies using Bitcoin, implement re-org handling; the ability to handle these re-orgs and account for this during the course of their normal business activity.

If they don't do this, even normal re-orgs can actually cause problems for some businesses where they don't expect a one or two black re-org, which are actually far more frequent, exponentially more frequent, than any other block re-org. A 100 block re-org is virtually unheard of compared to a 1 or 2 block re-org.

Peter McCormack: So these are happening quite regularly then? It's a natural part of Bitcoin?

Bryan Bishop: Actually over time the amount of re-orgs that have occurred has been going down I believe. There's actually not too many public sources that are tracking this data anymore, probably because people stop being interested in these numbers because they were dropping off a cliff. However, you do have to be prepared for them to occur.

Peter McCormack: Okay. This is a very different scenario from what was proposed for Binance, because this seems to be, what you're explaining to me, is a natural part of Bitcoin, in that it's really to do with propagation time. But with what was recommended with Binance and what was the name? Jeremy Rubin? This was actually suggested as a tool to reverse the hack. So what's different here?

Adam Back: Yeah, I mean it's not a new discussion in a sense that there have been many exchange hacks before and in each of those, this kind of conversation came up. Actually because Bitcoin's market cap was smaller and more coins were stolen, this is actually about a 100 times less money on the scale of Bitcoin, than Mt Gox in February 2014 and about 10 times less than the Bitfinex in August 2016. 

So just kind of a long... People resolved that there's nothing useful to do here and you just have to accept that Bitcoin is final because of a bunch of factors which we can get into. But basically, all of the infrastructure is set up to automatically just continue consuming and finalizing transactions. There's a lot of inertia and equipment that's just running away, mining transactions.

So it's very hard... The software is not designed to undo things, none of the infrastructure is designed to undo it. There's all kinds of side effects that would occur if you did and the side effects are both technical and economic and game theory. If you can do something well, the attacker can do other things and people who disagree with the re-org can do other things.

People are very incentivized to see it not happen because they've seen other coins have this happen and suffer a great loss of credibility as a result. There are also geopolitical issues, that you would establish a precedent that would erode one of the major benefits of Bitcoin being censorship resistant which ties back to this finality at the network level.

Bryan Bishop: So there are other incentive issues that Adam was mentioning, such as situations where in the event that you are doing a re-org to try to take back a heist, the thief can choose to make transactions that spend their coins, predominantly to pay miners to choose to continue to take the Blockchain with the heist that occurred. Then there's all sorts of other things that can be done on both sides, so it gets to be very complicated very quickly.

Peter McCormack: But this was debated many hours after the hack. So a number of blocks will have already been...

Bryan Bishop: Indeed! So in a previous company that I worked at... I guess I should step back for a second and say I actually believe that companies probably have a legal obligation to seriously consider how they do re-org handling both on the receiving end and then also making re-orgs in the event of a heist. I'm not saying it's a good idea for them to do this, but I do believe that they probably have the legal obligation to investigate this.

So actually at a previous company that I worked at, I did very thoroughly investigate this and put together a plan and implementation details for how to go about doing this. It looked pretty interesting and it required a lot of upfront work. If you are going to actually do this successfully within the few blocks that occur after a heist, you have to be prepared.

So that includes monitoring the Blockchain, having a very close eye on what's going on, and then also having infrastructure in place already, that you can quickly deploy to miners who already understand the situation and already know what this technology is. If you do not have this prepared and ready, then none of this is happening in a timely manner and it just won't happen.

Peter McCormack: And I guess the longer away from the heist it is, the harder it is?

Bryan Bishop: Oh absolutely. The more expensive and the harder it is, absolutely!

Peter McCormack: So theoretically it would almost have been impossible for Binance to do this.

Bryan Bishop: Yes, it would be impossible for Binance to do this after the fact.

Peter McCormack: But I guess people have discussed it because if it ever happens in the future, then perhaps if there's some kind of process in place, that the community could react? But we're not keen on it.

Bryan Bishop: I don't actually believe there would be a community reaction. I certainly believe it should be. But any company can unilaterally decide to try to implement the software and try to convince miners to implement it. Hopefully the miners would choose not to because they'll recognize the extreme risks this brings to the Bitcoin ecosystem

Peter McCormack: Because you could essentially then use this on any transactions?

Bryan Bishop: Well, first of all, this is coordinated mining. This is like a 51% attack. That's what we're talking about here. This is like a coordinated activity, that is actively against Bitcoin's best interests. It's something for short term gain to prevent a heist or undo a heist rather. It's a reactive security model and it is in spite of the long term situation that that would cause.

Adam Back: The problem is, as I was saying, we don't think it's really practical to do this, with or without planning in my opinion. Just acceptability to the ecosystem. Let's say on a technical basis, it's essentially impossible to do it reactively and on a social basis, it's implausible to do it with a planned approach, because the community doesn't agree with undoing things basically. This is not just philosophical or pedantic. It's very practical!

So the current financial world lives with imperfect finality and they have review boards that daily meets and decide how to socialize the losses and who takes a loss when they make a mistake, because you have the ability to do naked selling. So sometimes they sell things they don't have by accident, or fail to buy them in time. So in the Bitcoin case, it's a decentralized system, where you don't know who you are transacting with.

If things can randomly fail for policy reasons, people will game it on both sides of the equation and you'll be left with essentially a daily accounting mess to try and resolve with no identities. So I think it's really not compatible with Bitcoin's basic mechanism for operating. So there are the practical things, as Peter Wuille and Greg Maxwell commented on Stack Exchange, they wrote a summary of the scorched earth thing that Brian was talking about.

I guess you can post the link in the talk notes, where the attacker can also use the free money that he's stolen, to outbid you to pay miners, to not recover the theft, but to keep the theft. So that's why the scorched earth name comes from, like nothing is left, it all gets paid to miners.

But you have to consider... So what Greg and Peter was saying is that because of the coordination problem, you need over 50% or it's not happening and you're not going to get 100% because it's very hard to reach people and there's lots of unattended equipment. So let's say they started with 66%, two thirds of the hash rate, it's going to be running at half speed.

By the time, you do this high risk, unplanned thing with patched code, which is not really practical, it's going to take 24 hours to undo it because you're running at half speed and 12 hours has already elapsed. In those 24 hours, everybody knows that any transaction you make, you can undo it very easily. You just pay yourself, you make a transaction, swap your coins for other coins on the network or other assets that won't get undone.

Then you wait a couple of hours and then you send another transaction that pays higher fees to pay yourself, take the money back and you get away with it in the mix. So practically speaking, that means the entire exchange ecosystem is going to be down for 24 hours because people are going to notice this and they're going to suspend withdrawals, deposits, trading, and the loss of businesses implied by that is a factor of 10 or 100 of the loss.

So it's economically implausible from that perspective and these are just immediate things people will do. The other thing is, you could say, "well, what about blacklists", like Binance or whoever the attack is about, would advise exchanges don't transact these transactions. But the problem is that most of the ecosystem disagrees with undoing. So they may either just say, "no, I refuse" or they may pretend to do it, like say, "oh, we had a technical accident and we're sorry, but we swapped coins with this hacker and now we're the victim. Don't undo it because we're going to lose money!"

So given that there's so many people that really think that finality is important to Bitcoin, there's likely to be kind of willful, pretend victims. So if that happens, then you have to abort the undo because, you don't want to take money from the wrong people. Or if you do take away from the wrong people, somebody has to make them whole and if you don't know who to make whole, when you don't know who's lying, who's telling the truth, you've got an a forensic accounting mess.

We saw how long Mt Gox took to unwind, once you get forensic accountants in there, so it's just not worth it. It's not resolvable technically and it's only resolvable in a very slow and expensive and error prone way with forensic accounting that takes years. So I don't think it really works because Bitcoin is trying to do something very clean, with a fairly immediate, final settlement.

Bryan Bishop: So going back to the stack overflow link where Peter Wuille wrote a very interesting reply, I would like to read just a short excerpt here, where he says, "pretty interesting idea." He says, "in the extreme scenario where miners are coordinating a rollback and it's actually happening, the public and the community has time to react.

If a sufficiently large group of economically relevant parties in the network refuse to accept the rollback despite the large amount of hash rate behind it, miners would have no choice but to go along with this. This is a last resort option, sort of like a user activated soft fork."

Peter McCormack: How do they reject it though?

Bryan Bishop: Well, a user activated soft fork in this scenario or any sort of user activated response, would have to involve a source code change and the users would have to choose to run that software.

Peter McCormack: On their nodes?

Adam Back: Actually Luke gave an example, using a shell script to call invalidateblock in the loop, so that if a full node tries to go backwards and build on a previous transaction say or something like that... Anyway, it basically involves calling invalidateblock for any blocks you don't like and blocks you don't like are blocks that try to redo the transactions that were cancelled. So basically they could call invalidateblock, which is an existing RPC handle, but more cleanly you would want a patch because people don't want to run shell scripts and things like that.

Peter McCormack: Just run through the scenario that somebody chose to implement a block re-org. What actually practically happens? Do they go to the block previous to the hack or are they doing something new in the current blocks? I mean, you've got to remember, I don't know this stuff.

Bryan Bishop: So to be clear, a constructed re-org would be something that starts from the block before the heist occurred and then also it would have to include either enough sufficient blocks with enough proof of work attached to them for it to be considered better, than whatever the current main chain is considered by other nodes on the Bitcoin network.

Peter McCormack: And how would you actually coordinate having miner support for this?

Bryan Bishop: Yes, so that's an interesting question. How do you actually coordinate a 51% attack? Well, let me clearly outline it for all of our listeners here. Here's exactly how you attack the Bitcoin network. What you do is you get a very giant mining pool and you centralize everything and you tell all the miners to connect to this mining pool and then you construct whatever blocks you want and then Bitcoin dies.

Peter McCormack: So a block re-org is just a 51% attack?

Bryan Bishop: Yes. However, it's important to note that re-orgs are... When people talk about re-orgs, they're usually talking about non-adversarial re-orgs that are just a natural consequence of the Bitcoin rules.

Peter McCormack: But if you could coordinate 51% for a block re-org...

Bryan Bishop: By the way, I think it's important to note that actually when we say 51%, we actually don't mean 51% because there's actually situations where significantly less like 26 to 31% can actually do the job quite well.

Peter McCormack: Because?

Adam Back: Oh well, because of something subtle to do with the optimal game you can play. Somebody noticed, the selfish mining paper, they noticed that if you withhold blocks, you gain a strategic informational advantage and because of that and assuming that the other party doesn't adopt a counteracting strategy, which they would, so anyway it opens up a new branch in the game tree for behavior of mining and selfish mining.

Surprisingly to many people, almost everybody I think basically or almost everybody, I think there might've been one, it's kind of a bleak post on Bitcointalk a few years before that seem to have some ideas in the same direction. But generally speaking, it was a surprise that you can get an advantage. But if you've got multiple groups of miners that are colluding together, you can get two groups or three groups that are all trying to selfish mine and then it would start to cancel out.

So it was a bit of a dynamic situation. But generally speaking, I think Bryan's point is that if you had 51% fighting 49%, you're going to be there for months, until you have outbid the last 12 hours of work. Because it's going to be progressing at a rate of 1% and it's going to look like very low hash rate. So what would have been a 2016 block, a 2 week difficulty adjustment would potentially stretch into months of... You wouldn't try that because it would just be a mess.

Bryan Bishop: In the far future, there's also other re-org issues like this that occur in a situation where the Bitcoin subsidy has reached nearly zero, because if you make a Bitcoin transaction that has a high amount of fee attached to it, the miners actually have an incentive to do re-orgs to try to undo that block and take the fee for themselves and so some thinking needs to be done around that.

Peter McCormack: So it just seems to me like there's nothing in these block re-orgs. There's nothing here to be done. There's nothing here to be talked about. Okay, let's have the debate. Let's get it out there. Now it's done, there's no practical solution.

Bryan Bishop: That's correct. But there is important takeaways that users should hear from this, especially for merchants and people running Bitcoin businesses. I absolutely must stress that it is so important to have a high confirmation threshold for transactions that you accept and to please test your infrastructure against re-orgs. You can use Bitcoin Core RegTest. There's re-orgs that are very easy, very cheap and trivial to make in RegTest mode and every company should be testing that.

Peter McCormack: How many confirmations are enough then, because most people tend to operate with 6 but you're...

Bryan Bishop: So actually the answer to how many confirmations are enough is actually pretty interesting. It depends on the business. If you are working with miners and you have a somewhat adversarial business where miners are maybe hedging some of their Bitcoin against their mining expenses, then your actual confirmation threshold should probably be at least 100

In fact, I think that's the same for a Adam's Liquid network, which is about a hundred confirmations. Other situations though, I actually recommend scaling the confirmation threshold requirement based off of the quantity of Bitcoin in the transaction.

Then in other situations that's based off of monitoring the weather or the other events going on in the Bitcoin network, if this is a particularly tumultuous time where re-orgs are occurring, perhaps you should increase that number so that it is higher than the average re-org size lately or re-org depth lately.

Then finally, I've seen a number of companies or even individuals try to get away with zero confirmation transactions and the only way to do that safely is through the Lightning network at the moment. You should not be accepting zero confirmation transactions.

Peter McCormack: Actually Bitrefill accept that. I think they've calculated the risk.

Bryan Bishop: Yes, you can calculate the risk all you want, but there's still going to be damage to the community if your business, either fails or has some sort of consequence downstream for other people accepting transactions from Bitrefill or things like that.

Adam Back: Yeah, I mean I think it depends. Doing like derivatives or exchange trading is quite different because there's large amounts of money at stake and you're giving people the opportunity to extract value immediately in large quantity, and there you've provided them with the footholds to execute an economic attack if they're so inclined.

But for, I think Bitrefill is more like selling online goods and physical goods as well. If you are selling copies of PDF reports, your cost of production is effectively zero. So even if you had attempts at fraud, you might not really care that much. If that made it more convenient to buy and you made more than 10% more sales because of it, you might tolerate that.

You might also see a same kind of calculus in a coffee shop that you're like, people don't want to steal from you to face because you realize like, "oh, I just like walked out without paying for this coffee, that was sneaky", "we'll recognize him next time!" There's a social stigma attached to it.

So for low cost production things, that can make economic sense for a merchant. So there was a kind of previous era argument about RBF, where some people wanted to completely deprecate unconfirmed transactions and others wanted to say, "well, we can tolerate a certain amount of fraud in our particular use case because the cost of goods is zero, it's like electronic."

Bryan Bishop: So briefly talking about exchanges, there's actually a consideration for re-org attacks and exchanges. The attack goes like this, someone deposits a large amount of Bitcoin to an exchange, trades around and withdraws in another currency and then there's a re-org attack and the deposit that that exchange was expecting, no longer exists in the Blockchain.

This is how you can steal money from exchanges! Now there's some mitigations. One that I devise and I'm sure other people thought of it as well, is that exchanges that process withdrawal requests should taint their withdrawal Bitcoin transactions with references to the deposits that they recently received. This way if the deposits go away, then the withdrawal transactions are no longer valid.

Adam Back: Nice. I like it! I mean there are lots of interesting technical tricks that... So you asked the question, is it reasonable to elaborate and dig into this and I think there's no discouragement for technical discussion ever. There's no one reasonable question. It's like aircraft crash investigations, any reasonable question, any question should be answered. There's no unreasonable question! 

People who go through aircraft crash and aircraft systems line of business are indoctrinated and taught that you must question everything. You must not practice differentiality to senior people. The intern at the company should feel free to point out an obvious flaw that the CEO is making and not be cowed by the tone of his answer or something.

It's important because people are going to die and planes are going to fall out the sky, if people don't ask questions and Bitcoin has that kind of characteristic. So it's certainly useful to talk about it and to understand the game theory and to understand the history of why different things were considered a bad idea, because unfortunately with Bitcoin it's a very technical sphere with a wide range of topics.

Often the thinking behind design decisions is not explained. Bitcoin didn't come with design consideration documents, people kind of reverse engineered it. If you want to know like a shortcut to like, "well, what was the design consideration or something?" Typically the only way to find out, is to go on Bitcoin-wizards and ask people until they explain it to you. That's the fast path. The slow path is you read it for yourself and you think very hard and try and figure it out for yourself.

Maybe you'll make another variant or another realization, so it's kind of interesting, but it is useful to think about the trade offs and for people to be aware of the technical possibilities. If you ever do network programming or design of network protocols, it doesn't just have to work, it has to work in any conceivable however contrived network condition and network conditions are not clean! Sometimes a network can be connected in one direction and not another or intermittently.

So a very strange partition or partial partitions. So you have to design assuming all these things and Bitcoin is designed in that way and coded in that way. So it's important to explore all possibilities. So I think we talked about technical things, like technical countermeasures, economic counter measures, and there's another topic that arises from this, which is geopolitical risk.

So if a 51% attack or an undo were to get executed with or without planning and not just fail for some reason, it would set a policy precedent and that policy precedent would, to my mind greatly weaken the Bitcoin's credibility as a store of value and as a censorship resistant, electronic cash payment system. That's because many of the features of Bitcoin like censorship resistance, unfreezeability, unceaseability, could be undone by a policy like this and it's a slippery slope.

So this one, you can say, "well, it was an exchange, they probably really were hacked. They're probably not pretending they were hacked so they can keep the money or something." The next time maybe it is an exchange, but it's far less clear whether they actually stole the money themselves or not.

Or next time it's a Silk Road version and some people want to see that undone and some people don't. There are lots of geopolitical arguments and it's just a slippery slope. It will get easier and simpler each time until, people are wanting to undo it for a parking validation or something like that. So it kind of makes a mockery about what electronic cash means. Electronic cash means cash, you hand over the payment and it's done.

You can't undo, "oh sorry, I want my money back." There's a cost. If you have a payment system, where you can contest and undo payments, that's like credit cards and the fees in credit cards are not just high because companies are trying to make a profit or rent seek...

Peter McCormack: Chargebacks!

Adam Back: ... They're high because there are chargebacks, they're high because there's an insurance and investigation cost to validate whether a chargeback is okay or not and it gets passed on to consumers. If you get undo's in Bitcoin, it loses all these interesting properties that are baked into the way people use it in exchanges.

So it will basically come back to the 24 hour daily global, review board, looking at which transactions we believe should be undone or not, by these review board people and that's not Bitcoin. That's the status quo.

Peter McCormack: That's EOS.

Adam Back: Well, I mean it's even before that. It's how traditional exchanges and clearing houses work. They have technical failures and there are Blockchains that operate literally in this way. You mentioned EOS, I think is one that has a set of humans that make policy decisions and block transactions and things like that. That's considered by most people to be inadvisable on a legal basis and also not very interesting, because it's subject to human judgment and policy and that can be gamed in any direction.

So I think that that is also a part of why people are not keen to see re-orgs and you could probably construct some even more interesting arguments. Also the other thing to bear in mind, is that this is just us talking off the top of her head, the game tree of interesting and counter intuitive and unthought of, counter attacks. Somebody could and likely would do, if some of these things were to be considered, is much wider than we've covered here.

So in particular, if the users are extremely against this pivot of Bitcoin away from cash finality and into some kind of different network that has policy level undo’s with some authority, users are not going to agree with this. They are going to be a number of things they could do in terms of preventing it. They could make protocol extensions or use semi-automated mechanisms to prevent it and I think a couple of people have suggested the mining equipment itself could reject re-orgs.

So it turns out that Luke Dash Jr had something like this in a BFGminer software. So BFGminer is the original firmware for ASICs when they first came out, as a control board and it's running some software and Luke wrote the first version of that software, which is a driver for the hardware, but also enough software to connect to a mining pool and drive the software.

So typically you configure two pools and what he wrote is that, and that's in case one of them fails, you don't want to be dead and not receiving any funds while you're waiting for the pool to reboot. So you auto switch to the next pool, if one stops responding. But he also added an interesting feature where if the pool asked you to mine backwards, like at a lower height or something like that, something with that side effect, it would automatically switch to the other pool. 

So in a scenario that Bryan describes where people try to get a popular pool or a group of pools and change the policy, any minor using BFGminer, would ultimately switch to its backup pool and avoid that outcome. So if you philosophically think this is a good thing, you would install that software or other people would add features like that deeper and deeper into the miner firmware or even into the ASIC to make hardware that is a ratchet. Like the hardware itself doesn't know how to and has no way to go backwards.

We were talking earlier about the very far future, some decades down the line, where Bitcoin's price and deployment and adoption has reached the top of its S curve and the price will stabilize, plus or minus. Then there's a question about securing the network from reward. Reward will be less, because of more halvings, price will be stable and will that be enough to secure the network?

So there are fees, but another thing you could do to keep the network flowing, is to make equipment that can't undo things. Because then if there is a policy question about we want to undo this or some element that wants to undo it, they have to replace the hardware.

Bryan Bishop: Yeah, but you can't force people to use certain hardware though.

Adam Back: Right, but I think the point is it comes in the similar line of defense as us observing that you could do something, if you had time to plan ahead and people have two modes of thinking. One is the panic when they want to undo a mistake and the other one is the calm water steady state, where people agree that Bitcoin should be final. We agree, we understand how that works, that's how we've architected our system.

So it's much harder to find a political world to pivot Bitcoin's model to an undo model. So if people put in systems in different parts of the software stack, like at the protocol level in the miner, in the pool protocol, in exchange operation, there's a lot of inertia in software and to overcome inertia on a dime because you're panicked about something right now, it just makes it harder and harder. 

Particularly, if people agree when they're in the, let's make Bitcoin secure and final mode of thinking the normal, steady state, then yeah that feature should go in miners and a bunch of miners have it, let's say that happens. Now when you get to the panic mode of thinking, it's much more difficult to do it, because you have to disassemble the equipment and change it or maybe you can't, you have to replace the ASICs and then there's a different cost, which is not just the electrical cost but you have to replace $1 billion worth of ASICs or something.

So that would be a useful function to my point of view, because it would mean that Bitcoin could be secure, with less hash rate or less economic incentive. It's also interesting in a sense of this dichotomy between people's thinking in a panic mood, where something bad is happening, something unfortunate has happened and they feel remorse for the person that has been stolen from, they would really like to help them out.

But the side effects have receded from their mind because the immediacy of the problem, it has analogies with the Bitcoin genesis block, which is about, the chancellor on the brink of second bailout for banks. Now that same chancellor, Mervyn King had written really long essays in the Times of London in the months before that event, where he had really lectured his viewpoint that this is moral hazard and really should not happen.

Yet the same guy was basically forced by political expediency, because of the panic of the situation to go back and do the opposite. So it's kind of panic mode thinking. So Bitcoin's genesis block says that panic thinking is bad! Look at fiat and what happens when you do panic thinking and here are people trying to introduce this kind of problem into Bitcoin. That's my point of view.

Peter McCormack: Would this scenario have been different if, say, there was wider adoption of Matt Corallo's BetterHash? Would that change things?

Adam Back: I think it's interesting because it would complicate greatly the coordination problem, because what BetterHash... And at Blockstream we worked on something of similar design, which didn't get completed to fruition called Open Hash, but it's a similar idea.

It's basically the protocol is trying to give the decision about which transactions to put into a block, to more decentralize that and put that into hands of individual power users and miners, where they can still use a less decentralized pool to reduce variance and so if you've got more independence in choosing blocks, now if you want to assemble 51%, you have to coordinate with more people and all these power users are far less likely to follow the expediency to date.

Bryan Bishop: So I recall vaguely, it was probably a Blockstream idea, to have mining equipment that actually had some checks and balances built in. I'm wondering is that an extension of BetterHash or was that even before BetterHash and Open Hash?

Adam Back: So that was Greg Maxwell's idea and I don't know if it predated Blockstream or not, but anyway, it came out during a period where there were new mining manufacturers joining the market and this is a similar kind of question, that a lot of the equipment is in other people's data centers these days. In the earlier days, people were GPU mining, when they had the ASICs in the garage and it became so industrialized that realistically nobody had physical control.

So if you have your miners in a data centre, the data center operator could just change the rules. He could packet filter the network and impose from the network, so you wouldn't know. Or I could go and change the configuration on all the machines and they have tools to automatically change the configuration of thousand machines, at the click of a button.

So Greg's thinking there was, well, you could make a kind of smart contract, this is what Bryan was saying, where you tell the ASIC that, "hey, I'm the owner of this equipment and it's going to mine for me for the next month, because I paid for it and there's nothing the host can do to change the miner configuration." All they could do is switch it off, they can't change it.

So he came with some clever ideas of how to put something in the miner NVram or flash memory, in a very embedded, hard to get out place that would refuse to be changed until it reached a certain block height, where if you're leasing the machine, it would expire after a while. If it was your machine, maybe it would never expire.

Peter McCormack: I guess there isn't a game theory for miners anyway to implement these large block re-orgs, because if you can 51% attack the network, you kill the network?

Bryan Bishop: So one theoretical source, that perhaps is not too theoretical, is the idea of a miner that is actually incentivized by things other than Bitcoin. The idea though is that hopefully Bitcoin is valuable enough that miners are always going to be incentivized by Bitcoin.

Peter McCormack: I guess nation state attack is the only other incentive?

Adam Back: Well, I mean there are other coins that use the same proof of work function. So some coins have different proof of work functions and that's a trend lately, but there are some competing sha256s.

Bryan Bishop: In fact the existence of alternative Blockchains that use the same proof of work function, is actually a bad thing, because that actually splits up the hash rate onto multiple networks and it lowers the cost of forging an alternate history. The fact that we have, I think Peter Wuille calculated that we'd done at least 2 to the 80, or even 2 to the 90 cycles through the Bitcoin network by now, which is actually a proof of principle that certain forms of 80-bit or 90-bit security can actually be broken.

That it is also a form of proof of principle for Bitcoin that someone could amass a lot of mining hash rate and make that available. The fact that we're splitting it up between multiple Blockchains is actually really bad. I was working briefly on a project with a number of other collaborators who prefer to remain anonymous I think, on a project called fortune.tech. The idea was that it would automatically generate for a customer, a new fork of Bitcoin and the idea was that hopefully this would actually lower the value of all these forks of Bitcoin to zero.

Unfortunately, that's not what happened. They just said, "thank you for doing this for us. Thanks for doing all the hard work" and then they went and made a lot of forks of Bitcoin. But nonetheless, in some cases this means that the hash rate is now split up and there was of course a hash rate option to change the proof of work to script or something. But nonetheless, people can do whatever they want.

Peter McCormack: So do you think there's some required changes now for mining? I mean, can you actually even change the algorithm or does that require new hardware for everyone? How does that work?

Bryan Bishop: There are some ideas for changing the proof of work function for Bitcoin. In my opinion, the way that is most likely to go forward, is a situation that I would call a blended proof of work, where you have first the original sha256, double sha256 proof of work function.

Then you also soft fork in another requirement for an additional proof of work. Then over some time period this changes the proportions or something and then in the far future perhaps, you can stop the sha256 mining. But that's really far in the future!

Adam Back: Yeah and I think that changing the proof of work, to achieve ecosystem consensus for that, there would have to be a major defect with the current proof of work and it doesn't have a defect at the moment. It looks like it's good for years to come. So I think you would need to see something like, some of the very early hash functions, particularly like MD4, had some pretty bad cryptographic attacks and it became weaker and weaker over time.

So if you saw sha256 heading in that direction in a few decades’ time, then it might be time to start phasing in a new hash function. There are a couple, I mean sha3 is a new hash function, so they exist. But I think it's a difficult thing to see happen because there's a lot of people who own mining equipment and so if there was a change, it would need to happen over the course of years so that equipment was already obsolete for a tech effect.

There also seems to be cases in other coins where people are attached to the proof of work. They consider the 21 million coins to be a defining part of the protocol and I've certainly seen the case where, in other coins, people talked about changing proof of work and there's been heavy resistance to it, because they considered it to be a defining part of the network and they didn't want it to be messed with.

So I know it's a very dramatic thing. I don't think anybody's even mentioning that in this context generally speaking, it's just a kind of technical curiosity, as to in a very far future, how you could get algorithm agility, how you could phase it in and could it be done in a way that doesn't disrupt the functioning of the network or something.

Peter McCormack: Okay. So basically block re-orgs are bad for Bitcoin, they're antithetical to Bitcoin, they're impractical, unreliable and should just be dropped and forgotten?

Adam Back: Yeah I mean, I think the game theory doesn't really work out, because to react without foreplanning is too risky. Your chances are that it's going to be worse. It's probably going to technically fail. You're going to be disruptive to the network for days or weeks even while it's happening. So the Bitcoin economy of exchanges and payment processes is going to lose more money than the hack. 

The last hack, I mean there've been an ongoing series of hacks a hundred times smaller than one of the earliest big infamous hacks, the Mt Gox issue and Bitcoin economy is much bigger. So it sounds like a lot of money, but there are individual exchanges that have made more than that in a week, for example, they don't want to lose a business.

So it is certainly the case that for the whole network to switch off, it would be much cheaper for the exchanges to have a whip around and help somebody out, than to undergo this kind of disruption, which doesn't say anything about the longer term side effects. The geopolitical issues affect the other parties who try to abuse the situation while it was going on and just the negative news cycle that would be around it and the damage of Bitcoin as digital gold that's cash like. It's not cash like if somebody can undo it!

Peter McCormack: Okay. Well that's pretty good! Before we finish, I just want to ask you both about one, maybe two things following my chat with Luke. Something that came up that I wasn't fully aware of or didn't understand that the level that Luke explained, but Luke said he believes that for Bitcoin's security and future, that 85% of people using Bitcoin should be running a full node.

Bryan Bishop: Well, actually in my opinion, that number should actually be 100% of people, who should be running a full node.

Peter McCormack: But in the practical world?

Bryan Bishop: The philosophy behind this though is that if you're not running the Bitcoin rules, you're not actually running Bitcoin. Someone else has your keys or someone else is checking the blocks for you and you're not actually running the Bitcoin protocol. Bitcoin was not designed for that.

Peter McCormack: I know, but in a practical sense, as Bitcoin grows and more people adopt it, most people will use a hardware wallet. Some will keep money on the exchanges and yes, someone's holding their private keys for them, but I think getting to 100% of people to have a full node would be very difficult.

Bryan Bishop: So actually regarding exchanges holding coins and having custody, I actually disagree and I think that there will be centralized exchanges that have decentralized custody. Where either it's multisig with the participants or multisig with escrow agents or other ways of doing it. I think there's definitely options on the horizon.

Peter McCormack: I think reaching 100% people having a full node is unrealistic.

Adam Back: I mean I tend to agree and I think that you get most of the positive effects of everybody having a full node, by having some reasonable percentage of people having a full node, particularly if that's weighted towards people with more economically at stake. The analogy that has been drawn sometimes is it's like the paper cash counterfeit detectors in shops.

So if a few shops have them, counterfeit notes get detected. Not immediately, but after you take it out, the first shop doesn't notice, but the second shop does. So they get taken out of circulation or they get noticed. They get noticed because the person who accepted it, rejects it. I gave you the note and you say it's a dud, now I'm going to go back to the person who gave it to me and say, "what the heck man? You gave me a dud note and pay me some real money now."

So it doesn't take that higher percentage, it doesn't have to be 100% but 30% would catch that kind of thing pretty quickly. The people who have a lot at stake, like exchanges and payment processes, people selling high value items or doing LocalBitcoin transactions and traders, it doesn't cost them much to run a full node.

At Blockstream we have a product, while it's an Alpha stage thing at the moment, called ABcore, you can download and run it. It runs on Android devices, including a high end smartphone, where you can actually run a full node on a smartphone. Okay, you want to use Wi-Fi some of the time, because it uses kind of a lot of bandwidth.

But that tech is on the horizon and at the conference yesterday, HTC, which is a handset manufacturer, talked about their plans to provide a full node on HTC handset. So I don't know how they'll do that in terms of bandwidth, but maybe they can strike a deal with the network provider to exempt the Bitcoin traffic from your bandwidth allowance or something like that.

But it's possible, I mean, of course the transaction rates are, as people adopt Bitcoin, there's more demand for block space, the blocks get bigger. They're not at capacity yet. If people would adopt the tech, you can get a lot more transactions in a block, with batching and with SegWit adoption and Schnorr and Taproot and things in the future.

Bryan Bishop: So actually that HTC announcement was definitely about running Bitcoin Core on a phone. But also, another big aspect of that, was that they're planning to either release or open source a lot of their software that they're using on there and I think that's actually really interesting. HTC is a major cell phone manufacturer and it's great seeing... I mean this is huge for adoption. So I'm actually really looking forward to seeing how that unfolds.

Peter McCormack: How much space on the phone would that be using?

Bryan Bishop: Well in this case, they would of course be using a pruning, so that can be configured as much as you like. But it's quite trivial to get like easily a 100GB of SD card space on Android phones these days, so it's not a big deal.

Adam Back: I noticed the Chinese manufacturer called Pocophone. There are lots of new low cost phones with surprisingly high specs these days. So to Pocophone, it costs like $300 for an eight core, 64 CPU and a choice of 64GB, 128GB or 256GB on phone storage, plus SD card slot.

So phones are getting more powerful. I mean probably the bigger bottleneck, other than your network bandwidth, is the CPU to validate all that history. So it's probably going to be backlogged on the CPU, even with an ARM64

Bryan Bishop: That phone is better than my laptop!

Adam Back: Maybe you need a laptop upgrade...

Bryan Bishop: This is my crap-top! So I actually don't take my laptop from home. This is my crap-top that I take with me to conferences to type transcripts.

Adam Back: That sounds like a prudent move for security reasons. I approve!

Peter McCormack: All right. Well listen look, I think that was really useful for everyone. Thank you both for coming on at short notice, appreciated! You've both been on the past, so I'll just share out in the show notes everything you do, but thank you!

Bryan Bishop: Thank you!

Adam Back: Thanks!

PodcastPeter McCormackComment