There are currently two popular interrelated narratives on social media surrounding participation of the block making process on a public blockchain. The stories are most pronounced within the Bitcoin community but are also reused by Litecoin, Ethereum and other cryptocurrencies too.
This includes the unchallenged statements that:
(1) anyone can still participate in block making, it is ungated and “permissionless”
(2) following a reward halving (“halvening”), networks become more decentralized because large, centralized farms and actors split apart due to economic pressures
This post looks at both of these and show that in practice neither is really true as of April 2016.
Named block makers
A year ago I reflected on some of the debate surrounding permissioned and permissionless blockchains. Part of that post involved looking at how the mining market actually evolved in practice; not just based on the generalized claims made by enthusiasts at conferences.
For instance, based on block height below is a list of the first time a pool self-doxxed and signed a coinbase transaction, courtesy of Organ of Corti. Only the first 50 are chronologically included:
|| Block height
|Ass Penny Pool
|Generated by General
|ST Mining Corp
|For Pierce and Paul
Recall that even though it didn’t initially sign coinbase transactions, Slush began publicly operating at the end of November 2010. Eligius was announced on April 27, 2011. DeepBit publicly launched on February 26, 2011 and at one point was the most popular pool, reaching for a short period in July 2011, more than 50% of the network hashrate.
While many enthusiasts claim that “anyone can mine,” in practice, very few choose to for a number of reasons that will be discussed below.
But more to the point, the reason cryptocurrencies allegedly have a “permissionless” characteristic in the first place has to do exclusively with the fact that there is no administrative gating or vetting process for allowing actors on the network to participate in the block making process. In 2009 there was no whitelist, blacklist, KYC or KYM (know your miner) process.
That is to say, those wanting to create a block did not need permission from a network administrator. That is the sole context of the term “permissionless.”
It is not related to developing other platforms that plug into the network. It is not related to whether the network codebase is open source or not. It is not related to being able to build software products that somehow utilize the network. It is not related to being able to view or not view transactions.
Yet due to how the market evolved, today in 2016 while everyone is still paying for the high marginal costs to maintain a network designed for pseudonymous and anonymous interaction, few participants, specifically block makers, are actually capitalizing off of that utility.
(1) Acquiring the necessary hardware to become a profitable miner invariably leaves a paper trail. If instead you acquire the hardware on the second-hand market — in order to remain anonymous — you will still likely leave a paper trail with your legal identity in order to pay for the large energy bill and property taxes. This is one of the reasons why miners in locations such as China do not publicize their fundraising activities or annual revenue: they don’t want to leave a paper trail to pay any extra taxes.
(2) The other main mechanism for vetting miners now is through the use of data science itself. Roughly 10 companies globally provide law enforcement, compliance teams and regulators access to relatively robust analytics tools to track provenance of bitcoins (or other cryptocurrencies) back to coin generation itself. And in order to sell these mined bitcoins (e.g., to pay for the electricity and the mining hardware), nearly every bitcoin conversion to fiat marketplace now requires some compliance of local KYC and AML regulations.
While there are workarounds such as LocalBitcoins and SharedCoin, generally speaking the pseudonymous network itself in 2016 has largely become doxxed. Yet the high costs of maintaining pseudonymity, via proof-of-work, still remain.
Above is a pie chart that estimates the hashrate distribution among mining pools over the past 4 days (as of late April 2016). The 10 largest pools collectively made 97% of the blocks during that time period.
What about beyond 4 days?
Above is the pool distribution of the past year based on coinbase data aggregated by Blocktrail.
The 10 largest pools collectively account for roughly 91.6% of all block making activity. There is also a relatively long tail that includes roughly another 60 entities (some of whom do sign coinbase transactions) that represent the remaining 8.4% of all block making the past year.
Why do any actors sign transactions at all, after all, isn’t a core characteristic of a public blockchain pseudonymous consensus? To my knowledge, no one has formally published a thorough explanation for the reasons why. But one repeated rationale is that pools do so in order to prove to the miners (hashers) connected to the pool what the provenance of the block reward income is.
What does that mean?
For those who have never partaken in the mining process before, a quick history lesson: within the first two years of Bitcoin’s existence a division of labor arose in which block making became separated from hashing itself (e.g., generating proofs-of-work).
That is to say, the security of network security was outsourced to entities who create proofs-of-work and who are colloquially referred to as miners. Miners, in return for steady payouts of income, send their work to a pool operator who subsequently batches transactions together into blocks and pays workers based on a pre-arranged agreement (usually proportional, share-based).
Today, if average Joe buys ASIC mining equipment, he typically does not connect them to his own pool but instead connects them to a pool run by Bob the devops professional. And how can Joe trust Bob not to shave off pennies from each share of work that Joe submits?
Block signing in theory provides some semblance of transparency: letting the hashers know if pool operators are skimming off the proceeds by not accurately reporting blocks found (e.g., income).
For instance, if a pool operator makes a block based off of the proof-of-work submitted by one of the hashers connected to a pool, such as Joe, but does not sign the coinbase, the pool operator can try to pretend that it didn’t win the block reward in the first place and therefore would not have to pay the workers (hashers). This was allegedly more commonplace prior to 2013, before the advent of VC financed farms and pools. Now many of the medium and large hashing farm operators want to know the exact revenue number and hear good reasons for why some is missing or if the pool was just “unlucky.”
Why doesn’t everyone become a block maker, after all, the process is billed as being “open” to all?
There are multiple reasons why, but the most important reason boils down to economics. Dave Hudson has written about 10 different articles on the baked-in variance (inhomogenous Poisson process) that motivates individuals to continually pool their mining effort versus solo mine. Spoiler alert: you are likely to be struck by lightning before you will ever create a block and reap a block reward by solo mining off of your laptop at home.
Other reasons for why few decide to become block-makers include: the added costs of providing DOS protection to your pool and the need to hire competent staff that can prevent and be on the lookout for problems like BGP hijacking which results in lost revenue.
This has not changed for multiple years and will likely not change for reasons discussed below.
With the upcoming Bitcoin block reward halving that is expected to take place in mid-July, there is a growing chorus of ‘hope’ that it will somehow lead to fewer large mining farms and pools.
This probably won’t occur for several simple reasons, namely due to economic incentives.
Recall that the major reasons why mining activity itself has gravitated to locations such as China isn’t due to conspiracy theories involving lizards but instead ancillary costs.
Specifically the following factors:
- relatively low labor costs (e.g., professional hashing facilities need to be maintained by a workforce 24 x 7 and wages in China are lower than Russia and the US for this activity)
- relatively low property costs (e.g., if you have good guanxi, you can utilize and own land at rates below those found in parts of Russia and the US)
- lower energy costs; I and others have frequently written about this
- first-to-market with hardware; because a lot of the final assembly of hashing equipment takes place in southern China, in terms of logistics and transportation end-users have a lead-time advantage over other geographical regions
- close personal connections with hardware manufacturers and fabrication plants in China and Taiwan; acquiring hardware for mining cryptocurrencies is just as relationship driven as other specialized non-commoditized industries. Because medium and large miners know who the chip design teams are and what the ASIC roadmaps will be, they can stand in line at the front and acquire hardware before others.
What will happen after a block reward halving?
Just as oil producers with the highest marginal costs have been forced to exit the fracking market over the past couple of years, Bitcoin miners with the thinnest margins will likely exit the market immediately.
What this actually results in, at least the short run, is a more concentrated group of larger hashing farms and pools.
Because miners as a whole are effectively being given a 50% pay cut to provide the same utility as before. And ceteris paribus, if Alice doesn’t currently have thick 50% margins, then she will likely exit the market.
In contrast, some of the most profitable miners in China and Republic of Georgia are now operating — even with the large difficulty rise over the past 6 months — with 50+% margins. They may be squeezed, but they do not have to exit the market.
Basically, the less efficient players will be squeezed out and the more efficient players will remain. Who is likely be be more efficient? Larger farms in cheaper locations, or smaller pools made up of less sophisticated players with less capital?
But if the price of cryptocurrencies rise — in this case bitcoins — then won’t former miners come back into the market?
Maybe, but recall, we have seen this song and dance before and it is likely that the block reward halving is already factored into both the current market price and the hardware replacement cycle and as a result there probably will not be a doubling of the market price of bitcoins. However, that is a topic for a different post.
Other public blockchains
What do mining pool distributions look like for other cryptocurrencies?
Above is the distribution of mining pools for Litecoin over the past day. Interestingly, Coinotron — a pool I used when mining 3 years ago — currently represents 2.8% of the block making during that time frame. Two years ago, in May 2014, it represented about 50%.
In August 2015, Litecoin underwent its first block reward halving. Contrary to popular belief, its market price did not double. In fact, nine months later the price of a litecoin measured in USD is just fifty cents higher than what it was pre-halving.
Above is the distribution of mining pools for Ethereum over the past day.
Interestingly Ethereum formally launched in August 2015 and has seen the same consistent pattern of 3-4 pools representing the majority of block making activity as other cryptocurrencies have witnessed.
In fact, Dwarfpool, despite its name, has flirted with the 50% threshold several times, most notably in March. The Ethereum development team plans to transition the network from proof-of-work to proof-of-stake (Casper) later this year; it is unclear if the “staking” process will result in similar centralization.
Other cryptocurrencies continue to face similar pool centralization. This includes Namecoin which last year saw one pool, F2Pool provide more than 50% of the network hashrate for multiple months. While it does not appear that F2Pool behaved maliciously, the fact that one block maker could potentially rewrite history by doing block reorgs motivated Onename to migrate away from Namecoin.
It is surprising that with the 60%+ hashrate located in China that there is scant detail in English about how that ecosystem works. But there are reasons for this.
Recall that based on the current 25 BTC block reward, roughly $450 million in mining rewards has been divvied out over the past year to miners. On paper that would mean that China-based miners received more than $270 million in revenue, which cements this industry as one of two that continually see large annual revenue flows (the second being exchanges themselves).
I contacted a mining operator in China that currently operates about 40 petahashes per second in equipment. Note: miners use the abbreviated term ‘P’ and ‘PH’ to denote petahashes per second.
According to him:
“Our public hashing number is based on all our own hardware. This includes two facilities in western Sichuan plus a new Xinjiang site. All of these machines were originally S3’s from Bitmain but we have replaced them with S7’s. We want to build larger operations than what we have today, but our goal is to maintain a specific percentage of the entire network.”
“Remember our electric rates changes from season to season: different time of year and that hydro power has problems in the winter because of less melt water which results in an energy price that is twice as the rate in the summer.”
“The land is basically free because it is in the mountains and no one is interested in buying property there. So all it takes is construction materials and labor. We hired 10 people last year. We intentionally hired more than we needed so we can build a team and send them places. Our front end operation probably only needs 4-5 people and we pay them $1,000 a month which is actually very competitive for that region.”
“We know a Chinese guy, Mr. LY. He lives in Sichuan and was originally a hydroelectric operator but now owns his own hydro power station. He learned he could make more money mining than just running the station.”
“Why are people like us able to be competitive? In Yunnan, Guizhou and Sichuan there was an overinvestment in hydropower last decade and now there is a surplus of electricity. Dam operators couldn’t sell the electricity generated so that’s where Bitcoin miners moved to. Also, in Liaoning, some people can free electricity because of the proximity to oil fields – they are given cheap electricity to local residents as compensation for confiscated land/polluting the environment — it is subsidized electricity.”
“No one really pays taxes because miners don’t generate something considered valuable. That’s to say from the perspective of taxpayer, miners don’t generate something of value, because the government doesn’t really recognize bitcoin. Bitcoin mining isn’t illegal, we still pay a small amount of taxes but it’s like running a company that doesn’t make money. Instead a miner just pays a small amount of taxes and all the profit is invisible to the law as it stands today.”
I also reached out to another mining operator based in southern China who explained that in practice, mining farms that produce 1 PH or more are usually not based in cities:
“Most of the time they are not in cities, more like in the middle of nowhere and it would be inaccurate to name towns.”
Instead he listed provinces where they are spread out including: Heilongjiang,Liaoning, Hebei, Sichuan, Tianjin, Anhui, Jiangsu, Ghuizhou, Inner Mongolia, Shanxi, Guangdong. “Shenzhen for sure, there are testing facilities that are easily over 1P.”
What about ‘subprovincial’ locations?
“It is inaccurate to present information that way. A lot of the time, the sites are between borders because it’s in the middle of nowhere. And it normally spreads over lots of sites. One place has nearly 200 sites crossing two provinces; a lot of small ones representing about 100KW of power each. They are spread over several hundred kilometers; no economy of scale after a certain point.”
No service-level agreements
This type of self-doxxing, quasi-dynamic environment has led to another interesting phenomenon: ad hoc customer service via social media.
For example, two days ago, a user sent approximately 291.2409 bitcoins as a mining “fee.” A small pool called BitClub Network built the block that included this fee. This fee is equivalent to about $136,000.
The community as a whole then began a crowdsourced investigation into who may have sent this fee and the motivations for doing so, with many believing it to be a mistake. After all they reasoned, a typical “fee” that most mining pools require in order to be included in the next block is usually less than 25 cents on most days.
A user affiliated with BitClub has since publicly stated it would like to return the fee to the original entity that sent it, though it is unclear if he is speaking with any authority or if the whole thing was a ruse to begin with.
But, as I have argued before, this not only sets a bad precedent for miners as a whole due to a loss of revenue from the forthcoming ‘halvening,’ but the ability to contact a block maker sets a dangerous precedent for the core utility of the network: the disappearance of pseudonymous consensus.
Or as one redditor adroitly pointed out:
Or in other words, if block making was actually pseudoymous and decentralized, with 100+ unidentified pools creating blocks each day, it would be difficult if not impossible to locate and provide timely customer service to a user who made a mistake.
For instance, the most well-known block reorg occurred in March 2013 and it was only resolved when miners, including Slush and BTCGuild, contacted and coordinated with one another via IRC. If the network was more decentralized and pseudonymous, this coordination would have been very difficult to do, and this was by design.
I pointed out this irony on Twitter earlier this week as well: that there are trade-offs with this approach and the downside of using a bearer asset-based system that had no service level agreement, no EULA, no terms of service results in a world in which users who make mistakes have to complain on social media and hope someone is charitable.
And this happens on a regular basis: earlier this month a user accidentally sent 13.65 bitcoins to the BTCC pool and used reddit as his customer service forum.
That type of friction is not what most consumers want. It is a poor user experience which has gradually led to the creation of ‘trusted’ intermediaries in this ecosystem which as described in previous posts, recreates the existing financial system but without the same level of oversight and financial controls.
The cryptocurrency community is learning the hard way why intermediaries exist, why SLAs exist, why legal identities are required for financial transactions, why consumer protection laws arose and so forth. Pointing out these patterns is not malice or due to a lack of understanding of how cryptocurrencies work, but rather it serves as illustrations for why it has been hard to find real sustainable traction in the space.
How else is this visualized?
This past December an event was held in Hong Kong called “Scaling Bitcoin.”
One of the sessions involved a panel comprised of the world’s largest mining farm and pool operators.
The individuals in the photo above allegedly represent about 90% of the network hashrate.
Thus, for all the hype around “trust anchors” tied into public blockchains such as Bitcoin, claims of decentralization and “trust-lessness” are empirically untrue.
In practice, due to centralization and identity leakage, the cost to successfully reorganize a block isn’t through a Maginot Line attack (e.g., via hashrate), but through cheaper out-of-band attacks, such as hosting events in which self-doxxed miners participate. But that is also a topic for a different post.
16 months ago, Vitalik Buterin and others jokingly quipped that the trends towards centralization in Bitcoin mining (and other cryptocurrencies) resulted in a world where each coinbase transaction effectively arose from a multisig process.
To quote Buterin: “with Bitcoin, we’re paying $600 million a year on a 5-of-10 multisig.”
10 is roughly the amount of quasi-permanent block makers in a given day. And $600 million was the amount of revenue that miners received at that time due to the higher market value of bitcoin.
In theory, anyone can turn on their computer and hope to become a block maker on a public blockchain — no one has to register with a “Blockchain Admin” because there is no admin. However, in practice it requires a certain amount of technical knowledge and more importantly, capital, to profitably and sustainably operate a mining farm and pool.
And in order to scale this profitably, in practice, most miners at some point reveal their legal identities thereby negating the core characteristic of a public blockchain: pseudonymity. How? Miners, after having erected purpose-built facilities or to liquidate their holdings, may be required by external authorities to go through a gating / vetting process (such as KYC).
Ironically, a substantial increase in cryptocurrency prices may inevitably result in self-doxxing of all major farms. How? As market prices increase, miners in turn expend more capital to increase their own hashrate to chase the seigniorage rents.
Because of the KYC requirements of utilizing resources like electricity at a hydroelectric dam and the subsequent identity leakage, this turns the block making process itself into a mostly known, permissioned activity. Consequently, based on this past history, the term DMMS should probably be qualified with a “quasi” modifier in the front: QDMMS.
Similarly, while many enthusiasts have been led to believe a block reward halving will somehow re-decentralize the mining ecosystem, the fact of the matter is chip performance (as measured in hashrate efficiency) is only one factor in the total calculation that professional miners must account for.
Furthermore, semiconductor engineering itself is effectively on a known, mature trajectory and which appears to be lacking any significant leaps in technological improvement. The largest entities, such as Intel, see this relatively static path which is one of the reasons why they have formally abandoned their tick-tock roadmap and now plan to lay off 12,000 people.
In contrast, energy prices, land prices, labor costs and taxes are among other major components that professional mining operators look at as a whole and decide whether to stay in a market or not. Even if there is some price increase after the halvening, home mining by amateurs outside of China will likely continue to remain unprofitable after July.
Thus a year from now the mining ecosystem will probably look a lot like it does today, with most farms and pools being self-doxxed and relatively centralized.
[Special thanks to Antony Lewis for his constructive feedback]