What is the difference between Hyperledger and Hyperledger?

hyperledgerI am frequently asked this question because there is some confusion related to the legacy name and the current branding of certain technology. The two are distinct. And how we got there involves a little history.

Hyper, the parent company of Hyperledger, was founded by Dan O’Prey and Daniel Feichtinger in the spring of 2014. Fun fact: one of the alternative names they considered using was “Mintette.com” — after the term coined by Ben Laurie in his 2011 paper.

The simplest way to describe Hyperledger, the technology platform from Hyper, during its formative year in 2014 was: Ripple without the XRP. Consensus was achieved via PBFT.1 There were no blocks, transactions were individually validated one by one.

Hyperledger, the technology platform from Hyper, was one of the first platforms that was pitched as, what is now termed a permissioned distributed ledger: validators could be white listed and black listed. It was designed to be first and foremost a scalable ledger and looked to integrate projects like Codius, as a means of enabling contract execution.

Most importantly, Hyperledger in 2014 was not based off of the Bitcoin codebase.

Note: in the fall of 2014 Richard Brown and I both became the first two advisors to Hyper, the parent company of Hyperledger.  Our formal relationship ended with its acquisition by DAH.2

In June 2015, DAH acquired Hyper (the parent company of Hyperledger) which included the kit and caboodle: the name brand, IP and team (the two Dans).  During the same news release, it was announced that DAH had acquired Bits of Proof, a Hungary-based Bitcoin startup that had designed a Java-based reimplementation of Bitcoin (which previously had been acquired by CoinTerra).3

It was proposed at that time that Hyperledger, the Hyper product, would become the permissioned ledger project from DAH.  It’s product landing page (courtesy of the Internet Archive) uses roughly the same terminology as the team had previously pitched it (see also the October homepage older homepage for DAH as well).

digital asset homepage october 2015

Source: Digital Asset / Internet Archive

On November 9, 2015, on a public blog post DAH announced that it was “Retiring Hyperledger Beta, Re-Open Sourcing Soon, and Other Changes.”

The two most notable changes were:

(1) development would change from the languages of Erlang and Elixir to Java and Scala;

(2) switch to the UTXO transaction model

The team noted on its blog in the same post:

We are also switching from our simplistic notion of accounts and balances to adopt to de facto standard of the Bitcoin UTXO model, lightly modified. While Hyperledger does not use Bitcoin in any way, the Bitcoin system is still extremely large and innovative, with hundreds of millions of dollars invested. By adopting the Bitcoin transaction model as standard, users of Hyperledger will benefit from innovation in Bitcoin and vice versa, as well as making Hyperledger more interoperable.

During this same time frame, IBM was working on a project called OpenChain, which for trademark reasons was later renamed (now internally referred to as OpenBlockchain).4

IBM’s first public foray into distributed ledgers involved Ethereum vis-a-vis the ADEPT project with Samsung (first announced in January 2015). Over the subsequent months, IBM continued designing its own blockchain (see its current white paper here).

In December 2015, the Linux Foundation publicly announced it was creating a new forum for discussion and development of blockchain technology.  Multiple names were proposed for the project including Open Ledger (which was the name originally used in the first press release). However, in the end, the name “Hyperledger” was used.

How did that occur?

DAH, one of the founding members of the project, donated two things to the Linux Foundation: (1) the brand name “Hyperledger” and (2) the codebase from Bits of Proof.

Recall that Bits of Proof was the name of a Bitcoin startup that was acquired by DAH in the fall of 2014 (the Chief Ledger Architect at DAH was the co-founder of Bits of Proof). 5 Architecturally, Bits of Proof is a Java-implementation of Bitcoin. 6

In other words: today the term “Hyperledger” represents an entirely different architectural design and codebase than the original Hyperledger built by Hyper.7

The major architectural switch occurred in November 2015, which as noted above involved adopting the UTXO transaction set and Java language that Bits of Proof was built with.  Therefore, Hyperledger circa 2016 is not the same thing as Hyperledger circa 2014.

Over the past two months there have been multiple different codebases donated to the Linux Foundation all of which is collectively called “Hyperledger” including the IBM codebase (partly inspired by Ethereum) as well as the DAH and Blockstream codebase (one is a clone of Bitcoin and the other is a set of extensions to Bitcoin). The technical discussions surrounding this can be found on both the public Linux Foundation mailing list and its Slack channel.

How do different, incompatible codebases work as one?

This technical question is being discussed in the Linux Foundation. It bears mentioning that as of now, the codebases are incompatible largely due to the fact that Bitcoin uses the UTXO transaction set and OpenBlockchain uses an “accounts” based method for handling balances.  There are other reasons for incompatibility as well, including that they are written in completely different languages: Java/Scala versus Go versus C++ (Blockstream).

How extensive is the reuse of the Bits of Proof Bitcoin codebase donated to the Linux Foundation from the DAH team?  According to a quick scan of their GitHub repo:

So when someone asks “what is Hyperledger technology?” the short answer is: it is currently the name of a collective set of different codebases managed by the Linux Foundation and is not related to the original distributed ledger product called Hyperledger created by Hyper. The only tenuous connection is the name.

Timeline in brief: Hyperledger was originally created in Spring 2014 by Hyper; Hyper was acquired in June 2015 by DAH; the original Hyperledger architecture was entirely replaced with Bits of Proof in November 2015; the Hyperledger brand name and Bits of Proof code was donated to the Linux Foundation in December 2015.

  1. Interestingly enough, the current OpenBlockchain project from IBM also uses PBFT for its consensus mechanism and uses an “accounts” based method; two characteristics that the original Hyperledger platform from Hyper had too. []
  2. For more info on the original Hyperledger, see the Innotribe pitch; the description in Consensus-as-a-service from April 2015 and the Epicenter Bitcoin interview. []
  3. Following the bankruptcy of CoinTerra, the Bits of Proof team became independent once again. []
  4. CoinPrism launched a project called OpenChain, before IBM did. []
  5. Sometimes there is a confusion between Bits of Proof and Bits of Gold.  Bits of Proof was the independent Java-implementation of Bitcoin (which is not the same thing as bitcoinj).  Bits of Gold is an Israeli-based Bitcoin exchange.  A co-founder of Bits of Gold also works at DAH and is their current CTO. []
  6. In the future it may contain some modifications including Elements from Blockstream. []
  7. What was once the original Hyperledger GitHub repo has been handed over to the Linux Foundation but some of the original code base and documentation from the 2014 project can still be viewed elsewhere. []
Send to Kindle

Short interview with the CFA Institute

[Note: I neither own nor have any trading position on any cryptocurrency. The views expressed below are solely my own and do not necessarily represent the views of my employer or any organization I advise.]

Below are several questions I recently received from the CFA Institute along with my responses.

Q1.  In your book you make a convincing case that Bitcoin has a number of significant structural design flaws that will likely prevent it from ever develop into something of economically meaningful scale. Could you briefly outline the main reasons for your view?

A1.  The two fundamental challenges that do not appear surmountable in the short-run are:

(1)  An endogenous money-like informational commodity (such as bitcoin or litecoin) that lacks purchasing power stability relative to goods and services which live external to the system. This is a characteristic that is common to contemporary cryptocurrencies that are divorced from external information: how to securely provide information of the exogenous outside world back into the internal network in a trust-minimized manner?  There have been multiple proposals over the past 2 years but no production systems in large part because solving this is solving a public goods problem, so where does the funding come from to R&D it?

(2)  The second main challenge is sustainable decentralized security. Empirically all proof-of-work based cryptocurrencies have trended towards some form of centralization.  Looking at CoinGecko, all of the top PoW cryptocurrencies are currently dominated by a handful of pools.  The reason why has to do with the inhomogeneous Poisson process used by these systems which creates variance in payouts.  And as we see in the world of traditional finance, one way to reduce risks is to pool capital.  Thus, with the origination of the first mining pools in late 2010, we see miners – the security force – acting rationally by pooling hashrate to smooth out the variance in payouts.

Ernie Teo and Dave Hudson are just a handful of researchers who have looked into the long-term implications this has and have shown via simulations that as block rewards decline over time, the labor force declines as fewer participants can profitably compete in the mining process.  Thus there is an open question as to whether or not any PoW cryptocurrency can remain robustly decentralized and secure or if they just “self-destruct.”  Note: that there are over 100 dead altcoins, so empirically these networks are not automatically self-healing or anti-fragile.

Solving both of these issues – if they are indeed solvable – so far has remained in the realm of posturing on social media: very little real research and statistical modelling has taken place which is very surprising considering many companies have raised funds with the assumption (and promise) that these two issues will be solved.

I remain skeptical that the first is solvable without compromising the integrity of the network: how do you rebase the purchasing power of an endogenous unit of account without needing to trust the external data source?  Vitalik Buterin, Robert Sams and a few others have proposed solutions dubbed “stablecoins” but most of the community, especially early adopters of popular cryptocurrencies are against purchasing power stability, preferring volatility with the belief that external market forces will somehow coordinate and permanently smooth it out, usually in a trajectory towards the moon.

Similarly I have yet to see any modelling that shows how POW mining becomes more decentralized over time.  There have been companies that claim and market that they will “redecentralize” with embedded ASICs, but when you drill down deeper it is merely decentralizing hashing, not block making (the key part).

Q2.  There seems to be a new consensus developing in fintech circles and among incumbents of ‘Bitcoin bad, blockchain good’. Do you agree with this or is it too simplistic – can you truly have one without the other?

A2.  I think it is too simplistic and a little unfair to Bitcoin.  Satoshi, from his written accounts, did not appear interested in developing software for financial institutions.  He had a problem-set in his mind: how to build a censorship resistant payments system without introducing some kind of trusted third party to prevent double spending.  In 2007, when he began the project (or so he stated on a mailing list) if he had thought about how to build a distributed ledger for regulated financial institutions, the deliverable would look different than Bitcoin does.  We only have the benefit of hindsight to make that “Blockchain good, Bitcoin bad” claim today.

Why?  Because quite frankly, Bitcoin itself does not really solve anything for banks.

Banks have seen probably 100-200 proof-of-concept/pilot projects over the last 18 months and have rejected nearly all of them.  Not because it involved a cryptocurrency but because the tech didn’t solve their actual problems.  I have yet to be in a meeting where someone says “I hate bitcoin because it is bitcoin” — perhaps some banks do, but all of the people I interact with at banks want solutions to their problems and cryptocurrencies in their current form, were not designed to solve problems banks have.  So why should they use them?

For instance, if I built some typewriters and then claimed that banks weren’t buying them because they’re anti-typewriter.  It’s not because they are anti-typewriter it is because they don’t have a use for typewriters in 2016.  Yet the useful parts of typewriters are of course the keyboard which can be repurposed and used with laptops.  Similarly, the useful bits of cryptocurrencies are the cryptographic signing and shared data structure elements.

Q3.  Incumbent organisations experimenting with blockchain technology seem to be mostly designing permissioned blockchains. Could you elaborate on how these differ from, for example, the Bitcoin blockchain, and some of its advantages and disadvantages?

A3.  Since September 2015, R3 has been pitched by over 100 software companies ranging from pre-seed startups to large enterprises.  Among them are about 30 different distributed ledger proposals.  Some are very much half-baked altcoins.  A large number are highly modified derivatives of existing platforms (e.g., Bitcoin, Ethereum, Ripple) and a few others were customized and built from the ground up or with elements of existing systems.  Universally they all involve some kind of permissioning: in which the validators on the network are gated and vetted and the users of the network are KYC’ed.

Why are they building these?  There are a number of different motives but by and large this has to do with the operating environment their customers exist in: trusted, known relationships.  Those relationships, market structures and laws, much to the chagrin of cypherpunk prophecies, are not going to disappear.  So if you are building a commercial business and want to actually generate revenue and not permanently live off of venture funding, you will need to deliver products customers want and not just work on public goods problems.

Another advantage of designing these types of permissioned systems is that the validation model – the creation of contracts and service level agreements around who or what validates transactions – typically removes the probabilistic settlement issues found in public blockchains like Bitcoin.  Public blockchains cannot provide legal settlement finality of exogenous financial instruments.  And introducing new risks into the financial system via probabilistic finality is absurd.  Regulated financial institutions cannot and do not want to be in a position in which assets on their balance sheet only have a 95% possibility that they own them or that a block reorganization from a pool in a sanctioned country mines it.

Incidentally there are now Bitcoin mining companies that are pitching themselves as “trusted miners” – which is an oxymoron.  In fact, if the validation process (mining) of public blockchains becomes fully trusted, gated and permissioned then users lose the benefit of censorship resistance while they simultaneously have to pay the large operating costs that proof-of-work requires.  Or in other words, a permissioned-on-permissionless system that provides more kabuki theater than it does commercial utility.

Q4.  Increasingly, financial institutions are trying to figure out whether they can benefit from integrating blockchain technology into their operations, including your organisation R3CEV. What do you see as the main barriers of integrating blockchain into existing financial services?

A4.  There are multiple challenges each financial institution has and technology alone probably only solves a fraction of them.  For instance, what are the problems a blockchain actually solves for an organization?  Maybe there are only a handful if any.  What are the switching costs?  What are the total costs of operation?   How does it plug into their existing legacy systems?

Most startups lack the subject matter expertise or the relationships into the financial services industry to be able to answer those questions, so they end up building tech for tech sake.  Science fair projects that remain underutilized and even unused.  No amount of marketing can ultimately salvage a platform that does not solve a problem that customers do not have.

Send to Kindle