What’s the big deal about Decentralized Consensus?
From ancient warfare to blockchain fanfare
There is currently an incredible amount of speculation and discussion around blockchain technology, accompanied by exaggerated hype cycles and intermittent media frenzies. It is challenging to cut through the noise, jargon, scams, and over-marketing to discern the underlying advances with true technological value. This article is a non-technical introduction to essential concepts, terminology, and modern history — written to help anybody to understand the key principles and broader impact, without taking a detour through dense mathematics and cryptography.
To understand where blockchain technology fits into the bigger picture, it’s necessary to first consider the implications of humanity’s fundamentally new ability to create decentralized networks with strong universal consensus. The term “decentralized consensus” refers to a set of principles and techniques that allow participants on a distributed network to arrive at perfect agreement on a shared document or database. Systems built upon decentralized consensus methods are inherently tamper-proof, censorship-resistant, and permissionless.
The cryptographic and game theoretic principles underlying decentralized consensus unlock trust as an emergent property.
Centralized consensus has always been easy! It is trivial to share information among participants who trust each other (e.g. co-workers sharing information on an office whiteboard) or for participants to choose to trust some central third party (e.g. a cloud storage provider or banking system).
The defining feature of systems built upon decentralized consensus is the ability for many parties to safely store and share information, without having to rely on a central authority or trust any other participants in the network. Thanks to recent breakthroughs, described below, total strangers can now come together to form global networks with self-securing databases. Remarkably, even in applications where individual participants would benefit from fraudulently modifying the shared data (e.g. a financial ledger) the cryptographic and game theoretic principles underlying decentralized consensus unlock trust as an emergent property.
Any effective decentralized consensus system must solve a fundamental challenge: how can a system arrive at universal agreement under adversarial conditions where messages may be unknowingly lost and participants may behave dishonestly for their own gain? The problem was concisely expressed as the Byzantine Generals’ Problem in 1982:
Imagine a group of generals of the Byzantine army camped with troops around an enemy city. Communicating only by messenger, the generals must agree upon a common battle plan — whether to attack or retreat. Either way, they must arrive at agreement and act in unison, since an attack with only a portion of the troops would be disastrous. However, one or more of the generals may be traitors who will try and confuse the others. The problem is to find an algorithm to ensure that the loyal generals will reach agreement.
An individual Byzantine general may selfishly communicate an attack, then retreat themselves — ensuring their own safety at the expense of the overall campaign. Likewise, individual participants in other decentralized systems (e.g. a shared inventory list or financial ledger) may selfishly try to selectively communicate dishonest information to advance their own position, at the expense of network integrity.
For decades, the world’s greatest minds have been working on creating tools to enable self-securing decentralized systems. The entire landscape changed when the pseudonymous Satoshi Nakamoto brilliantly completed the decentralized consensus puzzle and publicly shared a game-changing solution in the short and elegant Bitcoin white paper.
Centralized consensus has always been easy! The paradigm-shattering consequence of effective decentralized consensus is that many parties can trust shared document even when the network participants do not know or trust each other.
Nakamoto consensus combined three key principles:
- “Peer-to-peer” network architecture
- The cryptographically-secured “blockchain” data structure
- “Proof of work” requirements to add new entries to the ledger
The peer-to-peer network allows anybody in the world to participate, which creates an incredibly robust global network that has no central points for failure or censorship. The Bitcoin network adds new transactions onto a tamper-proof ledger housed in a cryptographically-secured data structure, retroactively named a blockchain. Bitcoin’s game-changing contribution was the combination of this blockchain data structure with a proof of work consensus mechanism, which involves an energy-intensive “mining” task that makes it expensive to propose new blocks of data. This incentive model strongly penalizes malicious actors, since only honest miners whose contributions are accepted by the network receive reimbursement for the energy costs that went into producing the proof of work.
Decentralized consensus technology has changed dramatically in the decade since Bitcoin’s debut. The proof of work methods currently employed by many decentralized consensus networks have benefits such as a field-tested security record, and downsides such as high energy use and financial barriers for mining. However, various mechanisms such as ASIC-resistant proof of work, proof of stake, proof of spacetime, and even hybrid methods are currently in development and operation.
Early decentralized consensus networks distributed a public ledger that all participants could read, with no privacy protections for users. Newer projects have implemented a variety of security approaches and features, ranging from encryption of sensitive data to novel applications of zero-knowledge poofs. These protocols utilize clever cryptographic methods to allow all parties to establish decentralized consensus on the shared ledger, without posting all data in plaintext that the public can interpret and analyze.
The blockchain data structure is simply the first iteration of technical methods that enable humanity to achieve decentralized consensus, and fundamental new breakthroughs are arriving at an incredible pace. The global community is continuously improving the performance and expanding the functions of initial blockchain implementations through innovations such as second-layer scaling solutions, and sharding. Furthermore, decentralized consensus tools are being developed using alternatives to traditional blockchains, such as complex directed acyclic graphs (DAGs) and even constant-size recursive blockchains.
It is not surprising that the foundational tools of modern decentralized consensus methods were introduced in the context of a currency use case. Global economic turmoil exacerbated pain points related to general lack of financial sovereignty, and maintaining a digital ledger is an intuitive application of the new ability to create consensus on a shared document.
The blockchain data structure is simply the first iteration of technical methods that enable humanity to achieve decentralized consensus.
In the last decade, decentralized consensus technology has advanced to enable a range of uses beyond storage and transfer of value. We now have the ability to create smart contracts that interweave value with code, allowing multiple parties to craft autonomous self-enforcing agreements that are executed by open-source, public, blockchain-based distributed computing platforms, such as Ethereum. Smart contracts provide the infrastructure backbone to build decentralized application platforms (DApps) that have the potential to replace many of our third-party legacy institutions with decentralized peer-to-peer networks.
Decentralized consensus systems are being used to revolutionize supply chain management, run global computations, create digital assets, share resources, create digital IDs, enable data sovereignty, build financial infrastructure for the Internet of Things, manage property titles, vote securely, and interchange assets through a decentralized exchanges and atomic swaps.
We are now crossing horizons that society could not even envision a decade ago. Breakthroughs in decentralized consensus theory, practice, and applications are driving rapid technological innovation and platform development. We are now designing and refining the tools that will allow all 7 billion people on earth to interact and transact without borders.
To help with the creation of this new global infrastructure, we launched the Insight Decentralized Consensus Fellows Program, a professional fellowship to help software engineers and scientists transition to a career creating leading-edge technologies in the blockchain space. This is a free, full-time, 7 week fellowship based in San Francisco. Insight Fellows in this program work to build cutting-edge blockchain projects atop existing networks protocols or develop new applications under the mentorship of some of the top teams in the consensus space. We believe in accelerating cutting-edge technology by supporting and advancing the individuals that are building it.
To learn more, and apply visit: https://insightfellows.com/decentralized-consensus
