Three perspectives to understand the disruptive innovation of AO

AO can be understood as an infinite Sharding, an infinitely scalable network. Each process is a Sharding.

Author: 0xmiddle

Review: Sandy

Source: Content Guild - Investment Research

AO is not a traditional blockchain. Its unconventional and counterintuitive design can easily confuse researchers who are just getting to know AO, especially when they try to frame AO within the architecture of traditional blockchains:

1. What kind of consensus mechanism is AO referring to as ‘holographic consensus’ that is neither PoS nor PoW?

2. Without a hash chain, or even a block, how does AO ensure data immutability?

3. Without a coordinating center, how can AO ensure the consistency of the global state?

4. Without redundant computing mechanisms, who will ensure the reliability of the computation? What if there is a calculation error?

5. Without shared security, how to ensure interoperability between Processes?

I will use 3 perspectives to help everyone transition from the known to the unknown, turning the unknown into the known, and understand AO at an emotional level using familiar concepts in blockchain.

Sharding Perspective

After the education of public chains such as Ethereum 2.0, Polkadot, Near, etc., everyone should be familiar with ‘Sharding’.

Concept of Sharding: In blockchain, Sharding is a scalability solution that splits the network into multiple Shards. Each Shard independently validates and processes transactions and generates its own blocks, thereby improving the overall network efficiency. Synchronous interoperability can be achieved within Sharding, while asynchronous interoperability is achieved between Shards through a certain communication protocol.

Polkadot is the most typical Sharding architecture. In Polkadot, each parachain is a Sharding, and each parachain independently collects and packages its own blockchain, and is randomly assigned a validator group by the relay chain for verification. The uniform XCM message format is used for communication between parachains to achieve interoperability.

AO’s Ultimate Sharding

From the perspective of Sharding, AO can be understood as an extreme ‘Sharding’: each Process is a Sharding. Imagine, what would it be like if each smart contract on Ethereum ran on a separate Sharding? That’s right, this is AO. Each Process is independent, and the calls between Processes rely on message-driven, completely asynchronous manner.

Modular Perspective

However, we found a key point that in the design of Polkadot, there is a ‘relay chain’, and in ETH2.0, there is also a ‘beacon chain’. Their role is to serve as a unified consensus layer and provide shared security. The unified consensus layer is responsible for directly or indirectly providing verification services for all Sharding and message transmission between Sharding. It seems that AO does not have this component, so how is the consensus layer of AO designed?

The consensus layer of AO is actually Arweave. From a modular perspective, AO can be understood as the L2 of Arweave, with Arweave as the L1 Rollup. All the logs of messages generated during the operation of the AO network will be uploaded to Arweave for permanent storage, which means there is an immutable record of AO network operation on Arweave. So you may ask, Arweave is a decentralized storage platform and does not have much computing power. How does Arweave verify the data uploaded by the AO network?

The answer is: Arweave does not verify, the AO network itself has an optimistic arbitration mechanism. Arweave accepts all message data uploaded by the AO network, and each message carries the process ID of its sender and the signature of the CU (computing unit) that runs it, as well as the signature of the SU (sorting unit) that sorts it. When disputes occur, more nodes can be introduced to recompute based on the immutable message records on Arweave, create the correct fork, discard the original incorrect fork, and penalize the faulty CU or SU in the correct fork. It is important to note that the MU is only responsible for collecting pending messages from the process and delivering them to the SU, and it is trustless, does not require a deposit, and does not involve penalties.

AO is very similar to Optimistic Rollup with Arweave as L1, except that the verification and challenge process does not occur on L1, but within the AO network itself.

However, there is still a problem here. It is not possible to wait for every message to be recorded on Arweave before confirming. In fact, the final deterministic formation time of Arweave takes more than half an hour. So AO will have its own soft consensus layer, just like Ethereum’s Rollups have their own soft consensus layer, and most transactions will not wait for L1 confirmation before being recorded.

The Process in AO actually self-determines the verification strength.

As the message recipient, the Process needs to decide whether to wait for Arweave confirmation before processing the message, or to process the message immediately after the soft consensus layer confirms. Even in the soft consensus layer confirmation process, the Process can adopt a flexible strategy, such as processing after a single CU confirms, or redundant confirmation by multiple CUs and cross-verification before processing, with the redundancy also determined by the Process.

In practical applications, the strength of verification is often related to the amount of the transaction, for example

• Small transactions are processed using a fast verification strategy and are processed immediately after single-point confirmation.
• For medium-sized transactions, different redundancy strategies are adopted for processing after multi-point confirmation based on the specific amount.
• For large transactions, a cautious verification strategy is adopted, and processing is done after confirmation on the Arweave network.

This is what AO calls the ‘Holistic Consensus’ + ‘Flexible Validation’ model, which decouples the ‘verifiability’ and ‘validation’ behaviors. AO takes a completely different approach to the consensus problem than traditional blockchains, where the responsibility for message validation lies not in the network itself, but in the recipient’s Process itself, or in other words, in the application developers.

It is precisely because such a consensus model is adopted that AO can adopt the ‘ultimate Sharding’ model without hubs and unlimited scalability.

Of course, elastic validation leads to different levels of validation strength for different processes, which may result in trust chain breaks in complex interoperability, and the failure or error of individual links in a long invocation chain will lead to the failure or error of the overall transaction. In fact, such issues have already been exposed during the AO testnet phase. I believe AO should set a minimum validation strength standard for all validation tasks, and we look forward to seeing what new designs AO will have in its upcoming mainnet.

Resource Perspective

In traditional blockchain systems, resources are abstracted as ‘block space’, which can be understood as a collection of storage, computing, and transmission resources provided by nodes, and is organically combined through on-chain blocks to provide a carrier for on-chain applications. Block space is a limited resource, and in traditional blockchains, different applications need to compete for block space and pay for it, while nodes profit from this payment.

There is no concept of blocks in AO, nor is there a concept of ‘block space’. However, like smart contracts on other chains, each Process on AO also needs to consume resources when running. It needs nodes to temporarily store transaction and state data, and nodes also need to consume computing resources to perform computing tasks for it. Its messages need to be transmitted by MU and SU to the target Process.

In AO, nodes are divided into three types: CU (computing unit), MU (message unit), and SU (sorting unit), where CU is the core that carries out computing tasks. MU and SU carry out communication tasks. When a process needs to interact with other processes, a message is generated and stored in the outbound queue. The CU that runs the process will sign the message, and the MU will extract the message from the outbound queue and submit it to the SU. The SU assigns a unique serial number to the message and uploads it to Arweave for permanent storage. Then, the MU delivers the message to the inbound queue of the target process, completing the message delivery. MU can be understood as a collector and delivery agent of messages, and SU can be understood as a sorter and uploader of messages.

As for storage resources, the MU in the AO network only needs to store the temporary data required for computation, which can be discarded after the computation is completed. Arweave is responsible for permanent storage. Although Arweave cannot scale horizontally, its storage performance ceiling is very high. The storage demand of the AO network is still far from reaching the ceiling of Arweave in the foreseeable future.

We found that the computing resources, transmission resources, and storage resources in the AO network are decoupled. In addition to the unified storage resources provided by Arweave, computing resources and transmission resources can be horizontally scaled without any limitations.

The more and higher performance CU nodes join the network, the higher the network’s computing power will be, and it can support more Process running; similarly, the more and higher performance MU, SU nodes join the network, the faster the network’s transmission efficiency will be. That is to say, the ‘block space’ in AO can be continuously created. For applications, you can purchase public CU, MU, and SU node services in the open market, or run private nodes to serve your own applications. If the application’s business expands, it can completely improve performance by expanding its own nodes, just as Web2 applications do. This is unimaginable on traditional blockchains.

On the pricing level of resources, AO can flexibly adjust through supply and demand, so that the supply of resources can expand or shrink according to demand. This adjustment is very sensitive, and the joining and exiting of nodes can be done very quickly. Looking back at Ethereum, when resource demand rises sharply, everyone has no choice but to endure high Gas fees, because Ethereum cannot increase its performance by expanding the number of nodes.

Summary

Above, we start with concepts that are well known to most cryptographers, such as ‘Sharding’, ‘modularization’, ‘Rollup’, ‘block space’, etc., and delve into the principles and mechanisms of AO, helping everyone understand how AO achieves almost unlimited scalability through disruptive innovation.

Now looking back at the beginning, do you understand those initial questions?

1. Not PoS, not PoW, what kind of consensus mechanism is the “holographic consensus” mentioned by AO?

AO’s consensus mechanism is actually a design similar to Op Rollup. At the hard consensus level, it relies on Arweave, and at the soft consensus level, each process can independently decide the verification strength and the number of redundant calculations performed by CU nodes.

2. Without a hash chain, or even blocks, how does AO ensure data immutability?

The DA data uploaded to Arweave is immutable, providing verifiability for all computations and transmissions on AO. AO itself does not need to limit the processing capacity within a unit of time, so there is no need to set blocks. Arweave chain has the structures of ‘hash chain’ and ‘block’ that ensure the immutability of data.

**3. Without a coordinating center, how does AO ensure the consistency of the global state?

Each Process is an independent ‘Sharding’, independently managing its transactions and state, and Processes interact through message-driven manner. Therefore, global state consistency is not required. Arweave’s permanent storage provides global verifiability and historical traceability, combined with optimistic challenge mechanism, and can be used for dispute resolution.

**4. No redundant computing mechanism, who ensures the reliability of the computation? What if the computation goes wrong?

AO does not have a globally enforced redundant computing mechanism, and each Process can independently decide how to verify the reliability of each message received. If a computation error occurs, it can be discovered and corrected through optimistic challenge.

**5. No shared security, how to ensure the interoperability between Processes?

The Process needs to manage the credit of each interacting Process on its own, and different levels of verification strength can be used for Processes with different security levels. For complex interoperability of the call chain, in order to avoid the high cost of error correction caused by trust chain breakage, AO may have a minimum requirement for verification strength.