Introduction

opML (Optimistic Machine Learning), invented and developed by ORA, introduces a groundbreaking approach to integrating machine learning with blockchain technology. By leveraging similar principle of optimistic rollups, opML ensures the validity of computations in a decentralized manner. This framework enhances transparency and fosters trust in machine learning inference by allowing for onchain verification of AI computation.. ^[1]

Architecture

OpML is comprised from the following key components:

1. Fraud Proof Virtual Machine (Off-chain VM): A robust off-chain engine responsible for executing machine learning inference. This component executes machine learning inference, generating new VM states as outputs. When discrepancies occur, manifested as different VM states, the MIPS VM employs a bisection method to pinpoint the exact step, or instruction, where the divergence begins.

2. opML Smart Contracts (On-chain VM) : Utilized for the verification of computational results, ensuring the accuracy of the off-chain computation. These contracts allow the execution of a single MIPS instruction, enabling the on-chain environment to verify specific steps in the computation process. This capability is vital for resolving disputes and ensuring the integrity of the off-chain computation.

3. Fraud Proofs: In the event of a dispute, fraud proofs generated by the verifier serve as conclusive evidence, illustrating the discrepancy in computation and facilitating the resolution process through the opML smart contracts.

Verification Game

Verification game is the process where two or more parties are assumed to execute the same program. Then, the parties can challenge each other with a pinpoint style to locate the disputable step. This step is sent to the smart contract for the verification.
For the system to work as intended it's important to ensure:

• Deterministic ML execution
  opML ensures consistent ML execution by using fixed-point arithmetic and software-based floating-points, eliminating randomness and achieving deterministic outcomes with a state transition function.
• Separate Execution from Proving
  opML utilizes a dual-compilation method: one for optimized native execution and another for fraud-proof VM instructions for secure verification. This ensures both fast execution and reliable, machine-independent proof.
• Efficiency of AI model inference in VM
  The existing fraud proof systems that are widely adopted in the optimistic rollup systems need to cross-compile the whole computation into fraud proof VM instructions, which will result in inefficient execution and huge memory consumption. opML proposes a novel multi-phase protocol, which allows semi-native execution and lazy loading, which greatly speeds up the fraud proof process.

The whole opML process includes the following steps:

1. The requester first initiates an ML service task.

2. The server then finishes the ML service task and commits results on chain.

3. The verifier will validate the results. Suppose there exists a verifier who declares the results are wrong. It starts a verification game with verification game (bisection protocol) with the server and tries to disprove the claim by pinpointing one concrete erroneous step.

4. Finally, arbitration about a single step will be conducted on smart contract.

Multi-phase Verification Game

Represents an extension of single-phase verification game, which allows for a better utilization of computing resources.
Single phase verification game cross-compiles the whole ML inference code into the Fraud Proof VM instructions. This method is less efficient than the native execution (doesn't utilize the full potential of GPU/TPU acceleration and parallel processing). The Fraud Proof VM also has limited memory, which prevents loading of large models into the memory directly.
To address the issues above, multi-phase verification game introduces the following properties:

• Semi-Native Execution With the multi-phase design, we only need to conduct the computation in the VM only in the final phase, resembling the single-phase protocol. For other phases, we have the flexibility to perform computations that lead to state transitions in the native environment, leveraging the capabilities of parallel processing in CPU, GPU, or even TPU. By reducing the reliance on the VM, we significantly minimize overhead, resulting in a remarkable enhancement in the execution performance of opML, almost akin to that of the native environment
• Lazy Loading Design To optimize the memory usage and performance of the fraud proof VM, we implement a lazy loading technique. This means that we do not load all the data into the VM memory at once, but only the keys that identify each data item. When the VM needs to access a specific data item, it uses the key to fetch it from the external source and load it into the memory. Once the data item is no longer needed, it is swapped out of the memory to free up space for other data items. This way, we can handle large amounts of data without exceeding the memory capacity or compromising the efficiency of the VM.

Further readings

• Detailed explanation of opML can be found in our research paper. ^[1]
• Check out ORA's open-source implementation repository. ^[2]