Processing DDEX messages using EIP4844 BLOB transactions and ZK proofs

[cezary-stroczynski]

Overview of the idea:

Block scheme with the flow:
https://miro.com/app/board/uXjVKthq2gc=/

There are three types of participants in the network:

1. Data Providers

   • use Data Provider SDK to send packed DDEX messages as a BLOB using type 3 EIP4844 transaction
   • need to stake OWN tokens to be able to send messages

2. Validators

   • need to be approved by OWN Protocol
   • run Validator Node on their servers/clouds
   • pin BLOB to IPFS
   • Validator Node runs ZK circuit that checks if all messages in the BLOB are compatible with the schema:
     • all messages are compatible:
       1. extract key data (like ISRC, Release ID etc.) and create ZK proof of performed computation
       2. send proof and extracted data to the DDEX MESSAGE SEQUENCER contract
       3. DDEX MESSAGE SEQUENCER validates the proof and emit extracted data as events
       4. events will be picked up by subgraph and indexed creating REGISTRY
       5. reward Validator with OWN Tokens
     • at least one message is incompatible with schema:
       1. create ZK proof of incompatible BLOB
       2. send proof to the DDEX MESSAGE SEQUENCER contract
       3. stake of Data Provider who submitted incompatible BLOB is slashed
       4. reward Validators with OWN Tokens

3. OWN Protocol Administrators

   • is OWN Token minter
   • approve Validators

TBD:

• what happens to the slashed stake? Is it distributed between all validators or assigned to the one that provided the proof?
• how is the validator that places the proof selected? is it a race or is it based on some random factor?
• what gives the OWN Token it's value?
• how the validator's incentivise mechanism works? Is OWN Token minted or transfered to the Validator?

[revelator-labs]

Re TBD points:

• I believe that that slashed tokens (from Data or Payment Providers) will be returned to the protocol reserve for continuous budgeting of the fraud governance operations.
  But with regards to validator slashes, I think the tokens should probably be distributed between the honest/available validators.

• @cezary-stroczynski can you elaborate if there is significant variance with the time it will take to verify and validate the schema using blobs or if all validators run the same software on their local machine or are there operations they can do to make it faster? If we can technically incentivize faster verification times then we could base it as a "race" but I don't want it to be hardware based.
  If it's all about just making sure your machine is online and up-to-date (like the Beacon chain) then we should use some random factor... Perhaps even listening to the Beacon chain RanDAO value for the random seed generator.

• additionaly on that note - it sounds to me like we have a combination of POW (running the blob-based algorithm, calculating and submitting proofs...) and POS with staking.
  Perhaps we can penalize a small amount for any validator that is not online, and slash only for malicious behavior or (proven) fraud or lack of payment; On top of that we can also "lock" a stake against disputes or conflicting claims...

• $OWN is required to be spent for creating new OW records (i.e. excepting a new DDEX message), for creating licenses, tokenizing them and making payments. See more here

• I will defer to [email protected] to comment but I believe the first batch will be issued and rewarded to early adopters and then the protocol will continue issuing new tokens with the Network's growth according to the plan I. The Tokenomics doc (unless altered by the governing committee and the token holders who vote)

[revelator-labs]

@yehoshzl - We should also define the reward amount and how it is split between the validator submitting the proof, and the rest of the validator network attesting to the validity of the proof.

[yehoshzl]

Great questions @cezary-stroczynski:

• Slashed tokens from fraud proofs go to the validator who finds it (first come first served). If you think this is complex because of some the HW or ZK considerations, we should discuss. @revelator-labs you don't need to divide the reward among validators because (unless I'm very much mistake) this is a one way proof, like prooving a hash? In which case there is only one 'correct' answer and there's no need to verify it. Is this correct? @cezary-stroczynski ?
• As @revelator-labs mentioned, $OWN is needed for almost all protocol activities: paying RTs, being a payment provider, data provider, validator, creating a RT, governance, etc. It's the aligned unit of currency for the protocol that aligns everyones incentives towards creating a bigger music pie.
• Validator/data providers rewards mechanism: V1 (until tips are added) -> Rewards are paid by token incentives into a rewards pool. Every epoch (7 days, but we can decide something longer), rewards can be claimed from the pool by data providers based on the amount of data they've attested to on chain. Validators operate like fraud nodes in optimistic L2s - they receive rewards for finding the frauds.

[cezary-stroczynski]

I’m writing this post to share the design decisions and the reasoning behind the features we’ve implemented in the Protocol. As we work towards refining the system, we faced several challenges, particularly in ensuring the integrity of data passed between Data Providers, Validators, and the Risc0 guest code. In this post, I’ll walk through the solutions we considered, the trade-offs involved, and how we plan to address potential risks as we continue to improve the protocol.

Input Integrity in ZK Proofs

The protocol aim to be a decentralized platform that creates a secure and queryable registry of DDEX messages. The process involves Data Providers uploading blobs to the blockchain, along with cryptographic proofs of their validity. Validators download these blobs, process them using verifiable computation (Risc0 guest code), and generate receipts that are validated on-chain. To ensure the integrity of this process, we must prove that both Data Providers and Validators are referencing the same blob without introducing inefficiencies or vulnerabilities.

The problem

A key challenge we faced was tying a blob’s entry in the DdexSequencer smart contract with the receipt generated by Risc0 guest code while guaranteeing that both refer to the same data. Here are the solutions we explored in detail:

Solution 1: KZG Proof Verification in Risc0

In this solution, the Risc0 guest code input includes three elements: blob, KZG commitment, and a KZG proof. Inside the guest code, the verify_blob_kzg_proof function is run to verify that the proof matches the blob and its commitment. If successful, the guest code returns the KZG commitment in its receipt, ensuring the blob and commitment are tied together.
A key enabler of this solution is Solidity’s blobhash OPCODE. Calling this OPCODE will return a hash of the KZG commitment prefixed with its version. It’s available in the Solidity code for each EIP4844 transaction (type 3) sent to a smart contract.

Workflow:

1. Data Provider uploads a blob to the DdexSequencer smart contract, along with its KZG commitment and proof. The smart contract uses the blobhash OPCODE to create a new entry in the DdexSequencer.
2. Validator downloads the blob, KZG commitment, and proof, then passes these inputs into the Risc0 guest code. Inside the guest code, verify_blob_kzg_proof function is run in order to validate that the KZG commitment refers to the currently processed blob. If validation is successful then KZG commitment is included in the receipt.
3. The Validator submits the receipt to the DdexSequencer. The contract extracts the KZG commitment from the receipt and uses it as a key to associate the Validator’s output with the uploaded blob.

This solution provides strong cryptographic guarantees but has a significant drawback: verifying KZG proofs within the Risc0 guest code is computationally expensive, leading to very long processing times for each blob.

Solution 2: Using SHA-2 for Cross-Verification

In this approach, the Data Provider calculates both the KZG commitment and a SHA-2 hash of the blob. These are sent to the DdexSequencer and used to create an entry.

Workflow:

1. The Data Provider uploads the blob, KZG commitment, proof, and SHA-2 hash of the blob to the DdexSequencer.
2. The Validator downloads these values, passing them to the Risc0 guest code.
3. Inside the guest code, a SHA-2 hash is recalculated from the blob and compared with the input hash. If they match, the hash and commitment are included in the receipt.
4. The Validator submits the receipt to the DdexSequencer, where the extracted values are cross-checked against the on-chain entry.

Potential Risk: Malicious Data Provider

A Data Provider may attempt to act maliciously by passing a SHA-2 hash that does not correspond to the blob uploaded in the transaction.
Result:
The Validator downloads the blob and calculates the correct SHA-2 hash. If the hash does not match the one provided, the Validator cannot generate a valid receipt, rendering the blob unprocessable.
Mitigation:
If the Validator proves that the Data Provider submitted an incorrect SHA-2 hash, the Data Provider’s stake is slashed.

Potential Risk: Malicious Validator

A Validator may try to submit dishonest input to the guest code. For example, the Validator might use a blob stored locally on their device. In this case, the SHA-2 hash calculated inside the guest code will differ from the one stored in the DdexSequencer.
Result:
The Validator cannot execute the transaction in the DdexSequencer because the SHA-2 hash extracted from the receipt will not match the on-chain hash.

Potential Risk: Collusion Between Data Provider and Validator

A Data Provider may calculate a SHA-2 hash for Blob A and send it to the DdexSequencer alongside Blob B. If a Validator collaborates with the Data Provider, they could use Blob A as input to the Risc0 guest code, along with the fraudulent SHA-2 hash. This would allow Blob A, which was never uploaded to the blockchain, to be processed successfully.
Mitigation:
Initially, both Validators and Data Providers will be trusted parties that must be whitelisted to participate in the network, minimizing the incentive to act maliciously.
In the future, before the protocol is opened to a wider audience, we plan to implement the third solution, which addresses this risk comprehensively.

Third Solution: Proof of Equivalence

Inspired by Vitalik Buterin's post on equivalence between polynomial commitment schemes, this solution avoids verifying KZG proofs inside the guest code. Instead, the guest code only evaluates the blob's polynomial at a specific point Z (derived from the hashed input) and returns the evaluation result Y, along with the KZG proof and commitment that was calculated outside the guest code.

Workflow:

1. Data Provider uploads a new blob
2. Validator computes Z and Y inside the guest code and includes these in the receipt.
3. The receipt is submitted to the DdexSequencer, where an on-chain precompile verifies the KZG proof using Z, Y, and the KZG commitment and KZG proof

Benefits: This approach reduces computational overhead in the guest code while maintaining robust validation.
Status: This solution is our long-term goal but requires further research and development before implementation.

[revelator-labs]

The first version published, until further notice will be run by white-listed nodes and need to assume $OWN tokens are staked and ignore the issuance, as the $OWN token will only be deployed at a later stage