From 51be8d35b71408b66b5f25817fa7f6349234f792 Mon Sep 17 00:00:00 2001 From: Jack Grigg Date: Fri, 26 Apr 2024 17:33:48 +0100 Subject: [PATCH 01/11] ZIP 231: Empty template --- zips/zip-0231.md | 56 +++++++++++++++++++++++++++++++++++++++++++++++ zips/zip-0231.rst | 12 ---------- 2 files changed, 56 insertions(+), 12 deletions(-) create mode 100644 zips/zip-0231.md delete mode 100644 zips/zip-0231.rst diff --git a/zips/zip-0231.md b/zips/zip-0231.md new file mode 100644 index 000000000..c11433a87 --- /dev/null +++ b/zips/zip-0231.md @@ -0,0 +1,56 @@ +> ZIP: 231 +> Title: TBD +> Owners: Jack Grigg +> Kris Nuttycombe +> Daira-Emma Hopwood +> Arya Solhi +> Credits: Sean Bowe +> Nate Wilcox +> Status: Draft +> Category: Consensus / Wallet +> Created: 2024-04-26 +> License: MIT + + +# Terminology + +{Edit this to reflect the key words that are actually used.} +The key words "MUST", "REQUIRED", "MUST NOT", "SHOULD", and "MAY" in this +document are to be interpreted as described in BCP 14 [#BCP14]_ when, and +only when, they appear in all capitals. + + +# Abstract + +TBD + + +# Motivation + +TBD + + +# Requirements + +TBD + + +# Non-requirements + +TBD + + +# Specification + +TBD + + +# Reference implementation + +TBD + + +# References + +[#BCP14]: Information on BCP 14 — "RFC 2119: Key words for use in RFCs to Indicate Requirement Levels" and "RFC 8174: Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words" +[#protocol]: Zcash Protocol Specification, Version 2021.2.16 or later diff --git a/zips/zip-0231.rst b/zips/zip-0231.rst deleted file mode 100644 index 1bbed3699..000000000 --- a/zips/zip-0231.rst +++ /dev/null @@ -1,12 +0,0 @@ -:: - - ZIP: 231 - Title: Decouple Memos from Transaction Outputs - Owners: Jack Grigg - Kris Nuttycombe - Daira-Emma Hopwood - Credits: Sean Bowe - Nate Wilcox - Status: Reserved - Category: Consensus / Wallet - Discussions-To: From b43433d5ac7983248efc5101e11d55b36c37da6a Mon Sep 17 00:00:00 2001 From: Jack Grigg Date: Fri, 26 Apr 2024 18:05:26 +0000 Subject: [PATCH 02/11] ZIP 231: First pass at motivation, requirements, and visible structure --- zips/zip-0231.md | 116 +++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 112 insertions(+), 4 deletions(-) diff --git a/zips/zip-0231.md b/zips/zip-0231.md index c11433a87..ffa0d7c78 100644 --- a/zips/zip-0231.md +++ b/zips/zip-0231.md @@ -27,23 +27,131 @@ TBD # Motivation -TBD +In Zcash transaction versions v2-v5, each shielded output contains a ciphertext +comprised of a 52-byte note plaintext, and a corresponding 512-byte memo field. +Recipients can only decrypt the outputs sent to them, and thus can also only +observe the memo fields included with the outputs they can decrypt. + +The shielded transaction protocol hides the sender(s) (that is, the addresses +corresponding to the keys used to spend the input notes) from all of the +recipients. For certain kinds of transactions, it is desirable to make one or +more sender addresses available to one or more recipients (for example, a reply +address), and in subsets of those transactions it is desirable to authenticate +the sender addresses (to give the recipient a guarantee that the address is +controlled by a sender of the transaction). These Authenticated Reply Addresses +require zero-knowledge proofs, and for the Orchard protocol these proofs are too +large to fit into a 512-byte memo field. + +Something about light client truncation (simplifies the security argument). + +In the light client protocol, a recipient need not download full transaction +information if the compact transaction information indicates that they do not +receive any memo in the transaction. # Requirements -TBD +- Recipients can receive memo data that is greater than 512 bytes in length. +- Multiple recipients, across any of the shielded pools, can be given the + capability to view the same memo data. +- The exact number and exact lengths of distinct decryptable memos should not + be revealed, even to the transaction recipients, although an upper bound on + the total length of memo data that the observer does not have the capability + to view will be leaked. +- A recipient can determine whether or not they have been given the capability + to view any memo solely by decrypting the note ciphertext. +- Memo chunks within a transaction can be individually pruned without preventing + the transaction from being verified. + - As a consequence, a recipient with the capability to view a memo can prune + all of that memo's chunks. # Non-requirements -TBD +- Recipients do not need to be able to receive multiple memos per note. # Specification -TBD +## Memo encryption + +Sketch: + +In shielded bundles, the 512-byte memo field is replaced by `[flag_byte] || memo_key`. + +``` +STREAM: AEAD(key, counter, pt) + +[ + (key_a, 0), + (key_b, 0), + (key_a, 1), + (key_c, 0), + (key_c, 1), + (key_a, 2), +] +``` + +## Memo bundle format + +```rust +enum MemoChunk { + Encrypted(AEAD(MemoChunk, memo_key)), + Pruned { + memo_chunk_digest: Digest, + } +} + +struct MemoBundle { + chunks: Vec, +} +``` + +## Encoding in transactions + +TODO: Decide whether / how to enable memo data to be pruned from a transaction while still enabling it to be broadcast. + +```rust +enum MemoBundle { + Chunks(Vec), + Pruned { + memo_bundle_digest: Digest, + } +} +``` + +## Transaction sighash + +``` +memo_chunk_digest = H(AEAD(MemoChunk, memo_key)) +memo_bundle_digest = H(concat(memo_chunk_digests)) +``` + +The memo bundle digest structure is a performance optimisation for the case +where all memo chunks in a transaction have been pruned. + + +# Rationale + +## Pruned encoding + +The separation of memo data from note data, and the new ability to easily store +variable-length memo data, opens up an attack vector against node operators for +storing arbitrary data. The transaction digest commitments to the memo bundle +are structured such that if a node operator is presented with a memo key (i.e. +they are given the capability to decrypt a particular memo), they can identify +and prune the corresponding memo chunks, while still enabling the transaction to +be validated as part of its corresponding block and broadcast over the network. + +Note that broadcasting a partially-pruned transaction means that the pruned +chunks no longer contribute to the upper bound on memo data. +The prunable structure does not introduce a censorship axis; memo bundles do not +reveal which memo chunks correspond to which memos, and therefore a network +adversary cannot selectively censor individual memos. They can censor any/all +chunks within specific transactions, however shielded transactions do not reveal +their senders, recipients, or amounts, and thus also cannot be individually +targeted for censorship. # Reference implementation From 2402afa66e553489517bcfb99694f3c55cf3f171 Mon Sep 17 00:00:00 2001 From: Jack Grigg Date: Thu, 31 Oct 2024 11:40:58 +0000 Subject: [PATCH 03/11] ZIP 231: Set title, fix reference syntax --- zips/zip-0231.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/zips/zip-0231.md b/zips/zip-0231.md index ffa0d7c78..f3d6ecbea 100644 --- a/zips/zip-0231.md +++ b/zips/zip-0231.md @@ -1,5 +1,5 @@ > ZIP: 231 -> Title: TBD +> Title: Memo Bundles > Owners: Jack Grigg > Kris Nuttycombe > Daira-Emma Hopwood @@ -160,5 +160,5 @@ TBD # References -[#BCP14]: Information on BCP 14 — "RFC 2119: Key words for use in RFCs to Indicate Requirement Levels" and "RFC 8174: Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words" -[#protocol]: Zcash Protocol Specification, Version 2021.2.16 or later +[^BCP14]: [Information on BCP 14 — "RFC 2119: Key words for use in RFCs to Indicate Requirement Levels" and "RFC 8174: Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words"](https://www.rfc-editor.org/info/bcp14) +[^protocol]: Zcash Protocol Specification, Version 2024.5.1 [NU6] or later](protocol/protocol.pdf) From 100d0cf66ec79a910e167b0905a2dbe6a717f284 Mon Sep 17 00:00:00 2001 From: Jack Grigg Date: Thu, 31 Oct 2024 11:43:36 +0000 Subject: [PATCH 04/11] ZIP 231: Specify memo bundle structure, encryption, decryption --- zips/zip-0231.md | 87 +++++++++++++++++++++++++++++++++++------------- 1 file changed, 64 insertions(+), 23 deletions(-) diff --git a/zips/zip-0231.md b/zips/zip-0231.md index f3d6ecbea..30b77b148 100644 --- a/zips/zip-0231.md +++ b/zips/zip-0231.md @@ -73,39 +73,79 @@ receive any memo in the transaction. # Specification +## Memo bundle + +A memo bundle consists of a sequence of 256-byte memo chunks, each individually +encrypted. These memo chunks represent zero or more encrypted memos. + +Each transaction may contain a single memo bundle, and a memo bundle may contain +at most 40 memo chunks. This limits the total amount of memo data that can be +conveyed within a single transaction to 10 KiB. + ## Memo encryption -Sketch: +During transaction construction, each memo is assigned a 32-byte `memo_key`. +These keys SHOULD be generated randomly, and MUST NOT be used to encrypt more +than one memo within a single transaction. In shielded bundles, the 512-byte memo field is replaced by `[flag_byte] || memo_key`. -``` -STREAM: AEAD(key, counter, pt) +TODO: Define `flag_byte` format. Have one bit for "there is no memo". -[ - (key_a, 0), - (key_b, 0), - (key_a, 1), - (key_c, 0), - (key_c, 1), - (key_a, 2), -] -``` +The transaction builder generates a 32-byte nonce from a CSPRNG. A new nonce MUST be +generated for each memo bundle. -## Memo bundle format +The symmetric encryption key for a memo is derived from its `memo_key` as follows: -```rust -enum MemoChunk { - Encrypted(AEAD(MemoChunk, memo_key)), - Pruned { - memo_chunk_digest: Digest, - } -} + encryption_key = PRF^{expand}_{memo_key}([0xE0] || nonce) + +Each memo is padded to a multiple of 256 bytes with zeroes, and split into +256-byte chunks. Each memo chunk is encrypted with ChaCha20Poly1305 as follows: + + IETF_AEAD_CHACHA20_POLY1305(encryption_key, nonce = I2BEOSP_{88}(counter) || [final_chunk], memo_chunk) + +This is a variant of the STREAM construction [#stream]_. +- `counter` is a big-endian chunk counter starting at zero and incrementing by + one for each subsequent chunk within a particular memo. +- `final_chunk` is the byte `0x01` for the final memo chunk, and `0x00` for all + preceding chunks. + +Finally, the encrypted memo chunks for all memos are combined into a single +sequence using an order-preserving shuffle. Memo chunks from different memos MAY +be interleaved in any order, but memo chunks from the same memo MUST have the +same relative order. The following diagram shows an example shuffle of three +memos: -struct MemoBundle { - chunks: Vec, -} ``` +[ + (memo_a, 0), + (memo_b, 0), + (memo_a, 1), + (memo_c, 0), + (memo_c, 1), + (memo_a, 2), +] +``` + +## Memo decryption + +When a recipient decrypts a shielded output, they obtain a `memo_key`. From this +they derive `encryption_key` as above, and then proceed as follows: + +- Set `counter = 0` and `final_chunk = 0x00`. +- Attempt to decrypt each memo chunk in order. +- Each time decryption succeeds for a memo chunk, increment `counter` by 1, and + then continue attempting to decrypt subsequent chunks. +- Once all memo chunks have been trial-decrypted once, set `final_chunk = 0x01` + and then attempt to decrypt the memo chunks again, starting immediately after + the last successfully-decrypted chunk (or at the start if none were), and + without incrementing `counter`. + - This step can be made secret-independent by attempting to decrypt every memo + chunk again, and ignoring the results of all chunks up to and including the + last successfully-decrypted chunk. +- If no memo chunk decrypts successfully with `final_chunk = 0x01`, discard any + memo chunks that were decrypted, and return nothing. Otherwise, concatenate + the decrypted memo chunks in order and return the concatenation as the memo. ## Encoding in transactions @@ -162,3 +202,4 @@ TBD [^BCP14]: [Information on BCP 14 — "RFC 2119: Key words for use in RFCs to Indicate Requirement Levels" and "RFC 8174: Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words"](https://www.rfc-editor.org/info/bcp14) [^protocol]: Zcash Protocol Specification, Version 2024.5.1 [NU6] or later](protocol/protocol.pdf) +[^stream]: [Online Authenticated-Encryption and its Nonce-Reuse Misuse-Resistance](https://eprint.iacr.org/2015/189) From bfde4e23077e854ce755cdf9cc9118dfbdc86da3 Mon Sep 17 00:00:00 2001 From: Jack Grigg Date: Thu, 31 Oct 2024 12:00:01 +0000 Subject: [PATCH 05/11] ZIP 231: Add rationale for memo bundle structure and encryption Co-authored-by: Daira-Emma Hopwood Co-authored-by: Kris Nuttycombe --- zips/zip-0231.md | 82 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 82 insertions(+) diff --git a/zips/zip-0231.md b/zips/zip-0231.md index 30b77b148..f9713290a 100644 --- a/zips/zip-0231.md +++ b/zips/zip-0231.md @@ -173,6 +173,87 @@ where all memo chunks in a transaction have been pruned. # Rationale +## Memo bundle size restriction + +Restricting the total amount of memo data in a bundle to 10 KiB limits the rate +at which the chain size can grow cheaply (from a computational perspective; memo +bundles are much easier to produce than proofs or signatures). + +The current behaviour for previous transaction versions (no limit on the number +of memos) is not altered by this ZIP, because memos in those transactions are +tied to individual shielded outputs (incurring their computational cost), and +are not natively aggregable. + +## Memo chunk size + +With 10KiB limit on amount of memo data as the constant in this table, the +maximum number of unique memos you can create, and the cost in bytes of that +memo data plus auth when using a 32-byte memo key, is: + +| | Memo size | +| Chunk size | ≤ 256 bytes | 512 bytes | +|------------|----------------------|----------------------| +| Pre-231 | 20 @ 10240 ( 0.00%) | 20 @ 10240 ( 0.00%) | +| 512 | 20 @ 11220 (+ 9.57%) | 20 @ 11220 (+ 9.57%) | +| 256 | 40 @ 12200 (+19.14%) | 20 @ 11540 (+12.70%) | +| 256 20-out | 20 @ 6100 (-40.43%) | | + +In the "256 20-out" case you have a distinguisher compared to old transactions, +in that you can tell the transaction is sending at most 256 bytes per recipient +rather than 512 if it is sending the max number of memos. But that's inherently +baked into the decision to use a smaller memo chunk size (and it is still +possible for the chunks to all be a single memo sent to all outputs, or anything +in between). + +## Memo key size + +If we used a 16-byte memo key instead of 32 bytes, the transaction size overhead +becomes: + +| | Memo size | +| Chunk size | ≤ 256 bytes | 512 bytes | +|------------|----------------------|----------------------| +| Pre-231 | 20 @ 10240 ( 0.00%) | 20 @ 10240 ( 0.00%) | +| 512 | 20 @ 10900 (+ 6.45%) | 20 @ 10900 (+ 6.45%) | +| 256 | 40 @ 11560 (+12.89%) | 20 @ 11220 (+ 9.57%) | +| 256 20-out | 20 @ 5780 (-43.55%) | | + +The decrease in overhead is relatively modest in most cases, but more noticeable +for small memos with a 256-byte memo chunk. + +However, 16-byte memo keys don't meet Zcash's target security level of 125 bits. +The classical argument [#ref]_ is that you can build a machine of size 2^32 +("The variant key-search machine") that has a 2^-71 chance of finding a 128-bit +key (with sufficient targets, e.g. 2^36 *blocks*). In this scenario, you only +get 32+71 = 103 bits of security classically. + +The benefits of 32-byte keys are: +- They incur a small transaction size overhead above the minimum key size that + _would_ meet that target security level. +- It is simpler than reasoning about unusual byte lengths for keys. +- This key length matches what we already use elsewhere for symmetric keys. + +See also [#protocol-inbandrationale]_. + +## Encryption format + +Including a per-transaction `nonce` in the derivation of `encryption_key` gives +protection against accidental (or intentional) `memo_key` reuse across multiple +transactions. We do not protect against `memo_key` reuse within a transaction; +it is up to the transaction builder to ensure that the same `memo_key` is not +used to encrypt two different memos (and if they did so, normal clients would +either never observe the second memo, or would decrypt parts of each memo and +get a non-sensical and potentially insecure "spliced" memo). + +We do not include commitments to the shielded outputs in the derivation of +`encryption_key` for two reasons: +- It would force the transaction builder to fully define all shielded outputs + before encrypting the memos, which might prevent potential use cases of PCZTs. +- We don't want to unnecessarily prevent the ability to create a transaction + with a memo bundle and no shielded outputs, as there may be use cases for, + e.g. a fully-transparent transaction with encrypted memo, or a ZSA issuance + transaction with exposed memo data using a well-known `memo_key`. + ## Pruned encoding The separation of memo data from note data, and the new ability to easily store @@ -202,4 +283,5 @@ TBD [^BCP14]: [Information on BCP 14 — "RFC 2119: Key words for use in RFCs to Indicate Requirement Levels" and "RFC 8174: Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words"](https://www.rfc-editor.org/info/bcp14) [^protocol]: Zcash Protocol Specification, Version 2024.5.1 [NU6] or later](protocol/protocol.pdf) +[^protocol-inbandrationale]: [Zcash Protocol Specification, Version 2024.5.1 [NU6]. Section 8.7: In-band secret distribution](protocol/protocol.pdf#inbandrationale) [^stream]: [Online Authenticated-Encryption and its Nonce-Reuse Misuse-Resistance](https://eprint.iacr.org/2015/189) From b5c3ff5d178b389510106c97b9482a5bb55ed96e Mon Sep 17 00:00:00 2001 From: Jack Grigg Date: Thu, 31 Oct 2024 12:06:53 +0000 Subject: [PATCH 06/11] ZIP 231: Specify transaction fee for memo bundles Co-authored-by: Daira-Emma Hopwood Co-authored-by: Kris Nuttycombe --- zips/zip-0231.md | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/zips/zip-0231.md b/zips/zip-0231.md index f9713290a..aafc35caa 100644 --- a/zips/zip-0231.md +++ b/zips/zip-0231.md @@ -170,6 +170,14 @@ memo_bundle_digest = H(concat(memo_chunk_digests)) The memo bundle digest structure is a performance optimisation for the case where all memo chunks in a transaction have been pruned. +## Transaction fees + +(This section will become a modification to ZIP 317.) + +A memo bundle may contain two free chunks if there are any shielded outputs in +the transaction. Otherwise, each memo chunk requires `marginal_fee` as defined +in ZIP 317 [#zip-0317]_. + # Rationale @@ -274,6 +282,24 @@ chunks within specific transactions, however shielded transactions do not reveal their senders, recipients, or amounts, and thus also cannot be individually targeted for censorship. +## Transaction fees + +Making the fee linear in the number of chunks has the following properties: + +- The required fee to add more memo chunks scales at the same rate as adding + logical actions, so it isn't a cheaper mechanism for an adversary to bloat + chain size. +- A "baseline transaction" (one spent note, one output to an external recipient + with a memo, one change output without a memo) has the same fee as before. +- A "broadcast transaction" (many outputs to different recipients all given the + same memo) is the same fee as before (but a smaller transaction). +- A "many memos transaction" (many outputs to different recipients all with + unique memos) is at most around twice the fee as before. + +Combined with the memo bundle size restriction, the maximum additional fee for +a memo bundle over prior transactions is 0.0019 ZEC. + + # Reference implementation TBD @@ -285,3 +311,4 @@ TBD [^protocol]: Zcash Protocol Specification, Version 2024.5.1 [NU6] or later](protocol/protocol.pdf) [^protocol-inbandrationale]: [Zcash Protocol Specification, Version 2024.5.1 [NU6]. Section 8.7: In-band secret distribution](protocol/protocol.pdf#inbandrationale) [^stream]: [Online Authenticated-Encryption and its Nonce-Reuse Misuse-Resistance](https://eprint.iacr.org/2015/189) +[^zip-0317]: [ZIP 317: Proportional Transfer Fee Mechanism](zip-0317.rst) From 8a3eccb2cfe97dd8d0f24615e2538d0a7a118c97 Mon Sep 17 00:00:00 2001 From: Jack Grigg Date: Thu, 31 Oct 2024 12:39:44 +0000 Subject: [PATCH 07/11] ZIP 231: Specify transaction encoding and memo chunk pruning --- zips/zip-0231.md | 31 +++++++++++++++++++++---------- 1 file changed, 21 insertions(+), 10 deletions(-) diff --git a/zips/zip-0231.md b/zips/zip-0231.md index aafc35caa..5430df3b3 100644 --- a/zips/zip-0231.md +++ b/zips/zip-0231.md @@ -82,6 +82,9 @@ Each transaction may contain a single memo bundle, and a memo bundle may contain at most 40 memo chunks. This limits the total amount of memo data that can be conveyed within a single transaction to 10 KiB. +Memo bundles are encoded in transactions in a prunable manner: each memo chunk +can be replaced by its representative digest. + ## Memo encryption During transaction construction, each memo is assigned a 32-byte `memo_key`. @@ -133,7 +136,7 @@ When a recipient decrypts a shielded output, they obtain a `memo_key`. From this they derive `encryption_key` as above, and then proceed as follows: - Set `counter = 0` and `final_chunk = 0x00`. -- Attempt to decrypt each memo chunk in order. +- Attempt to decrypt each memo chunk in order. Pruned memo chunks are skipped. - Each time decryption succeeds for a memo chunk, increment `counter` by 1, and then continue attempting to decrypt subsequent chunks. - Once all memo chunks have been trial-decrypted once, set `final_chunk = 0x01` @@ -147,18 +150,21 @@ they derive `encryption_key` as above, and then proceed as follows: memo chunks that were decrypted, and return nothing. Otherwise, concatenate the decrypted memo chunks in order and return the concatenation as the memo. +If any chunk of the memo encrypted to `memo_key` has been pruned, the decryption +process above returns nothing (as `final_chunk` will be set to `0x01` with the +wrong counter value), ensuring that a malformed memo is not returned. + ## Encoding in transactions -TODO: Decide whether / how to enable memo data to be pruned from a transaction while still enabling it to be broadcast. +| Bytes | Name | Data Type | Description | +|--------|---------------|--------------------------|---------------------------------------- | +| 32 | `nonce` | `byte[32]` | The nonce for deriving encryption keys. | +| 8 | `pruned` | `uint64` | Bitflags indicating the type of each entry in `vMemoChunks`. Bits are little endian. 0 corresponds to `byte[272]` and is an encrypted memo chunk. 1 corresponds to `byte[32]` and is the `memo_chunk_digest` for a pruned chunk. | +| varies | `nMemoChunks` | `compactSize` | Length of `vMemoChunks`. | +| varies | `vMemoChunks` | `MemoChunk[nMemoChunks]` | A sequence of encrypted memo chunks. | -```rust -enum MemoBundle { - Chunks(Vec), - Pruned { - memo_bundle_digest: Digest, - } -} -``` +TODO: Decide whether to permit an optimised encoding for a fully-pruned memo +bundle that just contains `memo_bundle_digest`. ## Transaction sighash @@ -272,6 +278,11 @@ they are given the capability to decrypt a particular memo), they can identify and prune the corresponding memo chunks, while still enabling the transaction to be validated as part of its corresponding block and broadcast over the network. +The transaction encoding permits pruning at the individual chunk level in order +to faciliate pruning an individual memo from a transaction without affecting the +other memos. This enables node operators to be responsive to, for example, GDPR +deletion requests. + Note that broadcasting a partially-pruned transaction means that the pruned chunks no longer contribute to the upper bound on memo data. From b9e5367c7cd2a5539f1e38b8e8577a5ca3c74337 Mon Sep 17 00:00:00 2001 From: Jack Grigg Date: Thu, 31 Oct 2024 12:44:49 +0000 Subject: [PATCH 08/11] ZIP 231: Add some TODOs --- zips/zip-0231.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/zips/zip-0231.md b/zips/zip-0231.md index 5430df3b3..3efd7366f 100644 --- a/zips/zip-0231.md +++ b/zips/zip-0231.md @@ -176,6 +176,9 @@ memo_bundle_digest = H(concat(memo_chunk_digests)) The memo bundle digest structure is a performance optimisation for the case where all memo chunks in a transaction have been pruned. +TODO: finish this to be a modification to the equivalent of ZIP 244 for +transaction v6. + ## Transaction fees (This section will become a modification to ZIP 317.) @@ -184,6 +187,10 @@ A memo bundle may contain two free chunks if there are any shielded outputs in the transaction. Otherwise, each memo chunk requires `marginal_fee` as defined in ZIP 317 [#zip-0317]_. +## Consensus rules + +TODO: specify. + # Rationale From ff885d3cf18b8ddfb67bfa7c04558c5e62c56331 Mon Sep 17 00:00:00 2001 From: Kris Nuttycombe Date: Tue, 5 Nov 2024 10:50:06 -0700 Subject: [PATCH 09/11] ZIP 231: Specify the v6 note plaintext format & remaining spec changes. Co-authored-by: Daira-Emma Hopwood --- zips/zip-0231.md | 316 +++++++++++++++++++++++++++++++++++------------ 1 file changed, 240 insertions(+), 76 deletions(-) diff --git a/zips/zip-0231.md b/zips/zip-0231.md index 3efd7366f..33eae1361 100644 --- a/zips/zip-0231.md +++ b/zips/zip-0231.md @@ -14,40 +14,61 @@ # Terminology -{Edit this to reflect the key words that are actually used.} -The key words "MUST", "REQUIRED", "MUST NOT", "SHOULD", and "MAY" in this -document are to be interpreted as described in BCP 14 [#BCP14]_ when, and -only when, they appear in all capitals. +The key words "MUST", "MUST NOT", "SHOULD", and "MAY" in this document are to +be interpreted as described in BCP 14 [^BCP14] when, and only when, they appear +in all capitals. + +The character § is used when referring to sections of the Zcash Protocol +Specification. [^protocol] # Abstract -TBD +Currently, the memo sent in a shielded output is limited to at most 512 bytes. +This ZIP proposes to allow larger memos, and to enable memo data to be shared +between multiple recipients of a transaction. # Motivation -In Zcash transaction versions v2-v5, each shielded output contains a ciphertext -comprised of a 52-byte note plaintext, and a corresponding 512-byte memo field. -Recipients can only decrypt the outputs sent to them, and thus can also only -observe the memo fields included with the outputs they can decrypt. +In Zcash transaction versions v2 to v5 inclusive, each Sapling or Orchard +shielded output contains a ciphertext comprised of a 52-byte note plaintext, +and a corresponding 512-byte memo field. [^protocol-noteptconcept] Recipients +can only decrypt the outputs sent to them, and thus can also only observe the +memo fields included with the outputs they can decrypt. The shielded transaction protocol hides the sender(s) (that is, the addresses corresponding to the keys used to spend the input notes) from all of the recipients. For certain kinds of transactions, it is desirable to make one or more sender addresses available to one or more recipients (for example, a reply -address), and in subsets of those transactions it is desirable to authenticate -the sender addresses (to give the recipient a guarantee that the address is -controlled by a sender of the transaction). These Authenticated Reply Addresses -require zero-knowledge proofs, and for the Orchard protocol these proofs are too -large to fit into a 512-byte memo field. - -Something about light client truncation (simplifies the security argument). +address). In such circumstances it is important to authenticate the sender +addresses, to give the recipient a guarantee that the address is controlled by +a sender of the transaction; failure to authenticate this address can enable +phishing attacks. These Authenticated Reply Addresses require zero-knowledge +proofs, and for the Orchard protocol these proofs are too large to fit into a +512-byte memo field. + +It is also desirable, for clients with more stringent bandwidth constraints, +to be able to transmit encrypted notes to the client without including the +encrypted memo data. In the current light client protocol [^zip-0307], this +is done by truncating the note ciphertext to just the part that encrypts the +memo. However, that has the effect of truncating the authentication tag, and +so the resulting decryption algorithm does not meet standard security notions +for an authenticated encryption scheme. It is a goal of this proposal to +rectify this, simplifying the security argument. In the light client protocol, a recipient need not download full transaction information if the compact transaction information indicates that they do not receive any memo in the transaction. +At present, it is not possible to transmit the same memo data to multiple +transaction recipients without redundantly encoding that data, and sending +memo data greater than 512 bytes requires sending multiple outputs; the +problem is compounded when attempting to send more than 512 bytes to each +recipient. By separating memo data from the decryption capability for those +memos, it admits a greater variety of applications that utilize memo data, +while decreasing the amount of data that needs to be stored on-chain overall. + # Requirements @@ -60,58 +81,176 @@ receive any memo in the transaction. to view will be leaked. - A recipient can determine whether or not they have been given the capability to view any memo solely by decrypting the note ciphertext. -- Memo chunks within a transaction can be individually pruned without preventing - the transaction from being verified. - - As a consequence, a recipient with the capability to view a memo can prune - all of that memo's chunks. +- Memo chunks within a transaction can be individually pruned from block storage + without preventing the transaction from being verified when transmitting block + data to peers. +- The ciphertext of the note alone can be decrypted using a scheme that meets + a standard security notion for authenticated encryption, without more than a + small constant overhead in ciphertext size. # Non-requirements -- Recipients do not need to be able to receive multiple memos per note. +- Recipients do not need to be able to receive multiple memos per note. This + capability can however be enabled under the existing proposal by "chaining" + memos, including the decryption key for another memo within the memo that + is decryptabble by a recipient. # Specification +Since this proposal is defined only for v6 and later transactions, it is not +necessary to consider Sprout JoinSplit outputs. The following sections apply +to both Sapling and Orchard outputs. + +## Changes to the Zcash Protocol Specification + +The following changes are to be made affecting note plaintexts, note ciphertexts, +and the algorithms for encryption and decryption. + +In § 3.2.1 ‘Note Plaintexts and Memo Fields’: + +* Change + + > Each Sapling or Orchard note plaintext (denoted $\mathbf{np}$) consists of + > + > $\hspace{2em}(\mathsf{leadByte} ⦂ \mathbb{B^{Y}}, \mathsf{d} ⦂ \mathbb{B^{[\ell_{\mathsf{d}}]}}, \mathsf{rseed} ⦂ \mathbb{B^{Y[32]}}, \mathsf{memo} ⦂ \mathbb{B^{Y[512]}})$ + + to + + > The form of a Sapling or Orchard note plaintext depends on the version of + > the transaction in which it will be included; specifically whether that + > version is pre-v6, or v6-onward. + > + > Each pre-v6 Sapling or Orchard note plaintext (denoted $\mathbf{np}$) consists of + > + > $\hspace{2em}(\mathsf{leadByte} ⦂ \mathbb{B^{Y}}, \mathsf{d} ⦂ \mathbb{B^{[\ell_{\mathsf{d}}]}}, \mathsf{rseed} ⦂ \mathbb{B^{Y[32]}}, \mathsf{memo} ⦂ \mathbb{B^{Y[512]}})$ + > + > Each v6-onward Sapling or Orchard note plaintext (denoted $\mathbf{np}$) consists of + > + > $\hspace{2em}(\mathsf{leadByte} ⦂ \mathbb{B^{Y}}, \mathsf{d} ⦂ \mathbb{B^{[\ell_{\mathsf{d}}]}}, \mathsf{rseed} ⦂ \mathbb{B^{Y[32]}}, \mathsf{K^{memo}} ⦂ \mathbb{B^{Y[32]}})$ + +In § 5.5 ‘Encodings of Note Plaintexts and Memo Fields’ [^protocol-noteptencoding]: + +* Change the paragraph that describes "The encoding of a Sapling or Orchard note plaintext" + to refer to "The encoding of a pre-v6 Sapling or Orchard note plaintext". + +* Add a new paragraph at the end of the section: + + > The encoding of a v6-onward Sapling or Orchard note plaintext consists of: + > + > |---------------------------|---------------------|--------------------------|-----------------------------| + > | 8-bit $\mathsf{leadByte}$ | 88-bit $\mathsf{d}$ | 256-bit $\mathsf{rseed}$ | 32-byte $\mathsf{K^{memo}}$ | + > + > * A byte 0x03, indicating this version of the encoding of a v6-onward + > Sapling or Orchard note plaintext. + > * 11 bytes specifying $\mathsf{d}$. + > * 8 bytes specifying $\mathsf{v}$. + > * 32 bytes specifying $\mathsf{rseed}$. + > * 32 bytes specifying $\mathsf{K^{memo}}$. + > + > A value consisting of 32 $\mathtt{0xFF}$ bytes for $\mathsf{K^{memo}}$ is used + > to indicate that there is no memo for this note plaintext. + +In § 4.7.2 ‘Sending Notes (Sapling)’ [^protocol-saplingsend] and +§ 4.7.3 ‘Sending Notes (Orchard)’ [^protocol-orchardsend]: + +* Add a reference to this ZIP specifying the construction of the memo bundle and + derivation of $\mathsf{K^{memo}}$ in the case of a v6-onward note plaintext. + +* Change + + > Let $\mathbf{np} = (\mathsf{leadByte}, \mathsf{d}, \mathsf{v}, \mathsf{rseed}, \mathsf{memo})\!$. + + to + + > Let $\mathbf{np}$ be the encoding of a Sapling note plaintext using $\mathsf{leadByte}$, $\mathsf{d}$, + > $\mathsf{v}$, $\mathsf{rseed}$, and either $\mathsf{memo}$ for a pre-v6 note plaintext or + > $\mathsf{K^{memo}}$ for a v6-onward note plaintext. + + replacing "Sapling" with Orchard in the case of § 4.7.3. + +In § 4.20.1 ‘Encryption (Sapling and Orchard)’ [^protocol-saplingandorchardinband]: + +* Change + + > Let $\mathbf{np} = (\mathsf{leadByte}, \mathsf{d}, \mathsf{v}, \mathsf{rseed}, \mathsf{memo})$ + > be the Sapling or Orchard note plaintext. $\mathbf{np}$ is encoded as defined + > in § 5.5 ‘Encodings of Note Plaintexts and Memo Fields’. + + to + + > Let $\mathbf{np}$ be the encoding of the Sapling or Orchard note plaintext (which may be + > pre-v6 or v6-onward), as defined in § 5.5 ‘Encodings of Note Plaintexts and Memo Fields’. + +* Add another normative note to that section: + + > * $\mathsf{C^{enc}}$ will be of length either 580 or 100 bytes, depending on whether + > $\mathbf{np}$ is a pre-v6 or v6-onward note plaintext. + +In § 4.20.2 ‘Decryption using an Incoming Viewing Key (Sapling and Orchard)’ and +§ 4.20.3 ‘Decryption using a full Incoming Viewing Key (Sapling and Orchard)’: + +* Replace $\mathsf{memo} ⦂ \mathbb{B^{Y[512]}}$ with $\mathsf{memoOrKey}$. +* Specify that the type of $\mathsf{memoOrKey}$ is $\mathbb{B^{Y[512]}}$ when + decrypting a pre-v6 note ciphertext, or $\mathbb{B^{Y[32]}}$ when decrypting a + v6-onward note ciphertext. In the latter case, it is used as $\mathsf{K^{memo}}$ + to decrypt the memo bundle as described in subsequent sections of this ZIP. + ## Memo bundle A memo bundle consists of a sequence of 256-byte memo chunks, each individually encrypted. These memo chunks represent zero or more encrypted memos. Each transaction may contain a single memo bundle, and a memo bundle may contain -at most 40 memo chunks. This limits the total amount of memo data that can be -conveyed within a single transaction to 10 KiB. +at most `memo_chunk_limit` memo chunks. This limits the total amount of memo data +that can be conveyed within a single transaction to `memo_chunk_limit * 256` bytes. + +`memo_chunk_limit` is a parameter to this specification, to be decided upon +by the community. The authors of this ZIP propose a maximum of 64 chunks, +resulting in a maximum total memo data length of 16 KiB. Memo bundles are encoded in transactions in a prunable manner: each memo chunk can be replaced by its representative digest. ## Memo encryption -During transaction construction, each memo is assigned a 32-byte `memo_key`. -These keys SHOULD be generated randomly, and MUST NOT be used to encrypt more -than one memo within a single transaction. +During transaction construction, each output with memo data is assigned a 32-byte +memo key $\mathsf{K^{memo}}$. These keys SHOULD be generated randomly, and MUST NOT +be used to encrypt more than one memo within a single transaction. If an output has +no memo data, it is assigned the memo key consisting of 32 $\mathtt{0xFF}$ bytes. + +In note plaintexts of v6-onward transactions, the 512-byte memo field is replaced +by $\mathsf{K^{memo}}\!$. -In shielded bundles, the 512-byte memo field is replaced by `[flag_byte] || memo_key`. +The transaction builder generates a 32-byte salt value $\mathsf{salt}$ from a CSPRNG. +A new salt MUST be generated for each memo bundle. -TODO: Define `flag_byte` format. Have one bit for "there is no memo". +The symmetric encryption key for a memo is derived from its $\mathsf{K^{memo}}$ as follows: -The transaction builder generates a 32-byte nonce from a CSPRNG. A new nonce MUST be -generated for each memo bundle. +$$\mathsf{encryption\_key} = \mathsf{PRF^{expand}_{K^{memo}}}([\mathtt{0xE0}] \,||\, \mathsf{salt})$$ -The symmetric encryption key for a memo is derived from its `memo_key` as follows: +The first byte $\mathtt{0xE0}$ should be added to the documentation of inputs to +$\mathsf{PRF^{expand}}$ in § 4.1.2 ‘Pseudo Random Functions’ [^protocol-abstractprfs]. - encryption_key = PRF^{expand}_{memo_key}([0xE0] || nonce) +If the generated key is 32 $\mathtt{0xFF}$ bytes, the transaction constructor MAY +repeat this procedure with a different salt, in order to avoid the recipient +misinterpreting the output as having no memo data. Since that has negligible +probability, it alternatively MAY omit this check. Each memo is padded to a multiple of 256 bytes with zeroes, and split into -256-byte chunks. Each memo chunk is encrypted with ChaCha20Poly1305 as follows: +256-byte chunks. Each memo chunk is encrypted with ChaCha20Poly1305 [^rfc-7539] as +follows: + +$$\mathsf{IETF\_AEAD\_CHACHA20\_POLY1305}(\mathsf{encryption\_key}, \mathsf{nonce}, \mathsf{memo\_chunk})$$ - IETF_AEAD_CHACHA20_POLY1305(encryption_key, nonce = I2BEOSP_{88}(counter) || [final_chunk], memo_chunk) +where $\mathsf{nonce} = \mathsf{I2BEOSP}_{88}(\mathsf{counter}) || [\mathsf{final\_chunk}]\!$. -This is a variant of the STREAM construction [#stream]_. -- `counter` is a big-endian chunk counter starting at zero and incrementing by +This is a variant of the STREAM construction [^stream]. +- $\mathsf{counter}$ is a big-endian chunk counter starting at zero and incrementing by one for each subsequent chunk within a particular memo. -- `final_chunk` is the byte `0x01` for the final memo chunk, and `0x00` for all - preceding chunks. +- $\mathsf{final\_chunk}$ is the byte $\mathtt{0x01}$ for the final memo chunk, and + $\mathtt{0x00}$ for all preceding chunks. Finally, the encrypted memo chunks for all memos are combined into a single sequence using an order-preserving shuffle. Memo chunks from different memos MAY @@ -129,7 +268,6 @@ memos: (memo_a, 2), ] ``` - ## Memo decryption When a recipient decrypts a shielded output, they obtain a `memo_key`. From this @@ -156,15 +294,29 @@ wrong counter value), ensuring that a malformed memo is not returned. ## Encoding in transactions -| Bytes | Name | Data Type | Description | -|--------|---------------|--------------------------|---------------------------------------- | -| 32 | `nonce` | `byte[32]` | The nonce for deriving encryption keys. | -| 8 | `pruned` | `uint64` | Bitflags indicating the type of each entry in `vMemoChunks`. Bits are little endian. 0 corresponds to `byte[272]` and is an encrypted memo chunk. 1 corresponds to `byte[32]` and is the `memo_chunk_digest` for a pruned chunk. | -| varies | `nMemoChunks` | `compactSize` | Length of `vMemoChunks`. | -| varies | `vMemoChunks` | `MemoChunk[nMemoChunks]` | A sequence of encrypted memo chunks. | +| Bytes | Name | Data Type | Description | +|----------|---------------|--------------------------------|--------------------------------------------------------------| +| 1 | `fAllPruned` | `uint8` | 1 if all chunks have been pruned, otherwise 0. | +| 32 | `nonceOrHash` | `byte[32]` | The nonce for deriving encryption keys, or the overall hash. | +| † varies | `nMemoChunks` | `compactSize` | The number of memo chunks. | +| † varies | `pruned` | `byte[`$\mathsf{ceiling}(\mathtt{nMemoChunks}/8)$`]` | Bitflags indicating the type of each entry in `vMemoChunks`. | +| † varies | `vMemoChunks` | `MemoChunk[nMemoChunks]` | A sequence of encrypted memo chunks. | + +† These fields are present if and only if `fAllPruned == 0`. + +If `fAllPruned == 0`, then: -TODO: Decide whether to permit an optimised encoding for a fully-pruned memo -bundle that just contains `memo_bundle_digest`. +- `nonceOrHash` represents the nonce for deriving encryption keys. +- Each bit of `pruned`, in little-endian order, indicates the type of the + corresponding entry in `vMemoChunks`. A bit value of 0 indicates that + the entry will be of type `byte[272]` representing an encrypted memo + chunk. A bit value of 1 indicates the entry will be a `byte[32]` and + contains the `memo_chunk_digest` for a pruned chunk. + +If `fAllPruned == 1`, then: +- `nonceOrHash` represents the overall hash for the memo bundle as defined in + [Transaction sighash]. +- The `nMemoChunks`, `pruned`, and `vMemoChunks` fields will be absent. ## Transaction sighash @@ -185,29 +337,40 @@ transaction v6. A memo bundle may contain two free chunks if there are any shielded outputs in the transaction. Otherwise, each memo chunk requires `marginal_fee` as defined -in ZIP 317 [#zip-0317]_. +in ZIP 317 [^zip-0317]. + +## Network protocol + +Nodes must reject `GetData` responses having an `fAllPruned` value that is nonzero, +or any byte of `pruned` that is nonzero. -## Consensus rules -TODO: specify. +# Open issues + +## Limit on the number of memo chunks + +`memo_chunk_limit == 64` is recommended. This results in a maximum of 16 KiB +of memo data per transaction. # Rationale ## Memo bundle size restriction -Restricting the total amount of memo data in a bundle to 10 KiB limits the rate -at which the chain size can grow cheaply (from a computational perspective; memo -bundles are much easier to produce than proofs or signatures). +Restricting the total amount of memo data in a bundle, for example to 16 KiB, +limits the rate at which the chain size can grow cheaply (from a computational +perspective; memo bundles are much easier to produce than proofs or signatures). The current behaviour for previous transaction versions (no limit on the number of memos) is not altered by this ZIP, because memos in those transactions are tied to individual shielded outputs (incurring their computational cost), and -are not natively aggregable. +are not natively aggregatable. ## Memo chunk size -With 10KiB limit on amount of memo data as the constant in this table, the +TODO: this table needs to be recalculated with a 16 KiB limit. + +With 10 KiB limit on amount of memo data as the constant in this table, the maximum number of unique memos you can create, and the cost in bytes of that memo data plus auth when using a 32-byte memo key, is: @@ -242,38 +405,33 @@ becomes: The decrease in overhead is relatively modest in most cases, but more noticeable for small memos with a 256-byte memo chunk. -However, 16-byte memo keys don't meet Zcash's target security level of 125 bits. -The classical argument [#ref]_ is that you can build a machine of size 2^32 -("The variant key-search machine") that has a 2^-71 chance of finding a 128-bit -key (with sufficient targets, e.g. 2^36 *blocks*). In this scenario, you only -get 32+71 = 103 bits of security classically. +However, 128-bit keys don't meet Zcash's target security level of 125 bits, +as argued in [^protocol-inbandrationale]. -The benefits of 32-byte keys are: -- They incur a small transaction size overhead above the minimum key size that - _would_ meet that target security level. -- It is simpler than reasoning about unusual byte lengths for keys. +The benefits of 256-bit keys are: +- They incur only a small transaction size overhead above the minimum key size + that _would_ meet the target security level. - This key length matches what we already use elsewhere for symmetric keys. -See also [#protocol-inbandrationale]_. - ## Encryption format -Including a per-transaction `nonce` in the derivation of `encryption_key` gives -protection against accidental (or intentional) `memo_key` reuse across multiple -transactions. We do not protect against `memo_key` reuse within a transaction; -it is up to the transaction builder to ensure that the same `memo_key` is not +Including a per-transaction $\mathsf{salt}$ in the derivation of +$\mathsf{encryption_key}$ gives protection against accidental (or intentional) +reuse of $\mathsf{K^{memo}}$ reuse across multiple transactions. We do not +protect against $\mathsf{K^{memo}}$ reuse within a transaction; it is up to +the transaction builder to ensure that the same $\mathsf{K^{memo}}$ is not used to encrypt two different memos (and if they did so, normal clients would either never observe the second memo, or would decrypt parts of each memo and -get a non-sensical and potentially insecure "spliced" memo). +get a nonsensical and potentially insecure "spliced" memo). We do not include commitments to the shielded outputs in the derivation of -`encryption_key` for two reasons: +$\mathsf{encryption_key}$ for two reasons: - It would force the transaction builder to fully define all shielded outputs - before encrypting the memos, which might prevent potential use cases of PCZTs. + before encrypting the memos, which might prevent potential use cases of PCZTs [^pczt]. - We don't want to unnecessarily prevent the ability to create a transaction with a memo bundle and no shielded outputs, as there may be use cases for, e.g. a fully-transparent transaction with encrypted memo, or a ZSA issuance - transaction with exposed memo data using a well-known `memo_key`. + transaction with exposed memo data using a well-known $\mathsf{K^{memo}}$. ## Pruned encoding @@ -286,7 +444,7 @@ and prune the corresponding memo chunks, while still enabling the transaction to be validated as part of its corresponding block and broadcast over the network. The transaction encoding permits pruning at the individual chunk level in order -to faciliate pruning an individual memo from a transaction without affecting the +to facilitate pruning an individual memo from a transaction without affecting the other memos. This enables node operators to be responsive to, for example, GDPR deletion requests. @@ -326,7 +484,13 @@ TBD # References [^BCP14]: [Information on BCP 14 — "RFC 2119: Key words for use in RFCs to Indicate Requirement Levels" and "RFC 8174: Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words"](https://www.rfc-editor.org/info/bcp14) -[^protocol]: Zcash Protocol Specification, Version 2024.5.1 [NU6] or later](protocol/protocol.pdf) + +[^protocol-noteptconcept]: [Zcash Protocol Specification, Version 2024.5.1 [NU6]. Section 3.2.1: Note Plaintexts and Memo Fields](protocol/protocol.pdf) + [^protocol-inbandrationale]: [Zcash Protocol Specification, Version 2024.5.1 [NU6]. Section 8.7: In-band secret distribution](protocol/protocol.pdf#inbandrationale) + [^stream]: [Online Authenticated-Encryption and its Nonce-Reuse Misuse-Resistance](https://eprint.iacr.org/2015/189) + [^zip-0317]: [ZIP 317: Proportional Transfer Fee Mechanism](zip-0317.rst) + +[^pczt]: [zcash/zips issue #693: Standardize a protocol for creating shielded transactions offline](https://github.com/zcash/zips/issues/693) \ No newline at end of file From 2c9772f580980a42e74e59e9af5a159582db24f2 Mon Sep 17 00:00:00 2001 From: Daira-Emma Hopwood Date: Tue, 5 Nov 2024 22:00:20 +0000 Subject: [PATCH 10/11] ZIP 231: rendering fixes. Signed-off-by: Daira-Emma Hopwood --- zips/zip-0231.md | 83 ++++++++++++++++++++++++++++++++---------------- 1 file changed, 55 insertions(+), 28 deletions(-) diff --git a/zips/zip-0231.md b/zips/zip-0231.md index 33eae1361..d373536fb 100644 --- a/zips/zip-0231.md +++ b/zips/zip-0231.md @@ -1,15 +1,15 @@ -> ZIP: 231 -> Title: Memo Bundles -> Owners: Jack Grigg -> Kris Nuttycombe -> Daira-Emma Hopwood -> Arya Solhi -> Credits: Sean Bowe -> Nate Wilcox -> Status: Draft -> Category: Consensus / Wallet -> Created: 2024-04-26 -> License: MIT + ZIP: 231 + Title: Memo Bundles + Owners: Jack Grigg + Kris Nuttycombe + Daira-Emma Hopwood + Arya Solhi + Credits: Sean Bowe + Nate Wilcox + Status: Draft + Category: Consensus / Wallet + Created: 2024-04-26 + License: MIT # Terminology @@ -105,7 +105,7 @@ to both Sapling and Orchard outputs. ## Changes to the Zcash Protocol Specification -The following changes are to be made affecting note plaintexts, note ciphertexts, +The following changes affecting the definitions of note plaintexts and note ciphertexts, and the algorithms for encryption and decryption. In § 3.2.1 ‘Note Plaintexts and Memo Fields’: @@ -139,8 +139,9 @@ In § 5.5 ‘Encodings of Note Plaintexts and Memo Fields’ [^protocol-notepten > The encoding of a v6-onward Sapling or Orchard note plaintext consists of: > - > |---------------------------|---------------------|--------------------------|-----------------------------| - > | 8-bit $\mathsf{leadByte}$ | 88-bit $\mathsf{d}$ | 256-bit $\mathsf{rseed}$ | 32-byte $\mathsf{K^{memo}}$ | + > | | | | | | + > |---------------------------|---------------------|---------------------|--------------------------|-----------------------------| + > | 8-bit $\mathsf{leadByte}$ | 88-bit $\mathsf{d}$ | 64-bit $\mathsf{v}$ | 256-bit $\mathsf{rseed}$ | 32-byte $\mathsf{K^{memo}}$ | > > * A byte 0x03, indicating this version of the encoding of a v6-onward > Sapling or Orchard note plaintext. @@ -188,8 +189,8 @@ In § 4.20.1 ‘Encryption (Sapling and Orchard)’ [^protocol-saplingandorchard > * $\mathsf{C^{enc}}$ will be of length either 580 or 100 bytes, depending on whether > $\mathbf{np}$ is a pre-v6 or v6-onward note plaintext. -In § 4.20.2 ‘Decryption using an Incoming Viewing Key (Sapling and Orchard)’ and -§ 4.20.3 ‘Decryption using a full Incoming Viewing Key (Sapling and Orchard)’: +In § 4.20.2 ‘Decryption using an Incoming Viewing Key (Sapling and Orchard)’ [^protocol-decryptivk] +and § 4.20.3 ‘Decryption using a Full Viewing Key (Sapling and Orchard)’ [^protocol-decryptovk]: * Replace $\mathsf{memo} ⦂ \mathbb{B^{Y[512]}}$ with $\mathsf{memoOrKey}$. * Specify that the type of $\mathsf{memoOrKey}$ is $\mathbb{B^{Y[512]}}$ when @@ -228,7 +229,7 @@ A new salt MUST be generated for each memo bundle. The symmetric encryption key for a memo is derived from its $\mathsf{K^{memo}}$ as follows: -$$\mathsf{encryption\_key} = \mathsf{PRF^{expand}_{K^{memo}}}([\mathtt{0xE0}] \,||\, \mathsf{salt})$$ +$\hspace{2em}\mathsf{encryption\_key} = \mathsf{PRF^{expand}_{K^{memo}}}([\mathtt{0xE0}] \,||\, \mathsf{salt})$ The first byte $\mathtt{0xE0}$ should be added to the documentation of inputs to $\mathsf{PRF^{expand}}$ in § 4.1.2 ‘Pseudo Random Functions’ [^protocol-abstractprfs]. @@ -239,14 +240,15 @@ misinterpreting the output as having no memo data. Since that has negligible probability, it alternatively MAY omit this check. Each memo is padded to a multiple of 256 bytes with zeroes, and split into -256-byte chunks. Each memo chunk is encrypted with ChaCha20Poly1305 [^rfc-7539] as -follows: +256-byte chunks. Each memo chunk is encrypted with ChaCha20Poly1305 [^rfc-8439] +as follows: -$$\mathsf{IETF\_AEAD\_CHACHA20\_POLY1305}(\mathsf{encryption\_key}, \mathsf{nonce}, \mathsf{memo\_chunk})$$ +$\hspace{2em}\mathsf{IETF\_AEAD\_CHACHA20\_POLY1305}(\mathsf{encryption\_key}, \mathsf{nonce}, \mathsf{memo\_chunk})$ where $\mathsf{nonce} = \mathsf{I2BEOSP}_{88}(\mathsf{counter}) || [\mathsf{final\_chunk}]\!$. This is a variant of the STREAM construction [^stream]. + - $\mathsf{counter}$ is a big-endian chunk counter starting at zero and incrementing by one for each subsequent chunk within a particular memo. - $\mathsf{final\_chunk}$ is the byte $\mathtt{0x01}$ for the final memo chunk, and @@ -270,8 +272,8 @@ memos: ``` ## Memo decryption -When a recipient decrypts a shielded output, they obtain a `memo_key`. From this -they derive `encryption_key` as above, and then proceed as follows: +When a recipient decrypts a shielded output, they obtain a memo key $\mathsf{K^{memo}}$. +From this they derive `encryption_key` as above, and then proceed as follows: - Set `counter = 0` and `final_chunk = 0x00`. - Attempt to decrypt each memo chunk in order. Pruned memo chunks are skipped. @@ -314,6 +316,7 @@ If `fAllPruned == 0`, then: contains the `memo_chunk_digest` for a pruned chunk. If `fAllPruned == 1`, then: + - `nonceOrHash` represents the overall hash for the memo bundle as defined in [Transaction sighash]. - The `nMemoChunks`, `pruned`, and `vMemoChunks` fields will be absent. @@ -325,7 +328,7 @@ memo_chunk_digest = H(AEAD(MemoChunk, memo_key)) memo_bundle_digest = H(concat(memo_chunk_digests)) ``` -The memo bundle digest structure is a performance optimisation for the case +The memo bundle digest structure is a performance optimization for the case where all memo chunks in a transaction have been pruned. TODO: finish this to be a modification to the equivalent of ZIP 244 for @@ -375,8 +378,9 @@ maximum number of unique memos you can create, and the cost in bytes of that memo data plus auth when using a 32-byte memo key, is: | | Memo size | -| Chunk size | ≤ 256 bytes | 512 bytes | |------------|----------------------|----------------------| +| Chunk size | ≤ 256 bytes | 512 bytes | +|============|======================|======================| | Pre-231 | 20 @ 10240 ( 0.00%) | 20 @ 10240 ( 0.00%) | | 512 | 20 @ 11220 (+ 9.57%) | 20 @ 11220 (+ 9.57%) | | 256 | 40 @ 12200 (+19.14%) | 20 @ 11540 (+12.70%) | @@ -395,8 +399,9 @@ If we used a 16-byte memo key instead of 32 bytes, the transaction size overhead becomes: | | Memo size | -| Chunk size | ≤ 256 bytes | 512 bytes | |------------|----------------------|----------------------| +| Chunk size | ≤ 256 bytes | 512 bytes | +|============|======================|======================| | Pre-231 | 20 @ 10240 ( 0.00%) | 20 @ 10240 ( 0.00%) | | 512 | 20 @ 10900 (+ 6.45%) | 20 @ 10900 (+ 6.45%) | | 256 | 40 @ 11560 (+12.89%) | 20 @ 11220 (+ 9.57%) | @@ -409,6 +414,7 @@ However, 128-bit keys don't meet Zcash's target security level of 125 bits, as argued in [^protocol-inbandrationale]. The benefits of 256-bit keys are: + - They incur only a small transaction size overhead above the minimum key size that _would_ meet the target security level. - This key length matches what we already use elsewhere for symmetric keys. @@ -426,6 +432,7 @@ get a nonsensical and potentially insecure "spliced" memo). We do not include commitments to the shielded outputs in the derivation of $\mathsf{encryption_key}$ for two reasons: + - It would force the transaction builder to fully define all shielded outputs before encrypting the memos, which might prevent potential use cases of PCZTs [^pczt]. - We don't want to unnecessarily prevent the ability to create a transaction @@ -485,12 +492,32 @@ TBD [^BCP14]: [Information on BCP 14 — "RFC 2119: Key words for use in RFCs to Indicate Requirement Levels" and "RFC 8174: Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words"](https://www.rfc-editor.org/info/bcp14) -[^protocol-noteptconcept]: [Zcash Protocol Specification, Version 2024.5.1 [NU6]. Section 3.2.1: Note Plaintexts and Memo Fields](protocol/protocol.pdf) +[^protocol]: [Zcash Protocol Specification, Version 2024.5.1 [NU6] or later](protocol/protocol.pdf) + +[^protocol-noteptconcept]: [Zcash Protocol Specification, Version 2024.5.1 [NU6]. Section 3.2.1: Note Plaintexts and Memo Fields](protocol/protocol.pdf#noteptconcept) + +[^protocol-abstractprfs]: [Zcash Protocol Specification, Version 2024.5.1 [NU6]. Section 4.1.2: Pseudo Random Functions](protocol/protocol.pdf#abstractprfs) + +[^protocol-saplingsend]: [Zcash Protocol Specification, Version 2024.5.1 [NU6]. Section 4.7.2: Sending Notes (Sapling)](protocol/protocol.pdf#saplingsend) + +[^protocol-orchardsend]: [Zcash Protocol Specification, Version 2024.5.1 [NU6]. Section 4.7.3: Sending Notes (Orchard)](protocol/protocol.pdf#orchardsend) + +[^protocol-saplingandorchardinband]: [Zcash Protocol Specification, Version 2024.5.1 [NU6]. Section 4.20.1: Encryption (Sapling and Orchard)](protocol/protocol.pdf#saplingandorchardinband) + +[^protocol-decryptivk]: [Zcash Protocol Specification, Version 2024.5.1 [NU6]. Section 4.20.2: Decryption using an Incoming Viewing Key (Sapling and Orchard)](protocol/protocol.pdf#decryptivk) + +[^protocol-decryptovk]: [Zcash Protocol Specification, Version 2024.5.1 [NU6]. Section 4.20.3: Decryption using a Full Viewing Key (Sapling and Orchard)](protocol/protocol.pdf#decryptovk) + +[^protocol-noteptencoding]: [Zcash Protocol Specification, Version 2024.5.1 [NU6]. Section 5.5: Encodings of Note Plaintexts and Memo Fields](protocol/protocol.pdf#noteptencoding) [^protocol-inbandrationale]: [Zcash Protocol Specification, Version 2024.5.1 [NU6]. Section 8.7: In-band secret distribution](protocol/protocol.pdf#inbandrationale) [^stream]: [Online Authenticated-Encryption and its Nonce-Reuse Misuse-Resistance](https://eprint.iacr.org/2015/189) +[^zip-0307]: [ZIP 307: Light Client Protocol for Payment Detection](zip-0307.rst) + [^zip-0317]: [ZIP 317: Proportional Transfer Fee Mechanism](zip-0317.rst) -[^pczt]: [zcash/zips issue #693: Standardize a protocol for creating shielded transactions offline](https://github.com/zcash/zips/issues/693) \ No newline at end of file +[^pczt]: [zcash/zips issue #693: Standardize a protocol for creating shielded transactions offline](https://github.com/zcash/zips/issues/693) + +[^rfc-8439] [RFC 8439: ChaCha20 and Poly1305 for IETF Protocols](https://www.rfc-editor.org/rfc/rfc8439.html) \ No newline at end of file From ddc53dc39efb723e7aa8b9f7aa9fde3cf46aa3e7 Mon Sep 17 00:00:00 2001 From: Daira-Emma Hopwood Date: Wed, 6 Nov 2024 00:33:19 +0000 Subject: [PATCH 11/11] Apply suggestions from code review Co-authored-by: Kris Nuttycombe --- zips/zip-0231.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/zips/zip-0231.md b/zips/zip-0231.md index d373536fb..95f68ccbb 100644 --- a/zips/zip-0231.md +++ b/zips/zip-0231.md @@ -57,9 +57,10 @@ so the resulting decryption algorithm does not meet standard security notions for an authenticated encryption scheme. It is a goal of this proposal to rectify this, simplifying the security argument. -In the light client protocol, a recipient need not download full transaction -information if the compact transaction information indicates that they do not -receive any memo in the transaction. +Instead of the memo data, this ZIP proposes that it is possible to indicate whether +a memo is present for the recipient. When using the light client protocol, a recipient +need not download full transaction information if this indication tells them that they +have not received any memo in the transaction. At present, it is not possible to transmit the same memo data to multiple transaction recipients without redundantly encoding that data, and sending @@ -78,7 +79,8 @@ while decreasing the amount of data that needs to be stored on-chain overall. - The exact number and exact lengths of distinct decryptable memos should not be revealed, even to the transaction recipients, although an upper bound on the total length of memo data that the observer does not have the capability - to view will be leaked. + to view will be leaked to transaction recipients, and the overall maximum possible length of + memo data will be revealed on-chain. - A recipient can determine whether or not they have been given the capability to view any memo solely by decrypting the note ciphertext. - Memo chunks within a transaction can be individually pruned from block storage @@ -514,6 +516,8 @@ TBD [^stream]: [Online Authenticated-Encryption and its Nonce-Reuse Misuse-Resistance](https://eprint.iacr.org/2015/189) +[^zip-0302]: [ZIP 302: Standardized Memo Field Format](zip-0302.rst) + [^zip-0307]: [ZIP 307: Light Client Protocol for Payment Detection](zip-0307.rst) [^zip-0317]: [ZIP 317: Proportional Transfer Fee Mechanism](zip-0317.rst)