Authenticated stream of XChaCha20-Poly1305 - possible to have one authentication tag? How to initialize IV

[tomekit]

I am looking for a way to decrypt/encrypt big files (in ranges of GB) without allocating that much memory. I wouldn't like to give up the authentication checks.
I am also looking to use my IV instead of a random one (This requirement is because on my side the encryption logic is already being given the Nonce/Key which is stored separately in object metadata instead of file contents).

This API: https://doc.libsodium.org/secret-key_cryptography/secretstream appends the 17 bytes (authentication tag?) every time: crypto_secretstream_xchacha20poly1305_pull is being called and doesn't seem to allow to initialize custom IV.

This API on the other hand doesn't give option to authenticate streams: https://doc.libsodium.org/advanced/stream_ciphers/xchacha20

I am coming from AES-GCM background which is also an authenticated streaming cipher and such operations are supported in its ecosystem.

I am failing to find such option with libsodium for XChaCha20-Poly1305.
Is such use case supported?
In other words is it possible to decrypt/encrypt stream and use single authentication tag at the end?
Is it possible to initalize the IV without giving up the: Poly1305 part?

With Python you can initialize nonce, stream ciphertext (technically even up to 64GB AES limit) and sign with a single authentication tag at the end.
With Bouncycastle you can achieve the same using: processBytes and doFinal methods.
Please find code code samples below, which demonstrate such possibility with AES-GCM.

import Crypto.Cipher.AES
import binascii

key = bytes.fromhex('88fe0ff8c4eaf468d4cd9d9a9831662488fe0ff8c4eaf468d4cd9d9a98316624')
nonce = bytes.fromhex('3c95422167063c9542216706')

# streamed encryption then signing
cipher2 = Crypto.Cipher.AES.new(key, Crypto.Cipher.AES.MODE_GCM, nonce=nonce)
print(cipher2.encrypt(b'verylong').hex())                     # e7c5a84a16a30f43
print(cipher2.encrypt(b'video').hex())                     # 64700e4aed
output, mac = cipher2.encrypt_and_digest(b'files')
print(output.hex()) # 128085a073
print(mac.hex()) # e1d9a6971f30d5a1db4f5d36903ceb71

# streamed decryption and verification
cipher3 = Crypto.Cipher.AES.new(key, Crypto.Cipher.AES.MODE_GCM, nonce=nonce)

text = cipher3.decrypt(binascii.unhexlify('e7c5a84a16a30f43'))
print(text) # b'verylong'

text = cipher3.decrypt(binascii.unhexlify('64700e4aed'))
print(text) # b'video'

text = cipher3.decrypt_and_verify(binascii.unhexlify('128085a073'), mac)
print(text) # b'files'

# Pointycastle
final encryptedStream = blockStream.transform<Uint8List>(
  StreamTransformer.fromHandlers(
    handleData: (data, sink) {
      final encryptedBlockBuffer = Uint8List(encrypter.blockSize);
      final processedBytes = encrypter.processBlock(data, 0, encryptedBlockBuffer, 0);
      sink.add(encryptedBlockBuffer.sublist(0, processedBytes));
    },
    handleDone: (sink) {
      var macBuffer = Uint8List(encrypter.macSize);
      encrypter.doFinal(macBuffer, 0);
      sink.add(macBuffer);
      sink.close();
    },
  ),
);

[jedisct1]

No, libsodium doesn't allow applications to get a message, even partially, prior to having checked that it wasn't tampered with.

You need to split the message into pieces, and encrypt each piece individually.

[tomekit]

Thanks for your answer.

At least this clearly tells me to not spend more time trying to find something which libsodium doesn't support.

Regarding chunking the messages I can't really do that, since this would be limit my options to implement video streams:
#1252 (comment)
I am still researching that option though. If I used the 10MB chunk, that means that for every 10MB chunk there is a 17 bytes overhead.
For a 1GB video that is roughly up to 1700 bytes cipher/plain text position misalignment which is completely negligible.
However such big chunk might result for up to 10MB lost data (e.g. if user starts seeking a video from a 2nd byte - the whole chunk will have to be discarded, since it wouldn't pass the authentication due to missed first byte).
I might then reduce chunk size to avoid this, but then I increase the streams position misalignment.
That's something I need to figure out.

One question that I have, the cryptostream API has a following property: "Messages cannot be truncated, removed, reordered, duplicated or modified without this being detected by the decryption functions."
Given that each chunk is independent, how this is implemented within libsodium? Does it perform some additional HMAC over individual authentication tags?

Thanks for your help.