Cannot read properties of undefined (reading 'get') on referenceMap.get(id)

[jimmywarting]

Do you know what's wrong with this code?
is this a bug or a misstake on my side?

import { Encoder, decode, encode } from 'cbor-x'

let cbor = new Encoder({
  structuredClone: true,
  pack: true,
  mapsAsObjects: true
})

encodeDecode({
  foo: 'bar',
  arr: Array(2).fill({}),
  num: 1,
  date: new Date(),
  blob: new Blob(['foo'])
})

async function encodeDecode(data) {
  const blob = new Blob([...cbor.encodeAsIterable(data)])
  const ab = await blob.arrayBuffer()
  const res = cbor.decode(new Uint8Array(ab))
  console.log(res)
}

file:///<path>/node_modules/cbor-x/decode.js:1008
	let refEntry = referenceMap.get(id)
	                            ^

TypeError: Cannot read properties of undefined (reading 'get')
    at currentExtensions.<computed> (file:///<path>/node_modules/cbor-x/decode.js:1008:30)
    at read (file:///<path>/node_modules/cbor-x/decode.js:418:13)
    at read (file:///<path>/node_modules/cbor-x/decode.js:348:47)
    at read (file:///<path>/node_modules/cbor-x/decode.js:388:31)
    at checkedRead (file:///<path>/node_modules/cbor-x/decode.js:202:16)
    at Encoder.decode (file:///<path>/node_modules/cbor-x/decode.js:150:12)
    at encodeDecode (file:///<path>/main.js:20:20)

if i set structuredClone to false, then it works alright...

[jimmywarting]

Here is another version that fails:

import * as cbor from 'cbor-x'

const encoder = new cbor.Encoder({
  structuredClone: true,
  pack: true,
  mapsAsObjects: true
})

const f = {}
const res = [...encoder.encodeAsIterable({
  arr: Array(2).fill({}),
  // baz: new Blob(['ss'])
})]

const u8 = new Uint8Array(await new Blob(res).arrayBuffer())
console.log(u8)
console.log(encoder.decode(u8))

[kriszyp]

Yes, this is a bug; or perhaps I should say that there simply is no support/implementation for structuredClone for encodeAsIterable yet.

[jimmywarting]

Can you explain what the problem is, why there is no support or what needs to be fixed?

[kriszyp]

Sure, the standard encode function works by inserting all the ids necessary for marking and referencing linked objects (in your case the first object has to be marked so the second object can reference it).
https://github.com/kriszyp/cbor-x/blob/master/encode.js#L188-L195
But this iterable encoder has a different path because it has do the individual value encodings as additions to a buffer. I think all these encode code calls need to be updated to check if the referenceMap has ids that need to be inserted. This is certainly doable and would be nice to get done some time.
Also, it is worth noting that I think it would be impossible/undesirable for the structured cloning to do any referencing of objects from pass values in an iterable since they would require retaining the referenceMap and the memory for past iterations which is contrary to the goal of not retaining memory from past iterations.

[jimmywarting]

Hmm. Do you know what this reference talk reminds me of? trying to mimic browsers structural clone alg. and also of how node:v8 (de)serialize works like.

import v8 from 'node:v8'

const a = Array(2).fill({})
const ab = new ArrayBuffer(16)
const b = new Uint8Array(ab)
const c = new Uint16Array(ab)
const d = new Uint32Array(32)
const value = { a, b, c, d }

const buf = v8.serialize(value)
const deserialized = v8.deserialize(buf)

console.assert(deserialized.a[0] === deserialized.a[1])
console.assert(deserialized.b.buffer === deserialized.c.buffer)
console.assert(deserialized.b.buffer !== deserialized.d.buffer)

I think i would like to treat any ArrayBufferView as something more like a generic-object.
In my point of view there isn't any special TypedArray (like Uint16Array or Float32Array) only generic object linking something to an existing ArrayBuffer with a references, byteLength and also a byteOffset. That is what they are to me...

Two different type of TypedArrays can both references the same underlying ArrayBuffer. (like b & c in the example above)

---

So i would actually like to have TypedArrays to be encoded more like: ['Uint8array', Arraybuffer_Reference_Id_in_ referenceMap, offset, byteLength] rather than using cbor specific tags for defining that something is of type: Uint16 or something like that.