Error reading from delta table with >= 100 commits

[zachschuermann]

From duckdb/duckdb#12983

InvalidInputException: Invalid Input Error: Attempting to execute an unsuccessful or closed pending query result
Error: IO Error: Hit DeltaKernel FFI error (from: While trying to read from delta table: './test_table/'): Hit error: 2 (ArrowError) with message (Invalid argument error: Incorrect datatype for StructArray field "partitionValues", expected Map(Field { name: "entries", data_type: Struct([Field { name: "keys", data_type: Utf8, nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }, Field { name: "values", data_type: Utf8, nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }]), nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }, false) got Map(Field { name: "entries", data_type: Struct([Field { name: "key", data_type: Utf8, nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }, Field { name: "value", data_type: Utf8, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }]), nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }, false))

[ion-elgreco]

@zachschuermann it's a problem when reading the checkpoint, the engine assumes Map type is constructed with Entries, keys, values but it's receiving entries, key value

[zachschuermann]

Yep looks like it. I'm guessing we aren't reading the checkpoint when @samansmink and I are failing to repro the issue. First thing I'd like to do is understand the repro then fix

[zachschuermann]

upstream issue closed