Panic during BatchCommitWriteStreams

[mluiten]

What happened?

Writing an integration test using a pending stream, appending some data, then finalizing and committing the streams. All works well until the the commit is sent, which crashes the emulator:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x60 pc=0x15979f0]

goroutine 71 [running]:
github.com/goccy/bigquery-emulator/types.normalizeData({0x2ad6a40?, 0xc000870910?}, 0x0)
	/work/types/types.go:547 +0xb0
github.com/goccy/bigquery-emulator/types.normalizeData({0x2b76960?, 0xc0006b38c0?}, 0xc000bcd950)
	/work/types/types.go:575 +0x488
github.com/goccy/bigquery-emulator/types.normalizeData({0x2b76960?, 0xc0006b3830?}, 0xc000ad8000)
	/work/types/types.go:575 +0x488
github.com/goccy/bigquery-emulator/types.normalizeData({0x2abcf40?, 0xc000994d68?}, 0xc000bcd590)
	/work/types/types.go:553 +0x914
github.com/goccy/bigquery-emulator/types.NewTableWithSchema(0xc0009e1200, {0xc000a55c28, 0x1, 0x10?})
	/work/types/types.go:479 +0x479
github.com/goccy/bigquery-emulator/server.(*storageWriteServer).insertTableData(0xc00059e1e0, {0x30f44e8, 0xc000857140}, 0xc000992ba9?, 0xc00096a930, {0xc000a55c28?, 0xc?, 0xc0009a5940?})
	/work/server/storage_handler.go:662 +0x4e
github.com/goccy/bigquery-emulator/server.(*storageWriteServer).BatchCommitWriteStreams(0xc00059e1e0, {0x30f44e8, 0xc000857140}, 0x0?)
	/work/server/storage_handler.go:731 +0x472
cloud.google.com/go/bigquery/storage/apiv1/storagepb._BigQueryWrite_BatchCommitWriteStreams_Handler({0x2d53840?, 0xc00059e1e0}, {0x30f44e8, 0xc000857140}, 0xc000acc180, 0x0)
	/go/pkg/mod/cloud.google.com/go/[email protected]/storage/apiv1/storagepb/storage.pb.go:3432 +0x169
google.golang.org/grpc.(*Server).processUnaryRPC(0xc0005c0000, {0x30f44e8, 0xc000856f90}, {0x31034e0, 0xc0004f0b60}, 0xc000abe120, 0xc00059e2a0, 0x4b74e08, 0x0)
	/go/pkg/mod/google.golang.org/[email protected]/server.go:1386 +0xe23
google.golang.org/grpc.(*Server).handleStream(0xc0005c0000, {0x31034e0, 0xc0004f0b60}, 0xc000abe120)
	/go/pkg/mod/google.golang.org/[email protected]/server.go:1797 +0x100c
google.golang.org/grpc.(*Server).serveStreams.func2.1()
	/go/pkg/mod/google.golang.org/[email protected]/server.go:1027 +0x8b
created by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 12
	/go/pkg/mod/google.golang.org/[email protected]/server.go:1038 +0x135

What did you expect to happen?

The commit not to enter a panic error

How can we reproduce it (as minimally and precisely as possible)?

Still working on this -- not sure which part could be causing it as there is very little information to go on.

Anything else we need to know?

Using the latest docker image, testing using testcontainers

[mluiten]

Upon further investigation this seems to happen when the schema is created using different capitalization than the data -- for example, some fields called createdAt, then later the data is inserted using createdat ... this works in BQ because the columns are case insensitive, but this crashes the emulator.