Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

INT96 timestamp is read as OffsetDateTime, not LocalDateTime #12266

Open
2 of 3 tasks
anesterenok opened this issue Feb 14, 2025 · 0 comments
Open
2 of 3 tasks

INT96 timestamp is read as OffsetDateTime, not LocalDateTime #12266

anesterenok opened this issue Feb 14, 2025 · 0 comments
Labels
bug Something isn't working

Comments

@anesterenok
Copy link

Apache Iceberg version

1.8.0 (latest release)

Query engine

Other

Please describe the bug 🐞

When a Parquet int64 timestamp is read, it is read as LocalDateTime (correct):

static class TimestampReader extends ParquetValueReaders.PrimitiveReader<LocalDateTime> {

However, when an int96 timestamp is read, it is read as OffsetDateTime (wrong):

static class TimestampInt96Reader extends ParquetValueReaders.PrimitiveReader<OffsetDateTime> {

This is wrong because OffsetDateTime is a type for timestamps with time zone, like (correct):

static class TimestamptzReader extends ParquetValueReaders.PrimitiveReader<OffsetDateTime> {

Expected: all timestamp reads return LocalDateTime, all timestamps with time zone reads return OffsetDateTime. Because it is the schema that defines how a value is interpreted, not its storage format.

As for fixing this: if it's just changing GenericParquetReaders (and a test), I can do a PR. If it's something more substantial, I'm afraid I cannot help.

Willingness to contribute

  • I can contribute a fix for this bug independently
  • I would be willing to contribute a fix for this bug with guidance from the Iceberg community
  • I cannot contribute a fix for this bug at this time
@anesterenok anesterenok added the bug Something isn't working label Feb 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant