Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow Option to ingest file with Parquet Logical types #3562

Open
jw-amazon opened this issue Oct 30, 2023 · 2 comments
Open

Allow Option to ingest file with Parquet Logical types #3562

jw-amazon opened this issue Oct 30, 2023 · 2 comments
Labels
enhancement New feature or request

Comments

@jw-amazon
Copy link

Is your feature request related to a problem? Please describe.

Logical types are used to extend the types that parquet can be used to store, by specifying how the primitive types should be interpreted. This keeps the set of primitive types to a minimum and reuses parquet's efficient encodings. Currently, the logic types are ignored during the ingestion process.

Describe the solution you'd like
An configuration option to allow OSI to ingestion data with logic type

Describe alternatives you've considered (Optional)
Right now, I have to create another filed so OSI can ingest the field into the correct logic type (date)

Additional context
Add any other context or screenshots about the feature request here.

@dlvenable
Copy link
Member

@jw-amazon , Thank you for the request. For our own prioritization, do you have a list of logical types that you are most needing?

@dlvenable dlvenable added enhancement New feature or request and removed untriaged labels Oct 30, 2023
@jw-amazon
Copy link
Author

I am mostly interested to be able to ingest to opensearch as date if that is the logical type, by default, it is ingested as numeric value.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Development

No branches or pull requests

2 participants