Provide a configuration to allow ingesting the s3 data by file timestamp or some other configurable attributes.

[jw-amazon]

Is your feature request related to a problem? Please describe.
Provide an configuration to allow ingesting the s3 data by file timestamp or some other attribute.

Describe the solution you'd like
Our batch process creates the parquet files in batches, we like to enforce some order to ingest the source files.

Describe alternatives you've considered (Optional)
We have to create a cumbersome workaround to dump files in batches, and wait for the first batch to be ingested firstly before dumping the second batch.

Additional context
Feel free to reach out to me internally for clarfication.