Athena support #2
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
TODO: handle properly to allow casts for insert expressions without quoting all strings
using drop_relation() doesn't work in this context as type is null use explicit drop table for now
it should be done in athena-connector properly
artem-garmash
pushed a commit
that referenced
this pull request
Oct 10, 2023
) * get_test_buckets_min_and_max: bugfixes + clearer names * get_config_var: make the min_training_set_size much smaller than days_back * test_table_anomalies: log fix * tests: add coverage for metrics backfill logic in various cases * test-warehouse: remove --pre * test fixes * test fixes * get_buckets_configuration: add cast as timestamp to deal with BQ issue when the value is null * tests fix: don't use ilike since BQ doesn't have it * tests: copy the dbt project to a temp dir, to avoid races * CI: allow running with --pre as a parameter * CI: bugfix * CI: bugfix * tests: only copy the dbt project dir once per process * CI: add daily workflow for running on pre-releases * CI: bugfix * CI: bugfix * tests: temp test - remove project copy to see if it speeds things up * temporary test number #2 - don't run incremental query * return to regular logic * add HTML report as artifact * make HTML report self contained * test-warehouse: make report artifact unique * tests: use UTC today not date.today() * test fixes * tests performance optimization + add force_metrics_backfill var
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Athena support for elementary. Limited to Athena 3 and iceberg tables for incremental elementary tables. Testing with dbt-core 1.6.1, dbt-athena-community 1.6.0.
Clean rebase of #1
DROPinathena__create_or_replacetest_metrics(metrics ignored by missingrequire_partition_filter)test_pythontest_schema_changes(filtered_information_schema_columns is not updated with table materialization)