Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature/databricks-sql-warehouse-compatibility #121

Merged
Merged
Changes from 1 commit
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
137ed6c
feature/databricks-sql-warehouse-compatibility
fivetran-joemarkiewicz Apr 2, 2024
382dbe2
run databricks sql
fivetran-joemarkiewicz Apr 2, 2024
99d7f83
changelog pr ref and schema update
fivetran-joemarkiewicz Apr 2, 2024
12ea509
changes to incremental startegy selection
fivetran-joemarkiewicz Apr 2, 2024
c575861
databricks different schemas
fivetran-joemarkiewicz Apr 2, 2024
b0ac2fc
sql warehouse specific test runs
fivetran-joemarkiewicz Apr 2, 2024
bd2175b
adjustment for databricks sql and all other destinations
fivetran-joemarkiewicz Apr 2, 2024
d41ebf8
unique schema name to ensure drop wont conflict
fivetran-joemarkiewicz Apr 2, 2024
839c093
changelog reword
fivetran-joemarkiewicz Apr 2, 2024
4e87cfa
docs regen
fivetran-joemarkiewicz Apr 2, 2024
cf257e5
validations and docs regen
fivetran-joemarkiewicz Apr 2, 2024
37e5c52
variable adjustment
fivetran-joemarkiewicz Apr 2, 2024
91ffac9
change cleanup
fivetran-joemarkiewicz Apr 2, 2024
aa43436
changelog update
fivetran-joemarkiewicz Apr 2, 2024
bd30997
update readme & changelog
fivetran-catfritz Apr 3, 2024
2cc5a70
Update models/fivetran_platform__audit_table.sql
fivetran-joemarkiewicz Apr 3, 2024
8aad434
Update README.md
fivetran-joemarkiewicz Apr 3, 2024
9df06fb
docs regen after review
fivetran-joemarkiewicz Apr 3, 2024
064955d
spark removal from macro
fivetran-joemarkiewicz Apr 3, 2024
ce3abc9
Apply suggestions from code review
fivetran-joemarkiewicz Apr 3, 2024
ba8ef14
changelog for fileformat addition
fivetran-joemarkiewicz Apr 3, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update models/fivetran_platform__audit_table.sql
Co-authored-by: fivetran-catfritz <[email protected]>
commit 2cc5a7003838c155e4f4a5743e73d99fe508f0a7
2 changes: 1 addition & 1 deletion models/fivetran_platform__audit_table.sql
Original file line number Diff line number Diff line change
@@ -7,7 +7,7 @@
} if target.type == 'bigquery' else ['sync_start_day'],
cluster_by = ['sync_start_day'],
incremental_strategy='insert_overwrite' if target.type in ('bigquery','spark', 'databricks') else 'delete+insert',
file_format='parquet' if not is_databricks_sql_warehouse(target) else 'delta'
file_format='delta' if is_databricks_sql_warehouse(target) else 'parquet'
) }}

with sync_log as (