Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: Materialized view with auto-refresh will not redeploy when definition has changed #22

Closed
Tracked by #47
Luttik opened this issue Feb 9, 2022 · 3 comments
Closed
Tracked by #47

Comments

@Luttik
Copy link
Contributor

Luttik commented Feb 9, 2022

Currently materialized views with auto-refresh won't rebuild when the definition of the DBT model has changed. It is good that DBT doesn't refresh the views when either the calculated view is unchanged, but it definitely should be deployed the definition was changed.

I see two solutions for this:

  1. Check if the definition of the materialized view changed on the user's machine
  2. Check if the calculated materialized view differs from the materialized view that is currently deployed

I think the latter is the more elegant one, but I don't know which are feasible given the options working within dbt macro's.

The fix could either be applied in the refresh_materialized_view methods which are dependent on the specific data warehouse or extending the elif statement in line 19 of default/materialized_view to include one of the checks mentioned above.

@jtcohen6
Copy link
Collaborator

jtcohen6 commented Feb 9, 2022

@Luttik Love that you're thinking about this! What do you think about leveraging dbt's state-based selection to accomplish this? Then this would look something like:

# "full refresh" all MVs that have changes
dbt -s state:modified,config.materialized:materialized_view --full-refresh

# partial refresh of MVs without changes
dbt -s config.materialized:materialized_view --exclude state:modified

@Luttik
Copy link
Contributor Author

Luttik commented Feb 9, 2022

I'm not sure if I'm tracking completely. However, I think that this should just work automagically without having to specify anything in addition to the dbt run command, after a run you expect the live definitions to be in-sync with your dbt definitions.

Not refreshing data vs not syncing the model is very different in my eyes, the first will correct itself at some point while the latter does not.

@dataders
Copy link

dataders commented Jun 4, 2024

hey @Luttik thanks for opening this issue! Even though MVs & DTs now live in Core, this limitation that you call out still existis. I'm closing this issue in favor of dbt-labs/dbt-adapters#234 so that we can track the use case and priority over there moving forward

@dataders dataders closed this as not planned Won't fix, can't repro, duplicate, stale Jun 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants