You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi team,
I am encountering an issue when using Recce to evaluate a PR. Specifically, I am seeing the following error :
"Env var required but not provided."
From my understanding, this happens because DBT includes the Jinja expressions in their raw form in the manifest.json file when the nodes are generated. While DBT correctly evaluates the variables at runtime, some configurations are recorded in the manifest as literal expressions, rather than being evaluated beforehand.
For example, in my dbt_project.yml I define the following configuration:
Although the variable DBT_ENVIRONMENT_NAME is correctly defined in the environment, it seems that Recce cannot access or interpret it properly. This issue arises regardless of whether Recce is run locally or within a GitHub Action. The error suggests that the variable is required but not being passed correctly during the evaluation process. This may be due to DBT storing the Jinja expression as-is in the manifest.json file, leaving it unevaluated for Recce's runtime checks.
Running it locally:
Running it in Github Actions:
Questions
Is there a way to pass the environment variable to Recce so that it can evaluate it?
Can DBT be configured to avoid including unprocessed Jinja expressions in the manifest.json?
Any guidance or suggestions would be greatly appreciated!
Expected Behavior
Recce should correctly evaluate the environment variable DBT_ENVIRONMENT_NAME during the row_count_diff check, whether it is run locally or within a GitHub Action. This would allow the Jinja expressions in the DBT configuration to be resolved and prevent the "Env var required but not provided" error.
Steps To Reproduce
Define the DBT_ENVIRONMENT_NAME variable in the environment.
Configure dbt_project.yml to use the variable in a Jinja expression (e.g., +database: "{{env_var('DBT_ENVIRONMENT_NAME', 'DEV')}}_STAGING").
Run the Recce row_count_diff check locally or within a GitHub Action.
Observe the error: "Env var required but not provided."
even-wei
changed the title
[Bug] Issue with Environment Variable in Recce row_count_diff Check
[DRC-1007] [Bug] Issue with Environment Variable in Recce row_count_diff Check
Jan 3, 2025
Thank you for providing such a detailed description of the problem and reproducing steps.
However, I can't reproduce the same situation with jaffle shop in our snowflake environment.
Thank you for the detailed follow-up and for sharing your configuration and process. I reviewed the issue again with my manager, and we discovered something that might be relevant to the problem.
In our setup, the DBT_ENVIRONMENT_NAME environment variable is not only used in the dbt_project.yml file but is also referenced within some models and macros. We suspect that this additional usage might be affecting how Recce processes or resolves the variable during runtime.
Example Macro
Here's an example of a macro that references the DBT_ENVIRONMENT_NAME variable:
Could you help confirm this by repeating your tests but using a model or macro that references DBT_ENVIRONMENT_NAME? This might help us determine whether the issue is specific to its usage in dbt_project.yml or if it impacts other areas where the variable is used.
Your insights and guidance would be greatly appreciated as we continue troubleshooting!
Current Behavior
Hi team,
I am encountering an issue when using Recce to evaluate a PR. Specifically, I am seeing the following error :
"Env var required but not provided."
From my understanding, this happens because DBT includes the Jinja expressions in their raw form in the manifest.json file when the nodes are generated. While DBT correctly evaluates the variables at runtime, some configurations are recorded in the manifest as literal expressions, rather than being evaluated beforehand.
For example, in my
dbt_project.yml
I define the following configuration:This results in the following configuration being written in the
manifest.json
:Although the variable DBT_ENVIRONMENT_NAME is correctly defined in the environment, it seems that Recce cannot access or interpret it properly. This issue arises regardless of whether Recce is run locally or within a GitHub Action. The error suggests that the variable is required but not being passed correctly during the evaluation process. This may be due to DBT storing the Jinja expression as-is in the manifest.json file, leaving it unevaluated for Recce's runtime checks.
Running it locally:
Running it in Github Actions:
Questions
manifest.json
?Any guidance or suggestions would be greatly appreciated!
Expected Behavior
Recce should correctly evaluate the environment variable
DBT_ENVIRONMENT_NAME
during therow_count_diff
check, whether it is run locally or within a GitHub Action. This would allow the Jinja expressions in the DBT configuration to be resolved and prevent the "Env var required but not provided" error.Steps To Reproduce
DBT_ENVIRONMENT_NAME
variable in the environment.dbt_project.yml
to use the variable in a Jinja expression (e.g.,+database: "{{env_var('DBT_ENVIRONMENT_NAME', 'DEV')}}_STAGING"
).row_count_diff
check locally or within a GitHub Action."Env var required but not provided."
Environment
DRC-1007
The text was updated successfully, but these errors were encountered: