[ADAP-1055] [Regression] schema is no longer available in unrendered_config when doing a run with partial parsing

[Spince]

Is this a regression in a recent version of dbt-bigquery?

• I believe this is a regression in dbt-bigquery functionality
• I have searched the existing issues, and I could not find an existing issue for this regression

Current Behavior

for a model that has it's schema defined via dbt_project.yml:

• after a dbt clean run a model (full parse)
• check the model in the manifest for schema in unrendered_config it is there
• change the model
• run the same model (partial parse)
• check the model in the manifest for schema in unrendered_config it is not there

because of the above behavior it causes a failure. we have our own version of the generate_alias_name macro to write all objects to a single schema when running outside of production with the production schema prepended to the object name. inside we force an error if a schema isn't defined by using an invalid character in the placeholder schema name.

{% macro generate_alias_name(custom_alias_name=none, node=none) -%}
    {%- set node_type = node.resource_type | trim %}
    {%- if custom_alias_name is none -%}
        {%- set table_name = node.name -%}
    {%- else -%}
        {%- set table_name = custom_alias_name | trim -%}
    {%- endif -%}
    {%- if target.name == 'prod' or node_type == 'snapshot' -%}
        {{ table_name }}
    {%- else -%}
        {#- Get the custom schema name -#}
        {%- set schema_prefix = node.unrendered_config.schema | trim %}
        {#- Highlight if schema hasn't been assigned right -#}
        {%- if not schema_prefix -%}
            {%- set schema_prefix = '[assigned_schema_required]' %}
        {%- endif -%}
        {{  schema_prefix ~ "__" ~ table_name }}
    {%- endif -%}
{%- endmacro %}

if we define the schema in a model config then it also works but that's obviously less than ideal and not a real fix

we're using bigquery and are on dbt cloud but this error only occurs when running things locally because you wouldn't do the same steps anywhere else.

what leads me to believe it is a regression with partial parsing is the fact that after running the steps to reproduce and then a dbt clean the model works again.

Expected/Previous Behavior

• after a dbt clean run a model (full parse)
• check the model in the manifest for schema in unrendered_config it is there
• change the model
• run the same model (partial parse)
• check the model in the manifest for schema in unrendered_config it is still there

Verified these steps work fine on dbt-bigquery==1.6.0 and dbt-bigquery==1.5.9

Steps To Reproduce

1. create the macro with our code (see below)
2. create a new folder + config block in dbt_project.yml to specify the schema
3. create a model in the new folder select 1 as col_a
4. dbt run -s new_model should work
5. edit the model select 1 as col_a, 2 as col_b
6. dbt run -s new_model should now fail with the bad schema because schema is no longer in unrendered_config

{% macro generate_alias_name(custom_alias_name=none, node=none) -%}
    {%- set node_type = node.resource_type | trim %}
    {%- if custom_alias_name is none -%}
        {%- set table_name = node.name -%}
    {%- else -%}
        {%- set table_name = custom_alias_name | trim -%}
    {%- endif -%}
    {%- if target.name == 'prod' or node_type == 'snapshot' -%}
        {{ table_name }}
    {%- else -%}
        {#- Get the custom schema name -#}
        {%- set schema_prefix = node.unrendered_config.schema | trim %}
        {#- Highlight if schema hasn't been assigned right -#}
        {%- if not schema_prefix -%}
            {%- set schema_prefix = '[assigned_schema_required]' %}
        {%- endif -%}
        {{  schema_prefix ~ "__" ~ table_name }}
    {%- endif -%}
{%- endmacro %}

Relevant log output

No response

Environment

- OS: macOS Sonoma 14.1.1
- Python: 3.11.3 (plus others)
- dbt-core (working version): 1.6.0
- dbt-bigquery (working version): 1.6.0
- dbt-core (regression version): 1.7.1
- dbt-bigquery (regression version): 1.7.2

Additional Context

slack thread with perhaps an easier to follow sequence of information: https://getdbt.slack.com/archives/CBSQTAPLG/p1701365532730099

[Spince]

what's the process for getting something like this looked at?

[Spince]

dbt-core==1.7.3 seems to have resolved this issue

[dave-connors-3]

per @Spince , this problem persists cc: @dbeatty10

[Spince]

I am working on new steps to reproduce. The same error occurs but not with the same steps.

[dbeatty10]

@Spince were you able to come up with new steps to reproduce?

[Spince]

@dbeatty10 I have not been able to reliably reproduce since dbt-core 1.7.3. we can probably close this until i have clear steps to reproduce!