Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update data_loss_prevention_discovery_config to include field support for OtherCloudDiscoveryTarget #12114

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

patrickmoy
Copy link
Member

Note that OtherCloudDiscoveryTarget currently only supports AWS S3 buckets in this update.

Release Note Template for Downstream PRs (will be copied)

See Write release notes for guidance.

dlp: added `other_cloud_target` and `other_cloud_starting_location` to `google_data_loss_prevention_discovery_config`

@github-actions github-actions bot requested a review from SirGitsalot October 24, 2024 08:50
Copy link

Hello! I am a robot. Tests will require approval from a repository maintainer to run.

@SirGitsalot, a repository maintainer, has been assigned to review your changes. If you have not received review feedback within 2 business days, please leave a comment on this PR asking them to take a look.

You can help make sure that review is quick by doing a self-review and by running impacted tests locally.

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 2202 insertions(+), 608 deletions(-))
google-beta provider: Diff ( 3 files changed, 2202 insertions(+), 608 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 512 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_data_loss_prevention_discovery_config (26 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_data_loss_prevention_discovery_config" "primary" {
  other_cloud_starting_location {
    aws_location {
      all_asset_inventory_assets = # value needed
    }
  }
  targets {
    other_cloud_target {
      conditions {
        amazon_s3_bucket_conditions {
          bucket_types           = # value needed
          object_storage_classes = # value needed
        }
      }
      filter {
        single_resource {
          amazon_s3_bucket {
            aws_account {
              account_id = # value needed
            }
            bucket_name = # value needed
          }
        }
      }
    }
  }
}

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 2202 insertions(+), 608 deletions(-))
google-beta provider: Diff ( 3 files changed, 2202 insertions(+), 608 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 512 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_data_loss_prevention_discovery_config (26 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_data_loss_prevention_discovery_config" "primary" {
  other_cloud_starting_location {
    aws_location {
      all_asset_inventory_assets = # value needed
    }
  }
  targets {
    other_cloud_target {
      conditions {
        amazon_s3_bucket_conditions {
          bucket_types           = # value needed
          object_storage_classes = # value needed
        }
      }
      filter {
        single_resource {
          amazon_s3_bucket {
            aws_account {
              account_id = # value needed
            }
            bucket_name = # value needed
          }
        }
      }
    }
  }
}

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 63
Passed tests: 62
Skipped tests: 0
Affected tests: 1

Click here to see the affected service packages
  • datalossprevention

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccDataLossPreventionDiscoveryConfig_Update

Get to know how VCR tests work

1 similar comment
@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 63
Passed tests: 62
Skipped tests: 0
Affected tests: 1

Click here to see the affected service packages
  • datalossprevention

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccDataLossPreventionDiscoveryConfig_Update

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

🔴 Tests failed during RECORDING mode:
TestAccDataLossPreventionDiscoveryConfig_Update [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

@modular-magician
Copy link
Collaborator

🔴 Tests failed during RECORDING mode:
TestAccDataLossPreventionDiscoveryConfig_Update [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 2202 insertions(+), 608 deletions(-))
google-beta provider: Diff ( 3 files changed, 2202 insertions(+), 608 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 512 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_data_loss_prevention_discovery_config (26 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_data_loss_prevention_discovery_config" "primary" {
  other_cloud_starting_location {
    aws_location {
      all_asset_inventory_assets = # value needed
    }
  }
  targets {
    other_cloud_target {
      conditions {
        amazon_s3_bucket_conditions {
          bucket_types           = # value needed
          object_storage_classes = # value needed
        }
      }
      filter {
        single_resource {
          amazon_s3_bucket {
            aws_account {
              account_id = # value needed
            }
            bucket_name = # value needed
          }
        }
      }
    }
  }
}

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 63
Passed tests: 62
Skipped tests: 0
Affected tests: 1

Click here to see the affected service packages
  • datalossprevention

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccDataLossPreventionDiscoveryConfig_Update

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

🔴 Tests failed during RECORDING mode:
TestAccDataLossPreventionDiscoveryConfig_Update [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

@patrickmoy patrickmoy marked this pull request as draft October 24, 2024 18:59
@patrickmoy
Copy link
Member Author

/gcbrun

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 2202 insertions(+), 608 deletions(-))
google-beta provider: Diff ( 3 files changed, 2202 insertions(+), 608 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 512 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_data_loss_prevention_discovery_config (26 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_data_loss_prevention_discovery_config" "primary" {
  other_cloud_starting_location {
    aws_location {
      all_asset_inventory_assets = # value needed
    }
  }
  targets {
    other_cloud_target {
      conditions {
        amazon_s3_bucket_conditions {
          bucket_types           = # value needed
          object_storage_classes = # value needed
        }
      }
      filter {
        single_resource {
          amazon_s3_bucket {
            aws_account {
              account_id = # value needed
            }
            bucket_name = # value needed
          }
        }
      }
    }
  }
}

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 63
Passed tests: 62
Skipped tests: 0
Affected tests: 1

Click here to see the affected service packages
  • datalossprevention

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccDataLossPreventionDiscoveryConfig_Update

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

🔴 Tests failed during RECORDING mode:
TestAccDataLossPreventionDiscoveryConfig_Update [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

@patrickmoy
Copy link
Member Author

/gcbrun

@patrickmoy
Copy link
Member Author

/gcbrun

1 similar comment
@patrickmoy
Copy link
Member Author

/gcbrun

@patrickmoy patrickmoy marked this pull request as ready for review November 19, 2024 08:02
@patrickmoy
Copy link
Member Author

/gcbrun

@patrickmoy patrickmoy marked this pull request as draft November 19, 2024 18:59
@patrickmoy patrickmoy marked this pull request as ready for review November 19, 2024 20:25
Copy link
Member

@SirGitsalot SirGitsalot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One nit and a suggestion for a config addition, but otherwise LGTM!

- name: 'awsLocation'
type: NestedObject
properties:
- name: 'accountId'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to the docs it's possible to specify accountId or allAssetInventoryAssets, but not both. I'm not sure if leaving both unset is valid, but in either case you can improve the user experience by adding either:

}
resource "google_organization_iam_member" "dlp_role" {
org_id = "%{organization}"
role = "roles/dlp.orgdriver"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: there's some weird indentation in the test configs (GitHub doesn't show it, but I'll bet it's due to a combination of tabs and spaces)

Copy link
Member

@SirGitsalot SirGitsalot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If adding roles/dlp.admin will do the trick, that change will need to be made in the CI Terraform config

return acctest.Nprintf(`
data "google_project" "project" {
}
resource "google_organization_iam_member" "service_agent" {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These IAM grants are the reason that the tests are failing on our CI system - the service account running the test doesn't have permission to make org-level IAM changes. What are the minimum IAM roles needed to create the config?

My reading of the docs makes me think that granting the SA running the test roles/dlp.admin at the org level will be enough to run the test:

If you don't have the Organization Administrator (roles/resourcemanager.organizationAdmin) or Security Admin (roles/iam.securityAdmin) role, you can still create a scan configuration. After you create the scan configuration, someone in your organization who has one of these roles must grant discovery access to the service agent.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated CI terraform config; will see if tests pass

Copy link

github-actions bot commented Dec 9, 2024

@patrickmoy, this PR is waiting for action from you. If no action is taken, this PR will be closed in 28 days.

Please address any comments or change requests, or re-request review from a core reviewer if no action is required.

Image showing the re-request review button

This notification can be disabled with the disable-automatic-closure label.

Copy link

@patrickmoy, this PR is waiting for action from you. If no action is taken, this PR will be closed in 14 days.

Please address any comments or change requests, or re-request review from a core reviewer if no action is required.

Image showing the re-request review button

This notification can be disabled with the disable-automatic-closure label.

Copy link

github-actions bot commented Jan 2, 2025

@patrickmoy, this PR is waiting for action from you. If no action is taken, this PR will be closed in 2 weekdays.

Please address any comments or change requests, or re-request review from a core reviewer if no action is required.

Image showing the re-request review button

This notification can be disabled with the disable-automatic-closure label.

@github-actions github-actions bot requested a review from SirGitsalot January 3, 2025 02:47
@patrickmoy
Copy link
Member Author

/gcbrun

2 similar comments
@zli82016
Copy link
Member

zli82016 commented Jan 3, 2025

/gcbrun

@zli82016
Copy link
Member

zli82016 commented Jan 3, 2025

/gcbrun

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 2186 insertions(+), 607 deletions(-))
google-beta provider: Diff ( 3 files changed, 2186 insertions(+), 607 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 501 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_data_loss_prevention_discovery_config (26 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_data_loss_prevention_discovery_config" "primary" {
  other_cloud_starting_location {
    aws_location {
      all_asset_inventory_assets = # value needed
    }
  }
  targets {
    other_cloud_target {
      conditions {
        amazon_s3_bucket_conditions {
          bucket_types           = # value needed
          object_storage_classes = # value needed
        }
      }
      filter {
        single_resource {
          amazon_s3_bucket {
            aws_account {
              account_id = # value needed
            }
            bucket_name = # value needed
          }
        }
      }
    }
  }
}

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 63
Passed tests: 62
Skipped tests: 0
Affected tests: 1

Click here to see the affected service packages
  • datalossprevention

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccDataLossPreventionDiscoveryConfig_Update

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

🔴 Tests failed during RECORDING mode:
TestAccDataLossPreventionDiscoveryConfig_Update [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

Copy link

github-actions bot commented Jan 7, 2025

@SirGitsalot This PR has been waiting for review for 3 weekdays. Please take a look! Use the label disable-review-reminders to disable these notifications.

@patrickmoy
Copy link
Member Author

/gcbrun

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 2186 insertions(+), 607 deletions(-))
google-beta provider: Diff ( 3 files changed, 2186 insertions(+), 607 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 501 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_data_loss_prevention_discovery_config (26 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_data_loss_prevention_discovery_config" "primary" {
  other_cloud_starting_location {
    aws_location {
      all_asset_inventory_assets = # value needed
    }
  }
  targets {
    other_cloud_target {
      conditions {
        amazon_s3_bucket_conditions {
          bucket_types           = # value needed
          object_storage_classes = # value needed
        }
      }
      filter {
        single_resource {
          amazon_s3_bucket {
            aws_account {
              account_id = # value needed
            }
            bucket_name = # value needed
          }
        }
      }
    }
  }
}

1 similar comment
@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 2186 insertions(+), 607 deletions(-))
google-beta provider: Diff ( 3 files changed, 2186 insertions(+), 607 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 501 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_data_loss_prevention_discovery_config (26 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_data_loss_prevention_discovery_config" "primary" {
  other_cloud_starting_location {
    aws_location {
      all_asset_inventory_assets = # value needed
    }
  }
  targets {
    other_cloud_target {
      conditions {
        amazon_s3_bucket_conditions {
          bucket_types           = # value needed
          object_storage_classes = # value needed
        }
      }
      filter {
        single_resource {
          amazon_s3_bucket {
            aws_account {
              account_id = # value needed
            }
            bucket_name = # value needed
          }
        }
      }
    }
  }
}

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 63
Passed tests: 62
Skipped tests: 0
Affected tests: 1

Click here to see the affected service packages
  • datalossprevention

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccDataLossPreventionDiscoveryConfig_Update

Get to know how VCR tests work

1 similar comment
@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 63
Passed tests: 62
Skipped tests: 0
Affected tests: 1

Click here to see the affected service packages
  • datalossprevention

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccDataLossPreventionDiscoveryConfig_Update

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

🔴 Tests failed during RECORDING mode:
TestAccDataLossPreventionDiscoveryConfig_Update [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

@modular-magician
Copy link
Collaborator

🔴 Tests failed during RECORDING mode:
TestAccDataLossPreventionDiscoveryConfig_Update [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

…veryTarget, which currently only supports AWS S3 buckets.
@patrickmoy
Copy link
Member Author

/gcbrun

@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 2186 insertions(+), 607 deletions(-))
google-beta provider: Diff ( 3 files changed, 2186 insertions(+), 607 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 501 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_data_loss_prevention_discovery_config (26 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_data_loss_prevention_discovery_config" "primary" {
  other_cloud_starting_location {
    aws_location {
      all_asset_inventory_assets = # value needed
    }
  }
  targets {
    other_cloud_target {
      conditions {
        amazon_s3_bucket_conditions {
          bucket_types           = # value needed
          object_storage_classes = # value needed
        }
      }
      filter {
        single_resource {
          amazon_s3_bucket {
            aws_account {
              account_id = # value needed
            }
            bucket_name = # value needed
          }
        }
      }
    }
  }
}

1 similar comment
@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 2186 insertions(+), 607 deletions(-))
google-beta provider: Diff ( 3 files changed, 2186 insertions(+), 607 deletions(-))
terraform-google-conversion: Diff ( 1 file changed, 501 insertions(+))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_data_loss_prevention_discovery_config (26 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_data_loss_prevention_discovery_config" "primary" {
  other_cloud_starting_location {
    aws_location {
      all_asset_inventory_assets = # value needed
    }
  }
  targets {
    other_cloud_target {
      conditions {
        amazon_s3_bucket_conditions {
          bucket_types           = # value needed
          object_storage_classes = # value needed
        }
      }
      filter {
        single_resource {
          amazon_s3_bucket {
            aws_account {
              account_id = # value needed
            }
            bucket_name = # value needed
          }
        }
      }
    }
  }
}

@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 63
Passed tests: 62
Skipped tests: 0
Affected tests: 1

Click here to see the affected service packages
  • datalossprevention

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccDataLossPreventionDiscoveryConfig_Update

Get to know how VCR tests work

1 similar comment
@modular-magician
Copy link
Collaborator

Tests analytics

Total tests: 63
Passed tests: 62
Skipped tests: 0
Affected tests: 1

Click here to see the affected service packages
  • datalossprevention

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccDataLossPreventionDiscoveryConfig_Update

Get to know how VCR tests work

@modular-magician
Copy link
Collaborator

🔴 Tests failed during RECORDING mode:
TestAccDataLossPreventionDiscoveryConfig_Update [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

@modular-magician
Copy link
Collaborator

🔴 Tests failed during RECORDING mode:
TestAccDataLossPreventionDiscoveryConfig_Update [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

Copy link

github-actions bot commented Jan 9, 2025

@GoogleCloudPlatform/terraform-team @SirGitsalot This PR has been waiting for review for 1 week. Please take a look! Use the label disable-review-reminders to disable these notifications.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants