Skip to content

Commit

Permalink
Merge branch 'jng/CDI-3508' of github.com:chanzuckerberg/cztack into …
Browse files Browse the repository at this point in the history
…jng/CDI-3508
  • Loading branch information
jayengee committed Nov 6, 2024
2 parents 890fd54 + 08aaaf0 commit 3e0cdc3
Show file tree
Hide file tree
Showing 13 changed files with 336 additions and 31 deletions.
32 changes: 21 additions & 11 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,20 +11,30 @@ jobs:
run: echo "::set-output name=matrix::$(ls -d */|sed -e 's/\///'|grep -v 'bless-ca\|scripts'|jq -cnR '[inputs | select(length>0)]')"
lint:
name: lint
runs-on: ubuntu-latest
runs-on: [ARM64, self-hosted, Linux]
steps:
- uses: actions/checkout@v3
- uses: hashicorp/setup-terraform@v1
- name: Generate token
id: generate_token
uses: tibdex/github-app-token@v2
with:
terraform_version: 0.14.5
terraform_wrapper: "false"
- name: setup
run: make setup
- uses: golangci/golangci-lint-action@v3
app_id: ${{ secrets.CZI_GITHUB_HELPER_APP_ID }}
private_key: ${{ secrets.CZI_GITHUB_HELPER_PK }}
- uses: actions/checkout@v4
with:
token: ${{ steps.generate_token.outputs.token }}
ref: ${{ github.event.pull_request.head.ref }}
- uses: actions/setup-go@v5
with:
go-version-file: go.mod
cache: true
cache-dependency-path: |
go.sum
- name: golangci-lint
uses: golangci/golangci-lint-action@v6
with:
version: v1.56.2
args: --timeout=5m
github-token: ${{ secrets.GITHUB_TOKEN }}
version: v1.60.3
github-token: ${{ secrets.GITHUB_TOKEN }}

test:
name: test ${{ matrix.module }}
needs: get-modules
Expand Down
8 changes: 7 additions & 1 deletion .golangci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,10 @@ linters:
- gofmt
- whitespace
- unparam
- scopelint
- forbidigo
- gocritic

output:
formats:
- format: tab
show-stats: true
52 changes: 52 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,57 @@
# Changelog

## [0.87.2](https://github.com/chanzuckerberg/cztack/compare/v0.87.1...v0.87.2) (2024-11-01)


### Bug Fixes

* concat two list of strings then turn it into set ([#674](https://github.com/chanzuckerberg/cztack/issues/674)) ([8cd5290](https://github.com/chanzuckerberg/cztack/commit/8cd52908d7b812141cfa7b7298bd785f89370c27))

## [0.87.1](https://github.com/chanzuckerberg/cztack/compare/v0.87.0...v0.87.1) (2024-10-31)


### Bug Fixes

* update s3 bucket name typo ([#672](https://github.com/chanzuckerberg/cztack/issues/672)) ([22affc7](https://github.com/chanzuckerberg/cztack/commit/22affc7de02f4c9a222462827aaeeb3291c64a8b))

## [0.87.0](https://github.com/chanzuckerberg/cztack/compare/v0.86.1...v0.87.0) (2024-10-31)


### Features

* biohub s3 readonly role ([#669](https://github.com/chanzuckerberg/cztack/issues/669)) ([d64e240](https://github.com/chanzuckerberg/cztack/commit/d64e240f310286aa5b75c139ab235bbbbbf77b7f))


### Bug Fixes

* add oidc input to assume role policy ([#671](https://github.com/chanzuckerberg/cztack/issues/671)) ([4d41154](https://github.com/chanzuckerberg/cztack/commit/4d41154cc90fa37c729e2b1ae9ada3035d214ff4))

## [0.86.1](https://github.com/chanzuckerberg/cztack/compare/v0.86.0...v0.86.1) (2024-10-25)


### Bug Fixes

* Ensure Databricks external location role exists before making it self-referential ([#667](https://github.com/chanzuckerberg/cztack/issues/667)) ([6834b5b](https://github.com/chanzuckerberg/cztack/commit/6834b5bd2bcecdb8231b054c6fb4b37105e28aaa))

## [0.86.0](https://github.com/chanzuckerberg/cztack/compare/v0.85.0...v0.86.0) (2024-10-24)


### Features

* update go lint ([#665](https://github.com/chanzuckerberg/cztack/issues/665)) ([eb4a894](https://github.com/chanzuckerberg/cztack/commit/eb4a89415b1e61d66bba5fafac44695b4a11837e))


### Bug Fixes

* databricks-catalog-external-location - Make role self-assuming ([#664](https://github.com/chanzuckerberg/cztack/issues/664)) ([7ff6b93](https://github.com/chanzuckerberg/cztack/commit/7ff6b93aa8aaa1ed843079c75d26bbd9861e4806))

## [0.85.0](https://github.com/chanzuckerberg/cztack/compare/v0.84.1...v0.85.0) (2024-10-17)


### Features

* trigger release please ([#662](https://github.com/chanzuckerberg/cztack/issues/662)) ([9d80dd2](https://github.com/chanzuckerberg/cztack/commit/9d80dd27581444dbd336edef281777d335a0af55))

## [0.84.1](https://github.com/chanzuckerberg/cztack/compare/v0.84.0...v0.84.1) (2024-10-17)


Expand Down
51 changes: 51 additions & 0 deletions aws-iam-role-s3-readonly/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# AWS IAM Role S3 Readonly

This module will create a role which is granted readonly control over AWS S3 buckets.

<!-- START -->
## Requirements

No requirements.

## Providers

| Name | Version |
|------|---------|
| <a name="provider_aws"></a> [aws](#provider\_aws) | n/a |

## Modules

| Name | Source | Version |
|------|--------|---------|
| <a name="module_assume_role_policy"></a> [assume\_role\_policy](#module\_assume\_role\_policy) | ../aws-assume-role-policy | n/a |

## Resources

| Name | Type |
|------|------|
| [aws_iam_policy.s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_role.role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource |
| [aws_iam_role_policy_attachment.s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_iam_policy_document.s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |

## Inputs

| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| <a name="input_env"></a> [env](#input\_env) | Env for tagging and naming. See [doc](../README.md#consistent-tagging) | `string` | n/a | yes |
| <a name="input_iam_path"></a> [iam\_path](#input\_iam\_path) | n/a | `string` | `"/"` | no |
| <a name="input_owner"></a> [owner](#input\_owner) | Owner for tagging and naming. See [doc](../README.md#consistent-tagging) | `string` | n/a | yes |
| <a name="input_project"></a> [project](#input\_project) | Project for tagging and naming. See [doc](../README.md#consistent-tagging) | `string` | n/a | yes |
| <a name="input_role_name"></a> [role\_name](#input\_role\_name) | Name of the role to create | `string` | n/a | yes |
| <a name="input_s3_bucket_names"></a> [s3\_bucket\_prefixes](#input\_s3\_bucket\_prefixes) | Limits role permissions to buckets with specific prefixes. Empty for all buckets. | `list(any)` | <pre>[<br> ""<br>]</pre> | no |
| <a name="input_saml_idp_arns"></a> [saml\_idp\_arns](#input\_saml\_idp\_arns) | The AWS SAML IDP arns to establish a trust relationship. Ignored if empty or not provided. | `set(string)` | `[]` | no |
| <a name="input_service"></a> [service](#input\_service) | Service for tagging and naming. See [doc](../README.md#consistent-tagging) | `string` | n/a | yes |
| <a name="input_source_account_ids"></a> [source\_account\_ids](#input\_source\_account\_ids) | The source AWS account IDs to establish a trust relationship. Ignored if empty or not provided. | `set(string)` | `[]` | no |

## Outputs

| Name | Description |
|------|-------------|
| <a name="output_role_arn"></a> [role\_arn](#output\_role\_arn) | n/a |
| <a name="output_role_name"></a> [role\_name](#output\_role\_name) | n/a |
<!-- END -->
68 changes: 68 additions & 0 deletions aws-iam-role-s3-readonly/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
locals {
tags = {
env = var.env
owner = var.owner
service = var.service
project = var.project
}
}

module "assume_role_policy" {
source = "../aws-assume-role-policy"
source_account_ids = var.source_account_ids
saml_idp_arns = var.saml_idp_arns
oidc = var.oidc
env = var.env
owner = var.owner
service = var.service
project = var.project
}

resource "aws_iam_role" "role" {
name = var.role_name
assume_role_policy = module.assume_role_policy.json
tags = local.tags
}

data "aws_iam_policy_document" "s3-bucket-readonly" {
statement {
sid = "GetFiles"
effect = "Allow"
actions = [
"s3:GetObject",
"s3:GetObjectAcl",
"s3:GetObjectVersionAcl",
"s3:GetObjectVersion"
]
resources = formatlist("arn:aws:s3:::%s*/*", var.s3_bucket_names)
}
statement {
sid = "ListBucket"
effect = "Allow"
actions = [
"s3:ListBucket",
"s3:GetBucketLocation",
"s3:ListBucketMultipartUploads"
]
resources = formatlist("arn:aws:s3:::%s*/*", var.s3_bucket_names)
}
statement {
sid = "ShowAllowedBuckets"
effect = "Allow"
actions = [
"s3:ListAllMyBuckets"
]
resources = toset(concat(formatlist("arn:aws:s3:::%s", var.s3_bucket_names), formatlist("arn:aws:s3:::%s/*", var.s3_bucket_names)))
}
}

resource "aws_iam_policy" "s3" {
name = "${var.role_name}-s3"
description = "Provide access to s3 resources for a distribution ${var.role_name}"
policy = data.aws_iam_policy_document.s3-bucket-readonly.json
}

resource "aws_iam_role_policy_attachment" "s3" {
role = aws_iam_role.role.name
policy_arn = aws_iam_policy.s3.arn
}
42 changes: 42 additions & 0 deletions aws-iam-role-s3-readonly/module_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
package test

import (
"fmt"
"testing"

"github.com/chanzuckerberg/go-misc/tftest"
"github.com/gruntwork-io/terratest/modules/random"
"github.com/gruntwork-io/terratest/modules/terraform"
"github.com/stretchr/testify/require"
)

func TestAWSIAMRoleS3Readonly(t *testing.T) {
test := tftest.Test{
Setup: func(t *testing.T) *terraform.Options {
curAcct := tftest.AWSCurrentAccountID(t)

return tftest.Options(
tftest.IAMRegion,

map[string]interface{}{
"role_name": random.UniqueId(),
"iam_path": fmt.Sprintf("/%s/", random.UniqueId()),
"source_account_ids": []string{curAcct},
},
)
},
Validate: func(t *testing.T, options *terraform.Options) {
r := require.New(t)
region := options.EnvVars["AWS_DEFAULT_REGION"]
r.NotEmpty(region)
outputs := terraform.OutputAll(t, options)
r.NotEmpty(outputs)
roleName := outputs["role_name"].(string)
r.NotEmpty(roleName)
roleARN := outputs["role_arn"].(string)
r.NotEmpty(roleARN)
},
}

test.Run(t)
}
7 changes: 7 additions & 0 deletions aws-iam-role-s3-readonly/outputs.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
output "role_name" {
value = aws_iam_role.role.name
}

output "role_arn" {
value = aws_iam_role.role.arn
}
63 changes: 63 additions & 0 deletions aws-iam-role-s3-readonly/variables.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
variable "role_name" {
type = string
description = "Name of the role to create"
}

variable "s3_bucket_names" {
type = set(string)
description = "Limits role permissions to buckets with specific prefixes. Empty for all buckets."

default = [
"",
]
}

variable "iam_path" {
type = string
default = "/"
}

variable "source_account_ids" {
type = set(string)
default = []
description = "The source AWS account IDs to establish a trust relationship. Ignored if empty or not provided."
}

variable "saml_idp_arns" {
type = set(string)
default = []
description = "The AWS SAML IDP arns to establish a trust relationship. Ignored if empty or not provided."
}

variable "oidc" {
type = list(object(
{
idp_arn : string, # the AWS IAM IDP arn
client_ids : list(string), # a list of oidc client ids
provider : string # your provider url, such as foo.okta.com
}
))

default = []
description = "A list of AWS OIDC IDPs to establish a trust relationship for this role."
}

variable "project" {
type = string
description = "Project for tagging and naming. See [doc](../README.md#consistent-tagging)"
}

variable "env" {
type = string
description = "Env for tagging and naming. See [doc](../README.md#consistent-tagging)"
}

variable "service" {
type = string
description = "Service for tagging and naming. See [doc](../README.md#consistent-tagging)"
}

variable "owner" {
type = string
description = "Owner for tagging and naming. See [doc](../README.md#consistent-tagging)"
}
2 changes: 1 addition & 1 deletion aws-lambda-edge-add-security-headers/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ module "lambda" {
filename = data.archive_file.lambda.output_path
source_code_hash = data.archive_file.lambda.output_base64sha256
handler = "index.handler"
runtime = "nodejs14.x"
runtime = "nodejs18.x"
at_edge = true
publish_lambda = true
log_retention_in_days = var.lambda_cloudwatch_log_retention_in_days
Expand Down
7 changes: 4 additions & 3 deletions aws-s3-public-bucket/module_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -111,9 +111,10 @@ func TestPublicBucketDefaults(t *testing.T) {
}

for _, test := range sims {
resp := tftest.S3SimulateRequest(t, region, test.action, test.arn, bucketPolicy, test.secureTransport)
fmt.Println("Testing ", test.action, " with https enabled=", test.secureTransport)
r.Equal(test.result, *resp.EvalDecision)
t.Run(fmt.Sprintf("Testing %s with https enabled=%t", test.action, test.secureTransport), func(t *testing.T) {
resp := tftest.S3SimulateRequest(t, region, test.action, test.arn, bucketPolicy, test.secureTransport)
r.Equal(test.result, *resp.EvalDecision)
})
}
},
}
Expand Down
2 changes: 1 addition & 1 deletion bless-ca/test/module_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ func TestBlessCAInitAndApply(t *testing.T) {
return tftest.Options(
region,
map[string]interface{}{
//test only
// test only
"region": region,
"bless_provider_aws_profile": tftest.EnvVar(tftest.EnvAWSProfile),
"test_user_name": fmt.Sprintf("bless-%s", tftest.UniqueID()),
Expand Down
Loading

0 comments on commit 3e0cdc3

Please sign in to comment.