From c2659c5320e3594bfbfc4657f92ce25718bef3a8 Mon Sep 17 00:00:00 2001 From: John Skiles Skinner Date: Thu, 1 Aug 2024 09:28:42 -0700 Subject: [PATCH 01/10] Link to yesterday's video --- terraform/README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/terraform/README.md b/terraform/README.md index bbb63424a..223adc451 100644 --- a/terraform/README.md +++ b/terraform/README.md @@ -7,6 +7,8 @@ This directory holds the Terraform modules for maintaining Notify.gov's API infr The Admin app repo [has its own terraform directory](https://github.com/GSA/notifications-admin/tree/main/terraform) but a lot of the below instructions apply to both apps. +:tv: [Video introduction](https://drive.google.com/file/d/13SR3M8IowYBa4Wp_YEcuAURZ74EcCYoc/) to Notify infrastructure + ## Retrieving existing bucket credentials :green_book: New developers start here! From 7344fbf78b57c82d2ef2cb9c4714681410f6c280 Mon Sep 17 00:00:00 2001 From: John Skiles Skinner Date: Thu, 1 Aug 2024 09:34:53 -0700 Subject: [PATCH 02/10] Initial setup instructions no longer needed --- terraform/README.md | 37 ------------------------------------- 1 file changed, 37 deletions(-) diff --git a/terraform/README.md b/terraform/README.md index 223adc451..4ec3d1a39 100644 --- a/terraform/README.md +++ b/terraform/README.md @@ -33,43 +33,6 @@ Assuming [initial setup](#initial-setup) is complete — which it should be These credentials will allow Terraform to access the AWS/Cloud.gov bucket in which developers share Terraform state files. Now you are ready to develop Terraform using the [Workflow for deployed environments](#workflow-for-deployed-environments). -## Initial setup - -These instructions were used for deploying the project for the first time, years ago. We should not have to perform these steps again. They are provided here for reference. - -1. Manually run the bootstrap module following instructions under [Terraform State Credentials](#terraform-state-credentials) -1. Setup CI/CD Pipeline to run Terraform - 1. Copy bootstrap credentials to your CI/CD secrets using the instructions in the base README - 1. Create a cloud.gov SpaceDeployer by following the instructions under [SpaceDeployers](#spacedeployers) - 1. Copy SpaceDeployer credentials to your CI/CD secrets using the instructions in the base README -1. Manually Running Terraform - 1. Follow instructions under [Workflow for deployed environments](#workflow-for-deployed-environments) to create your infrastructure - -### Terraform state credentials - -The bootstrap module is used to create an s3 bucket for later terraform runs to store their state in. (If the bucket is already created, you should [Use bootstrap credentials](#use-bootstrap-credentials)) - -#### Bootstrapping the state storage s3 buckets for the first time - -1. Within the `bootstrap` directory, run `terraform init` -1. Run `./run.sh plan` to verify that the changes are what you expect -1. Run `./run.sh apply` to set up the bucket -1. Follow instructions under [Use bootstrap credentials](#use-bootstrap-credentials) -1. Ensure that `import.sh` includes a line and correct IDs for any resources created -1. Run `./teardown_creds.sh` to remove the space deployer account used to create the s3 bucket -1. Copy `bucket` from `bucket_credentials` output to the backend block of `staging/providers.tf` and `production/providers.tf` - -#### To make changes to the bootstrap module - -*This should not be necessary in most cases* - -1. Run `terraform init` -1. If you don't have terraform state locally: - 1. run `./import.sh` - 1. optionally run `./run.sh apply` to include the existing outputs in the state file -1. Make your changes -1. Continue from step 2 of the boostrapping instructions - ## SpaceDeployers A [SpaceDeployer](https://cloud.gov/docs/services/cloud-gov-service-account/) account is required to run terraform or From 73ba76fd2af3946a96ebb0c55323694d1e966ba8 Mon Sep 17 00:00:00 2001 From: John Skiles Skinner Date: Thu, 1 Aug 2024 09:38:01 -0700 Subject: [PATCH 03/10] Move SpaceDeployers section down --- terraform/README.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/terraform/README.md b/terraform/README.md index 4ec3d1a39..3f2414410 100644 --- a/terraform/README.md +++ b/terraform/README.md @@ -33,15 +33,6 @@ Assuming [initial setup](#initial-setup) is complete — which it should be These credentials will allow Terraform to access the AWS/Cloud.gov bucket in which developers share Terraform state files. Now you are ready to develop Terraform using the [Workflow for deployed environments](#workflow-for-deployed-environments). -## SpaceDeployers - -A [SpaceDeployer](https://cloud.gov/docs/services/cloud-gov-service-account/) account is required to run terraform or -deploy the application from the CI/CD pipeline. Create a new account by running: - -`./create_service_account.sh -s -u ` - -SpaceDeployers are also needed to run Terraform locally — they fill user and password input variables (via `deployers` within `main.tf`) that some of our Terraform modules require when they start running. Using a SpaceDeployer account locally is covered in [the next section](#workflow-for-deployed-environments). - ## Workflow for deployed environments These are the steps for developing Terraform code for our deployed environment modules (`sandbox`, `demo`, `staging` and `production`) locally on your laptop. Or for setting up a new deployment environment, or otherwise for running Terraform manually in any module that uses remote state. You don't need to do all this to run code in the `development` module, because it is not a deployed environment and it does not use remote state. @@ -111,6 +102,15 @@ These steps assume shared [Terraform state credentials](#terraform-state-credent Optionally, you can also `rm secrets.auto.tfvars` +## SpaceDeployers + +A [SpaceDeployer](https://cloud.gov/docs/services/cloud-gov-service-account/) account is required to run terraform or +deploy the application from the CI/CD pipeline. Create a new account by running: + +`./create_service_account.sh -s -u ` + +SpaceDeployers are also needed to run Terraform locally — they fill user and password input variables (via `deployers` within `main.tf`) that some of our Terraform modules require when they start running. Using a SpaceDeployer account locally is covered in [the next section](#workflow-for-deployed-environments). + ## Structure The `terraform` directory contains sub-directories (`staging`, `production`, etc.) named for deployment environments. Each of these is a *module*, which is just Terraform's word for a directory with some .tf files in it. Each module governs the infrastructure of the environment for which it is named. This directory structure forms "[bulkheads](https://blog.gruntwork.io/how-to-manage-terraform-state-28f5697e68fa)" which isolate Terraform commands to a single environment, limiting accidental damage. From d8238852ee5e32d0a097dd44ee0ad9291cedd204 Mon Sep 17 00:00:00 2001 From: John Skiles Skinner Date: Thu, 1 Aug 2024 10:26:23 -0700 Subject: [PATCH 04/10] Experiment with relative link, image --- terraform/README.md | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/terraform/README.md b/terraform/README.md index 3f2414410..b3feb3e53 100644 --- a/terraform/README.md +++ b/terraform/README.md @@ -1,5 +1,7 @@ # Terraform + + This directory holds the Terraform modules for maintaining Notify.gov's API infrastructure. You might want to: * [Set up](#retrieving-existing-bucket-credentials) the Sandbox and develop Terraform, * [Learn](#structure) about the directory structure, or @@ -75,7 +77,7 @@ These steps assume shared [Terraform state credentials](#terraform-state-credent This will show you any pending changes that Terraform is ready to make. - :pencil: Now is the time to write any HCL code you are planning to write, re-running `terraform plan` to confirm that the code works as you develop. Keep in mind that any changes to the codebase that you commit will be run by the CI/CD pipeline. + :pencil: Now is the time to write any HCL code (aka Terraform code) you are planning to write, re-running `terraform plan` to confirm that the code works as you develop. Keep in mind that any changes to the codebase that you commit will be run by the CI/CD pipeline. 1. **Only if it is safe to do so**, apply your changes. @@ -102,6 +104,13 @@ These steps assume shared [Terraform state credentials](#terraform-state-credent Optionally, you can also `rm secrets.auto.tfvars` +## Maintenance + +### Version upgrade checklist + +* Cloud Foundry Terraform plugin in every module, [here for example](sandbox/providers.tf#L6) + + ## SpaceDeployers A [SpaceDeployer](https://cloud.gov/docs/services/cloud-gov-service-account/) account is required to run terraform or @@ -109,7 +118,7 @@ deploy the application from the CI/CD pipeline. Create a new account by running: `./create_service_account.sh -s -u ` -SpaceDeployers are also needed to run Terraform locally — they fill user and password input variables (via `deployers` within `main.tf`) that some of our Terraform modules require when they start running. Using a SpaceDeployer account locally is covered in [the next section](#workflow-for-deployed-environments). +SpaceDeployers are also needed to run Terraform locally — they fill user and password input variables (via `deployers` within `main.tf`) that some of our Terraform modules require when they start running. Using a SpaceDeployer account locally is covered in [Workflow for deployed environments](#workflow-for-deployed-environments). ## Structure From ac577eff625f0ac52fc90ca651af4c25bee354d4 Mon Sep 17 00:00:00 2001 From: John Skiles Skinner Date: Thu, 1 Aug 2024 11:05:37 -0700 Subject: [PATCH 05/10] Add items to the version upgrade checklist --- terraform/README.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/terraform/README.md b/terraform/README.md index b3feb3e53..cb1605ae2 100644 --- a/terraform/README.md +++ b/terraform/README.md @@ -1,6 +1,6 @@ # Terraform - +illustration of space exploration This directory holds the Terraform modules for maintaining Notify.gov's API infrastructure. You might want to: * [Set up](#retrieving-existing-bucket-credentials) the Sandbox and develop Terraform, @@ -108,7 +108,11 @@ These steps assume shared [Terraform state credentials](#terraform-state-credent ### Version upgrade checklist -* Cloud Foundry Terraform plugin in every module, [here for example](sandbox/providers.tf#L6) +* Cloud Foundry Terraform plugin in every module in the API and Admin apps, [here for example](sandbox/providers.tf#L6). +* The [terraform-cloudgov module](https://github.com/GSA-TTS/terraform-cloudgov/), the version of which is referred to serveral times in most modules, [here for example](sandbox/main.tf#16). +* Cloud Service Broker (CSB) version in [the SMS](https://github.com/GSA/usnotify-ssb/blob/main/app-setup-sms.sh) and [the SMTP](https://github.com/GSA/usnotify-ssb/blob/main/app-setup-smtp.sh) download scripts of the usnotify-ssb repo. +* SMS and SMTP brokerpak versions, also in the download scripts of the usnotify-ssb repo, along with with the [SMTP brokerpak project](https://github.com/GSA-TTS/datagov-brokerpak-smtp) itself. +* The version of Redis used in deployed environment modules, [here for example](sandbox/main.tf#33). To upgrade, the resource must be destroyed and replaced. The versions supported are limited by Cloud.gov. ## SpaceDeployers From 0fff92b0b084426f3eff28e7c0f8849dc5c39f78 Mon Sep 17 00:00:00 2001 From: John Skiles Skinner Date: Thu, 1 Aug 2024 11:07:51 -0700 Subject: [PATCH 06/10] Add Maintenance to the table of contents --- terraform/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/terraform/README.md b/terraform/README.md index cb1605ae2..e9ee0c37d 100644 --- a/terraform/README.md +++ b/terraform/README.md @@ -4,6 +4,7 @@ This directory holds the Terraform modules for maintaining Notify.gov's API infrastructure. You might want to: * [Set up](#retrieving-existing-bucket-credentials) the Sandbox and develop Terraform, +* [Maintain](#maintenance) software versions or CI/CD, * [Learn](#structure) about the directory structure, or * [Troubleshoot](#troubleshooting) error messages From 0f0e550e68c6b870371ecab90c05115b0aca21d9 Mon Sep 17 00:00:00 2001 From: John Skiles Skinner Date: Thu, 1 Aug 2024 11:17:32 -0700 Subject: [PATCH 07/10] Put SpaceDeployers section within Maintenance --- terraform/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/terraform/README.md b/terraform/README.md index e9ee0c37d..90160066e 100644 --- a/terraform/README.md +++ b/terraform/README.md @@ -116,10 +116,10 @@ These steps assume shared [Terraform state credentials](#terraform-state-credent * The version of Redis used in deployed environment modules, [here for example](sandbox/main.tf#33). To upgrade, the resource must be destroyed and replaced. The versions supported are limited by Cloud.gov. -## SpaceDeployers +### SpaceDeployers A [SpaceDeployer](https://cloud.gov/docs/services/cloud-gov-service-account/) account is required to run terraform or -deploy the application from the CI/CD pipeline. Create a new account by running: +deploy the application from the CI/CD pipeline. During CI/CD maintenance you might need to create a new account: `./create_service_account.sh -s -u ` From c8a7e6a5ff8921e0562ebe342e5651004c12f689 Mon Sep 17 00:00:00 2001 From: John Skiles Skinner Date: Thu, 1 Aug 2024 11:31:54 -0700 Subject: [PATCH 08/10] Add intro to the upgrade checklist, and tf version note --- terraform/README.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/terraform/README.md b/terraform/README.md index 90160066e..c9cbf285f 100644 --- a/terraform/README.md +++ b/terraform/README.md @@ -109,12 +109,13 @@ These steps assume shared [Terraform state credentials](#terraform-state-credent ### Version upgrade checklist +These version numbers are hardcoded in Terraform or shell scripts. We need to periodically check them for needed updates. Our [video introduction](https://drive.google.com/file/d/13SR3M8IowYBa4Wp_YEcuAURZ74EcCYoc/) also talks about some of these. * Cloud Foundry Terraform plugin in every module in the API and Admin apps, [here for example](sandbox/providers.tf#L6). * The [terraform-cloudgov module](https://github.com/GSA-TTS/terraform-cloudgov/), the version of which is referred to serveral times in most modules, [here for example](sandbox/main.tf#16). * Cloud Service Broker (CSB) version in [the SMS](https://github.com/GSA/usnotify-ssb/blob/main/app-setup-sms.sh) and [the SMTP](https://github.com/GSA/usnotify-ssb/blob/main/app-setup-smtp.sh) download scripts of the usnotify-ssb repo. -* SMS and SMTP brokerpak versions, also in the download scripts of the usnotify-ssb repo, along with with the [SMTP brokerpak project](https://github.com/GSA-TTS/datagov-brokerpak-smtp) itself. +* SMS and SMTP brokerpak versions, also in the download scripts of the usnotify-ssb repo. (And we may have to help maintain the [SMTP brokerpak project](https://github.com/GSA-TTS/datagov-brokerpak-smtp) itself.) * The version of Redis used in deployed environment modules, [here for example](sandbox/main.tf#33). To upgrade, the resource must be destroyed and replaced. The versions supported are limited by Cloud.gov. - +* A required minimum version of Terraform is noted in every providers.tf file, [here for example](sandbox/providers.tf#L2). It would be best to keep it in sync with the version used by our CI/CD deployment pipeline. But, it does not need to be updated with every new Terraform release. ### SpaceDeployers From d8153eb3ad805e35f4964ec723516876f7afe33e Mon Sep 17 00:00:00 2001 From: John Skiles Skinner Date: Thu, 1 Aug 2024 11:36:02 -0700 Subject: [PATCH 09/10] Move the video link below the upgrade checklist --- terraform/README.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/terraform/README.md b/terraform/README.md index c9cbf285f..e9ffe4d13 100644 --- a/terraform/README.md +++ b/terraform/README.md @@ -109,7 +109,8 @@ These steps assume shared [Terraform state credentials](#terraform-state-credent ### Version upgrade checklist -These version numbers are hardcoded in Terraform or shell scripts. We need to periodically check them for needed updates. Our [video introduction](https://drive.google.com/file/d/13SR3M8IowYBa4Wp_YEcuAURZ74EcCYoc/) also talks about some of these. +These version numbers are hardcoded in Terraform or shell scripts. We should periodically check them for upgrades. + * Cloud Foundry Terraform plugin in every module in the API and Admin apps, [here for example](sandbox/providers.tf#L6). * The [terraform-cloudgov module](https://github.com/GSA-TTS/terraform-cloudgov/), the version of which is referred to serveral times in most modules, [here for example](sandbox/main.tf#16). * Cloud Service Broker (CSB) version in [the SMS](https://github.com/GSA/usnotify-ssb/blob/main/app-setup-sms.sh) and [the SMTP](https://github.com/GSA/usnotify-ssb/blob/main/app-setup-smtp.sh) download scripts of the usnotify-ssb repo. @@ -117,6 +118,8 @@ These version numbers are hardcoded in Terraform or shell scripts. We need to pe * The version of Redis used in deployed environment modules, [here for example](sandbox/main.tf#33). To upgrade, the resource must be destroyed and replaced. The versions supported are limited by Cloud.gov. * A required minimum version of Terraform is noted in every providers.tf file, [here for example](sandbox/providers.tf#L2). It would be best to keep it in sync with the version used by our CI/CD deployment pipeline. But, it does not need to be updated with every new Terraform release. +:tv: Some of these version upgrades are discussed in our [video introduction](https://drive.google.com/file/d/13SR3M8IowYBa4Wp_YEcuAURZ74EcCYoc/). + ### SpaceDeployers A [SpaceDeployer](https://cloud.gov/docs/services/cloud-gov-service-account/) account is required to run terraform or From 56b7d025937b94235e68e774380aefe671b8e972 Mon Sep 17 00:00:00 2001 From: John Skiles Skinner Date: Thu, 1 Aug 2024 11:40:06 -0700 Subject: [PATCH 10/10] Fix link line numbers --- terraform/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/terraform/README.md b/terraform/README.md index e9ffe4d13..24e9a55f3 100644 --- a/terraform/README.md +++ b/terraform/README.md @@ -112,10 +112,10 @@ These steps assume shared [Terraform state credentials](#terraform-state-credent These version numbers are hardcoded in Terraform or shell scripts. We should periodically check them for upgrades. * Cloud Foundry Terraform plugin in every module in the API and Admin apps, [here for example](sandbox/providers.tf#L6). -* The [terraform-cloudgov module](https://github.com/GSA-TTS/terraform-cloudgov/), the version of which is referred to serveral times in most modules, [here for example](sandbox/main.tf#16). +* The [terraform-cloudgov module](https://github.com/GSA-TTS/terraform-cloudgov/), the version of which is referred to serveral times in most modules, [here for example](sandbox/main.tf#L16). * Cloud Service Broker (CSB) version in [the SMS](https://github.com/GSA/usnotify-ssb/blob/main/app-setup-sms.sh) and [the SMTP](https://github.com/GSA/usnotify-ssb/blob/main/app-setup-smtp.sh) download scripts of the usnotify-ssb repo. * SMS and SMTP brokerpak versions, also in the download scripts of the usnotify-ssb repo. (And we may have to help maintain the [SMTP brokerpak project](https://github.com/GSA-TTS/datagov-brokerpak-smtp) itself.) -* The version of Redis used in deployed environment modules, [here for example](sandbox/main.tf#33). To upgrade, the resource must be destroyed and replaced. The versions supported are limited by Cloud.gov. +* The version of Redis used in deployed environment modules, [here for example](sandbox/main.tf#L33). To upgrade, the resource must be destroyed and replaced. The versions supported are limited by Cloud.gov. * A required minimum version of Terraform is noted in every providers.tf file, [here for example](sandbox/providers.tf#L2). It would be best to keep it in sync with the version used by our CI/CD deployment pipeline. But, it does not need to be updated with every new Terraform release. :tv: Some of these version upgrades are discussed in our [video introduction](https://drive.google.com/file/d/13SR3M8IowYBa4Wp_YEcuAURZ74EcCYoc/).