feat: Allow deployment into an externally managed resource group (#35)

* #34 Added the option for the vault to be deployed into an externally managed resource group.

* #34 added commentary to explain the slightly quirky behaviour of the resource group.

* #34 updated go packages to resolve reported vulnerabilities.

README.md

@@ -16,7 +16,7 @@ The solution consists of a configurable Terraform module which deploys the follo
     * PostgreSQL flexible server
 * Integration of diagnostic settings with Azure Monitor
 
-The resources created by the module reside in their own resource group.
+By default the module will create a dedicated resource group to house the vault, however you can overide this behaviour and use your own resource group managed externally to the module.
 
 See the following key docs for more information:
 

docs/design.md

@@ -22,7 +22,7 @@ The following diagram illustrates the high level architecture:
 
 ### Description
 
-1. The **backup vault** stores the backups of a variety of different Azure resources. A number of **backup instances** are created in the vault, which have a policy applied that defines the configuration for a backup such as the retention period and schedule. The vault is configured as **immutable** and **locked** to enforce tamper proof backups. The **backup vault** resides in it's own isolated **resource group**.
+1. The **backup vault** stores the backups of a variety of different Azure resources. A number of **backup instances** are created in the vault, which have a policy applied that defines the configuration for a backup such as the retention period and schedule. The vault is configured as **immutable** and **locked** to enforce tamper proof backups. The **backup vault** resides in it's own isolated **resource group** (NOTE this behaviour can be overridden if the vault needs to be deployed into an externally managed resource group).
 
 1. **Backup instances** link the resources to be backed up and an associated **backup policy**, and one registered trigger the backup process. The resources directly supported are Azure Blob Storage, Managed Disks, PostgreSQL (single server and flexible server) and AKS instances, although other resources are supported indirectly through Azure Storage (see **point 7** for more details). **Backup instances** are created based on the variables supplied to module, which include configuration and details of the resources that need to be backed up.
 
@@ -45,7 +45,7 @@ The following diagram illustrates the terraform design:
 ### Description
 
 1. The **az-backup** module is essentially everything within the `./infrastructure` directory of this repository. It consists of the following resources:
-   * A **resource group** which will contain _most_ of the other resources in the module.
+   * A **resource group** which will contain the other resources in the module.
    * A **backup vault** within which backup policies and instances are configured..
    * A **role assignment** which provides read access to the vault.
    * A number of **backup modules** which can backup a specific type of resource.

docs/developer-guide.md

@@ -170,12 +170,18 @@ To run the tests, take the following steps:
 
 1. Run the tests
 
-    Run the tests with the following command:
+    Run all the tests with the following command:
 
     ````pwsh
     go test -v -timeout 10m
     ````
 
+    Run a single test with the following command:
+
+    ````pwsh
+    go test -v -timeout 10m -run <TestFunctionName>
+    ````
+
 #### Debugging
 
 To debug the tests in vscode, add the following configuration to launch settings and run the configuration:

docs/usage.md

@@ -10,7 +10,7 @@ The az-backup module resides in the `./infrastructure` sub directory of the repo
 
 In future we will use release tags to ensure consumers can depend on a specific release of the module, however this has not currently been implemented.
 
-The module will create a dedicated resource group to contain the backup vault, therefore the resource group name provided to the module must be unique within the scope of the subscription.
+By default, the module will create a dedicated resource group to contain the backup vault, therefore the resource group name provided to the module must be unique within the scope of the subscription. The creation of a dedicated resource group can be overridden if the vault needs to be deployed into an externally managed resource group.
 
 ## Example
 
@@ -21,6 +21,7 @@ module "my_backup" {
   source                     = "github.com/nhsdigital/az-backup//infrastructure?ref=<version-number>"
   resource_group_name        = "rg-mybackup"
   resource_group_location    = "uksouth"
+  create_resource_group      = true
   backup_vault_name          = "bvault-mybackup"
   backup_vault_redundancy    = "LocallyRedundant"
   backup_vault_immutability  = "Unlocked"
@@ -106,11 +107,12 @@ To deploy the module an Azure identity (typically an app registration with clien
 |------|-------------|-----------|---------|
 | `resource_group_name` | The name of the resource group that is created to contain the vault - this cannot be an existing resource group. | Yes | n/a |
 | `resource_group_location` | The location of the resource group that is created to contain the vault. | No | `uksouth` |
+| `create_resource_group` | States whether a resource group should be created. Setting this to false means the vault will be deployed into an externally managed resource group, the name of which is defined in `resource_group_name`. | No | `true` |
 | `backup_vault_name` | The name of the backup vault. The value supplied will be automatically prefixed with `rg-nhsbackup-`. If more than one az-backup module is created, this value must be unique across them. | Yes | n/a |
 | `backup_vault_redundancy` | The redundancy of the vault, e.g. `GeoRedundant`. [See the following link for the possible values.](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/data_protection_backup_vault#redundancy) | No | `LocallyRedundant` |
 | `backup_vault_immutability` | The immutability of the vault, e.g. `Locked`. [See the following link for the possible values.](https://learn.microsoft.com/en-us/azure/templates/microsoft.dataprotection/backupvaults?pivots=deployment-language-terraform#immutabilitysettings-2) | No | `Disabled` |
 | `log_analytics_workspace_id` | The id of the log analytics workspace that backup telemetry and diagnostics should be sent to. When no value is provided then diagnostics will not be sent anywhere. | No | n/a |
-| `tags` | A map of tags which will be applied to the resource group and backup vault. When no tags are specified then no tags are added. | No | n/a |
+| `tags` | A map of tags which will be applied to the resource group and backup vault. When no tags are specified then no tags are added. NOTE when using an externally managed resource group the tags will not be applied to it (they will still be applied to the backup vault). | No | n/a |
 | `use_extended_retention` | If set to true, then the backup retention periods can be set to anything, otherwise they are limited to 7 days. | No | `false` |
 | `blob_storage_backups` | A map of blob storage backups that should be created. For each backup the following values should be provided: `storage_account_id`, `backup_name` and `retention_period`. When no value is provided then no backups are created. | No | n/a |
 | `blob_storage_backups.storage_account_id` | The id of the storage account that should be backed up. | Yes | n/a |

infrastructure/backup_vault.tf

@@ -1,7 +1,7 @@
 resource "azurerm_data_protection_backup_vault" "backup_vault" {
   name                = var.backup_vault_name
-  resource_group_name = azurerm_resource_group.resource_group.name
-  location            = azurerm_resource_group.resource_group.location
+  resource_group_name = local.resource_group.name
+  location            = local.resource_group.location
   datastore_type      = "VaultStore"
   redundancy          = var.backup_vault_redundancy
   soft_delete         = "Off"

infrastructure/resource_group.tf

@@ -1,5 +1,24 @@
+# The consumer of the module can configure whether the resource group should be created or
+# not. If the resource group should be created, the module will create it. If the resource 
+# group should not be created, the module will look up the existing resource group and 
+# reference it as data. 
+#
+# The local variable resource_group means the resource group - whether data or resource, 
+# can be easily referenced in other parts of the module.
+###########################################################################################
+
 resource "azurerm_resource_group" "resource_group" {
+  count    = var.create_resource_group ? 1 : 0
   location = var.resource_group_location
   name     = var.resource_group_name
   tags     = var.tags
 }
+
+data "azurerm_resource_group" "resource_group" {
+  count = !var.create_resource_group ? 1 : 0
+  name  = var.resource_group_name
+}
+
+locals {
+  resource_group = var.create_resource_group ? azurerm_resource_group.resource_group[0] : data.azurerm_resource_group.resource_group[0]
+}

infrastructure/variables.tf

@@ -14,6 +14,12 @@ variable "resource_group_location" {
   default     = "uksouth"
 }
 
+variable "create_resource_group" {
+  description = "States whether the resource group should be created or not (if not, then it must already exist)"
+  type        = bool
+  default     = true
+}
+
 variable "backup_vault_name" {
   description = "The name of the backup vault"
   type        = string

tests/end-to-end-tests/existing_resource_group_test.go

@@ -0,0 +1,97 @@
+package e2e_tests
+
+import (
+	"fmt"
+	"testing"
+
+	"github.com/Azure/azure-sdk-for-go/sdk/azidentity"
+	"github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/resources/armresources"
+	"github.com/gruntwork-io/terratest/modules/random"
+	"github.com/gruntwork-io/terratest/modules/terraform"
+	test_structure "github.com/gruntwork-io/terratest/modules/test-structure"
+	"github.com/stretchr/testify/assert"
+)
+
+type TestExistingResourceGroupExternalResources struct {
+	ResourceGroup armresources.ResourceGroup
+}
+
+/*
+ * Creates resources which are "external" to the az-backup module.
+ */
+func setupExternalResourcesForExistingResourceGroupTest(t *testing.T, credential *azidentity.ClientSecretCredential, subscriptionID string, resourceGroupName string, resourceGroupLocation string) *TestExistingResourceGroupExternalResources {
+	resourceGroup := CreateResourceGroup(t, credential, subscriptionID, resourceGroupName, resourceGroupLocation)
+
+	externalResources := &TestExistingResourceGroupExternalResources{
+		ResourceGroup: resourceGroup,
+	}
+
+	return externalResources
+}
+
+/*
+ * TestExistingResourceGroup tests the deployment of a backup vault into a pre-existing resource group.
+ */
+func TestExistingResourceGroup(t *testing.T) {
+	t.Parallel()
+
+	environment := GetEnvironmentConfiguration(t)
+	credential := GetAzureCredential(t, environment)
+
+	uniqueId := random.UniqueId()
+	resourceGroupName := fmt.Sprintf("rg-nhsbackup-%s", uniqueId)
+	resourceGroupLocation := "uksouth"
+	backupVaultName := fmt.Sprintf("bvault-nhsbackup-%s", uniqueId)
+
+	externalResources := setupExternalResourcesForExistingResourceGroupTest(t, credential, environment.SubscriptionID, resourceGroupName, resourceGroupLocation)
+
+	// Teardown stage
+	// ...
+
+	defer test_structure.RunTestStage(t, "teardown", func() {
+		terraformOptions := test_structure.LoadTerraformOptions(t, environment.TerraformFolder)
+
+		terraform.Destroy(t, terraformOptions)
+
+		DeleteResourceGroup(t, credential, environment.SubscriptionID, *externalResources.ResourceGroup.Name)
+	})
+
+	// Setup stage
+	// ...
+
+	test_structure.RunTestStage(t, "setup", func() {
+		terraformOptions := &terraform.Options{
+			TerraformDir: environment.TerraformFolder,
+
+			Vars: map[string]interface{}{
+				"resource_group_name":     resourceGroupName,
+				"resource_group_location": resourceGroupLocation,
+				"create_resource_group":   false,
+				"backup_vault_name":       backupVaultName,
+			},
+
+			BackendConfig: map[string]interface{}{
+				"resource_group_name":  environment.TerraformStateResourceGroup,
+				"storage_account_name": environment.TerraformStateStorageAccount,
+				"container_name":       environment.TerraformStateContainer,
+				"key":                  backupVaultName + ".tfstate",
+			},
+		}
+
+		// Save options for later test stages
+		test_structure.SaveTerraformOptions(t, environment.TerraformFolder, terraformOptions)
+
+		terraform.InitAndApply(t, terraformOptions)
+	})
+
+	// Validate stage
+	// ...
+
+	test_structure.RunTestStage(t, "validate", func() {
+		// Validate resource group
+		resourceGroup := GetResourceGroup(t, environment.SubscriptionID, credential, resourceGroupName)
+		assert.NotNil(t, resourceGroup, "Resource group does not exist")
+		assert.Equal(t, resourceGroupName, *resourceGroup.Name, "Resource group name does not match")
+		assert.Equal(t, resourceGroupLocation, *resourceGroup.Location, "Resource group location does not match")
+	})
+}