diff --git a/.github/workflows/update-CloudformationTemplate-auto-set-fsxn-auto-grow.yml b/.github/workflows/update-CloudformationTemplate-auto-set-fsxn-auto-grow.yml new file mode 100644 index 0000000..5416969 --- /dev/null +++ b/.github/workflows/update-CloudformationTemplate-auto-set-fsxn-auto-grow.yml @@ -0,0 +1,37 @@ +--- +# Copyright (c) NetApp, Inc. +# SPDX-License-Identifier: Apache-2.0 + +name: "Update Cloudformation Template" + +on: + pull_request: + paths: + - 'Management-Utilities/auto_set_fsxn_auto_grow/set_fsxn_volume_auto_grow.py' + push: + paths: + - 'Management-Utilities/auto_set_fsxn_auto_grow/set_fsxn_volume_auto_grow.py' + branches: + - main + +jobs: + update-Cloudformation-Template: + runs-on: ubuntu-latest + permissions: + # Give the default GITHUB_TOKEN write permission to commit and push the + # added or changed files to the repository. + contents: write + + steps: + - name: Checkout pull request + uses: actions/checkout@v4 + with: + ref: ${{ github.event.pull_request.head.ref }} + + - name: Update the Cloudformation Template + shell: bash + working-directory: Management-Utilities/auto_set_fsxn_auto_grow + run: ./update_auto_set_fsxn_auto_grow_CF_Template + + - name: Commit the changes + uses: stefanzweifel/git-auto-commit-action@v5 diff --git a/Management-Utilities/auto_set_fsxn_auto_grow/README.md b/Management-Utilities/auto_set_fsxn_auto_grow/README.md index f7d0233..982bf01 100644 --- a/Management-Utilities/auto_set_fsxn_auto_grow/README.md +++ b/Management-Utilities/auto_set_fsxn_auto_grow/README.md @@ -2,57 +2,158 @@ ## Introduction This sample shows one way to mitigate the issue of not being able to set the auto size mode -on an FSxN volume when creating it from the AWS console or API. It does this by providing +on an FSx for ONTAP volume when creating it from the AWS console or API. It does this by providing a Lambda function that will set the mode for you, and instructions on how to set up a CloudWatch event to trigger the Lambda function whenever a volume is created. With this combination it ensures that all volumes are effectively created with the auto size mode set up the way you want for all volumes. -## Set Up -There are just a few things you have to do to set this up: +Note that a CloudWatch event is not created when a volume is created directly from the +ONTAP side, either using the ONTAP CLI, System Manager, or REST API. So, it is assumed +if you are creating them that way, that you will set them with the auto size mode set +the way you want. -### Create secrets in AWS Secrets Manager +Since the Lambda function has to communicate with the FSx for ONTAP management +endpoint, it has to run within a VPC that has that connectivity. Because of the way +AWS allows a Lambda function to run within a VPC, it will not have access to the Internet +even if normally it would from that subnet. Therefore, you will have to set up +VPC endpoints for the AWS services that the Lambda function uses. This includes: +- FSx +- AWS Secrets Manager +- DynamoDB if you are using it to store the secrets table + +If you use the CloudFormation template provided in this repository to deploy the sample +you will be given the option to have it create these service endpoints for you. If you are +setting up the Lambda function manually, you will have to create these endpoints yourself. + +Note that you can only have one service endpoint per service per VPC. So, don't attempt +to add one if one already exists for the VPC you are going to run the Lambda function in. + +The way this script authenticates to the FSx for ONTAP management endpoint is by using +the credentials stored in AWS Secrets Manager. Since it can manage multiple FSxN file +systems a table is used to specify which secret to use for each file system. This `secretsTable` +can either be stored in a DynamoDB table, or just hard coded in the source code of the +Lambda function. The schema for the `secretsTable` is as follows: +```json +[ + {"fsxId": "fs-XXXXXXXXXXXXXXXXX", "secretName": "fsxn-credentials", "usernameKey": "username", "passwordKey": "password"}, + {"fsxId": "fs-XXXXXXXXXXXXXXXXX", "secretName": "fsxn-credentials", "usernameKey": "username", "passwordKey": "password"}, + {"fsxId": "fs-XXXXXXXXXXXXXXXXX", "secretName": "fsxn-credentials", "usernameKey": "username", "passwordKey": "password"}, + {"fsxId": "fs-XXXXXXXXXXXXXXXXX", "secretName": "fsxn-credentials", "usernameKey": "username", "passwordKey": "password"} +] +``` +Where the values associated with each key are as follows: + +| Key | Value | Example Value shown above| +|:----|:------| :------------------------| +| `fsxId` | The ID of the FSxN file system. | `fs-XXXXXXXXXXXXXXXXX` | +| `secretName` | The name of the secret in Secrets Manager. | `fsxn-credentials` | +| `usernameKey` | The key in the secret that contains the username. | `username` | +| `passwordKey` | The key in the secret that contains the password. | `password` | + +:bulb: **NOTE:** If you are going to maintain the `secretsTable` in the source code, and use the +CloudFormation template to deploy the Lambda function, you will have to update the `secretsTable` +variable in the code after the CloudFormation stack is created. Or, edit the source code within +the Cloudformation template itself. + +## Deployment +There are two ways to deploy this script. The first way to is use the CloudFormation +template provided in the `cloudformation.yaml` file. The second way is to follow the +steps in the "Manual Setup" section below. + +### CloudFormation Deployment +Copy the `cloudformation.yaml` file to your local machine. Then, go to the CloudFormation +service in the AWS console, and click on "Create stack." Select the "Upload a template file" +option and upload the `cloudformation.yaml` file. Click "Next." + +On the next page, give the stack a name. Note that this name is used as a suffix to most of the resources it creates +so you might want to keep it short, but meaningful. After the stack name you will need to fill in the following parameters: + +| Parameter Name | Description | +|:--------------|:------------| +| subNetIds| List the subnets that you want the Lambda function to run in. They must have connectivity to the FSxN file systems management endpoints. | +| vpcId | The VPC that contains the subnets. This is only used if you are having this CloudFormation template create the AWS service VPC endpoints. | +| securityGroupIds | The security group that the Lambda function will use. This security group should allow access to the AWS service endpoints and the FSx for ONTAP management endpoint over TCP port 443. | +| dynamoDbSecretsTableName | The name of the DynamoDB table that contains the `secretsTable` described above. This value is optional, but if not set, the table commented out in the code will have to be updated to provide the needed information.| +| dynamoDbRegion| The region where the DynamoDB table is located. | +| secretsManagerRegion| The region where the AWS Secrets Manager secrets are located. | +| createWatchdogAlarm | If set to `true` a CloudWatch alarm will be created that will trigger if the Lambda function fails while trying to set the auto size mode on a volume. | +| snsTopicArn| The ARN of the SNS topic that the CloudWatch alarm will send a message to if the Lambda function fails. | +| createSecretManagerEndpoint| If set to `true` a Secrets Manager VPC endpoint will be created. Note that you can only have one VPC service endpoint per service per VPC. | +| createFSxEndpoint| If set to `true` a FSx VPC endpoint will be created. Note that you can only have one VPC service endpoint per service per VPC. | +| createDynamoDbEndpoint| If set to `true` a DynamoDB VPC endpoint will be created. Note that you can only have one VPC service endpoint per service per VPC. | +| routeTableIds| Since the DynamoDB endpoint is a `Gateway` type, routing tables have to be updated to use it. Set this parameter to any route table IDs you want updated. | +| endpointSecurityGroupIds| The security group that the VPC endpoints will use. This security group should allow access to the AWS service the endpoints from the Lambda function over port 443. Since the Lambda function will have the security group specified above assigned to it, it can be used as a network `source` for this security group. | +| autoSizeMode| The auto size mode you want to set the volume to. Valid values are: `grow`, `grow_shrink`, and `off`. | +| growThresholdPrecentage| The percentage of the volume that must be used before a volume will grow. | +| maxGrowSizePercentage| The maximum size the volume can auto grow to expressed in terms of a percentage of the initial volume size. | +| shrinkThresholdPrecentage| The percentage of the volume that must be used before a volume will shrink. | +| minShrinkSizePercentage| The minimum size the volume can auto shrink to expressed in terms of a percentage of the initial volume size. | +| maxWaitTime| The maximum time, in seconds, that the script will wait for the volume to be created before it will give up and exit. This can happen if a lot of volumes are created at the same time. | + +Once you have filled in these parameters, click `Next`. On the next page you must accept that this +template can, and does, create roles. Click `Next`. Finally, on the last page, you can review the stack and click `Submit`. + +After the stack has been created if you plan to maintain the `secretsTable` within the source code, now would +be the best time to modify it. To do so, go to the Lambda service, find the Lambda function (the name +will start with "auto-set-fsxn-auto-grow" and end with the name you gave the CloudFormation stack) +and use the inline editor to modify the `secretsTable` variable. + +To test the function, simply create a volume in the AWS console and check from the ONTAP CLI +that auto size mode appropriately. If it isn't set, check the CloudWatch +logs for the Lambda function to see what went wrong. + +:warning: **NOTE:** This program is expecting to be called by a CloudWatch event, if you just click +on the `Test` button within the Lambda console, it will fail since the 'event' structure will not +be set appropriately. + +### Manual Setup +If for some reason you can't run the CloudFormation template, here are the steps you can use to manually setup the service: + +#### Create secrets in AWS Secrets Manager Create a secret in Secrets Manager for each of the FSxN file systems you want to manage with this script. Each secret should have two key value pairs. One that specifies the user account to use when issuing API calls, and the other that specifies the password for that account. Note that if you use the same username and password, it is okay to use the same secret for multiple file systems. -### Create a role for the Lambda function +#### Create a role for the Lambda function The Lambda function doesn't leverage that many AWS services, so only a few permissions are required: - | Permission | Minimal Scope | Notes |:------------------------|:----------------|:----------------| -| Allow:logs:CreateLogGroup | arn:aws:logs:::* | This is required so you can get logs from the Lambda function. | -| Allow:logs:CreateLogStream
Allow:logs:PutLogEvents | arn:aws:logs:::/aws/lambda/:* | This is required so you can get logs from the Lambda function. | -| Allow:secretsmanager:GetSecretValue | | This is required so the Lambda function can get the credentials for the FSxN file system. | -| Allow:dynamodb:Scan | | This is optional, depending on if you put your secretsTable in a DynamoDB. | -| Allow:fsx:DescribeFileSystems
Allow:fsx:DescribeVolumes | * | You can't limit these API. They are required to get information regarding the file system and volumes. | -| Allow:ec2:CreateNetworkInterface
Allow:ec2:DeleteNetworkInterface
Allow:ec2:DescribeNetworkInterfaces | * | Since the Lambda function is going to run within your VPC, it has to be able to create a network interface to communicate with the FSxn file system API. | - -### Create AWS Endpoints -Since the Lambda function will be configured to run within the VPC that contains the FSxN -file system, so it can issue API calls against it, there will need to be AWS endpoints so -the Lambda function can access some of the AWS service. If you have a Transit Gateway setup +| Allow:logs:CreateLogGroup | arn:aws:logs:\:\:* | This is required so you can get logs from the Lambda function. | +| Allow:logs:CreateLogStream
Allow:logs:PutLogEvents | arn:aws:logs:\:\:/aws/lambda/\:* | This is required so you can get logs from the Lambda function. | +| Allow:secretsmanager:GetSecretValue | \ | This is required so the Lambda function can get the credentials for the FSxN file system. | +| Allow:dynamodb:Scan | \ | This is optional, depending on if you put your `secretsTable` in a DynamoDB table. | +| Allow:fsx:DescribeFileSystems
Allow:fsx:DescribeVolumes | * | You can't limit the scope of these APIs. They are required to get information regarding the file system and volumes. | +| Allow:ec2:CreateNetworkInterface
Allow:ec2:DeleteNetworkInterface
Allow:ec2:DescribeNetworkInterfaces | * | Since the Lambda function is going to run within your VPC, it has to be able to create a network interface to communicate with the FSxN file system endpoints. | + +#### Create AWS Endpoints +Since the Lambda function will be configured to run within a VPC that can communicate with the FSxN +file systems, so it can issue API calls against them, there will need to be AWS endpoints so +the Lambda function can also access some of the AWS services. If you have a Transit Gateway setup that allows access to the Internet, you may not have to create these endpoints, otherwise, the -following endpoints will need to be created, and attached to the VPC and subnets that the -FSxN file system is attached to. +following endpoints will need to be created, and attached to the VPC and subnets that the Lambda +function will run in: - FSx - SecretsManager -- DynamoDB - You only need this one if you are going to store you secrtsTable in DynamoDB. It can be a Gateway endpoint. +- DynamoDB - You only need this one if you are going to store your `secretsTable` in DynamoDB. It is recommended that this be a `Gateway` type endpoint. However, if you do that you will also have to update the routing tables associated with the subnets that the Lambda function is deployed on in order for the Lambda function to be able to use it. + +:warning: Note that you can only have one service endpoint per service per VPC. So, don't attempt +to add one if one already exists for the VPC you are going to run the Lambda function in. -### Create the Lambda Function +#### Create the Lambda Function Create a Lambda function with the following parameters: - Authored from scratch. -- Uses the Python runtime. +- Use the Python runtime. - Set the permissions to the role created above. - Enable VPC. Found under the Advanced Settings. - - Attached to the VPC that contains the FSxN file system - - Attached to the Subnets that contain the FSxN file system. - - Attached a security group that allows access from any IP within the two subnets. + - Attached to the VPC that can communicate with the FSxN file systems. + - Attached to the Subnets that can communicate with the FSxN file systems. + - Attached to a security group that allows access from any IP within the two subnets over port 443. After you create the function, you will be able to insert the code included with this sample into the code box. Once you have inserted the code, modify the definitions @@ -63,41 +164,40 @@ is a dictionary with the following keys: - usernameKey - The name of the key in the secret that contains the username. - passwordKey - The name of the key in the secret that contains the password. - **NOTE:** Instead of defining the secretsTable in the script, you can define -dynamodbSecretsTableName and dynamodbRegion and the script will read in the + :bulb: **NOTE:** Instead of defining the secretsTable in the code, you can define +dynamoDbSecretsTableName and dynamoDbRegion and the program will read in the secretsTable information from the specified DynamoDB table. The table should have -the same fields as the secretsTable defined above. +the same fields as the `secretsTable` defined above. - secretsManagerRegion - Defines the region where your secrets are stored. - autoSizeMode - Defines the auto size mode you want to set the volume to. Valid values are: - grow - The volume will automatically grow when it reaches the grow threshold. - - grow_shrink - The volume will automatically grow, and shrink when it reachs the shrink threshold. + - grow_shrink - The volume will automatically grow, and shrink when it reaches the shrink threshold. - off - The volume will not automatically grow or shrink. -- growThresholdPercentage - The percentage of the volume that must be used before the volume will grow. -- maxGrowSizePercentage - The maximum size the volume can auto grow to expressed in terms of a percentage of the volume size. The default is 200%. -- shrinkThresholdPercentage - The percentage of the volume that must be used before the volume will shrink. -- minShrinkSizePercentage - The minimum size the volume can auto shrink to expressed in terms of a percentage of the volume size. The default is 50%. +- growThresholdPercentage - The percentage of the volume that must be in use before the volume will grow. +- maxGrowSizePercentage - The maximum size the volume can auto grow to, expressed in terms of a percentage of the initial volume size. +- shrinkThresholdPercentage - The percentage of the volume that must be in use before the volume will shrink. +- minShrinkSizePercentage - The minimum size the volume can auto shrink to, expressed in terms of a percentage of the initial volume size. - maxWaitTime - The maximum time, in seconds, the script will wait for the volume to be created before it will give up and exit. -**NOTE:** Do not delete the variables or set them to None or empty -strings, as the script will not run properly if done so. +:warning: **NOTE:** Do not delete the variables or set them to None or empty strings, as the script will not run properly if done so. Once you have updated the program, click on the "Deploy" button. Next, click on the Configuration tab, then General and set the timeout to 2 minutes, or -two times the number of seconds you set the maxWaitTime variable. Note that typically +two times the number of seconds you set the `maxWaitTime` variable. Note that typically the program will not run this long, but if there are a lot of volumes being created at the same time, it may have to wait a while for the volume to get created on the ONTAP side before it can set the auto size mode. -### Create an Event Bridge Rule (a.k.a. CloudWatch Event) that will trigger when a FSx Volume is created +#### Create an Event Bridge Rule (a.k.a. CloudWatch Event) that will trigger when a FSx Volume is created Once on the "Event Bridge" page, click on Rules on the left-hand side. From there click on Create Rule. Give the rule a name, and make sure to put the rule on the "Default" bus. Finally select "Rule with an event pattern" and click Next. Select "other" as the event source, skip pass the "Sample Event" section, and click on -"Custom pattern (JSON editor)" under the Creation Method. Paste the following in the -Edit Event Pattern text box: +"Custom pattern (JSON editor)" under the Creation Method paste the following in the +`Edit Event Pattern` text box: ```json { "detail-type": [ @@ -114,11 +214,14 @@ Edit Event Pattern text box: } ``` -Click Next. This next page will allow you to select the Lambda function you created above. +Click `Next`. The next page will allow you to select the Lambda function you created above. Just take the defaults for the remaining pages and click on "Create Rule." At this point every time a volume is created the Lambda function will be called, and it will attempt to set the auto size mode as specified via the variables at the top of the code. +To confirm it is working, create a volume in the AWS console and check the auto size mode +from the ONTAP CLI. If it isn't set, check the CloudWatch logs for the Lambda function to +see what went wrong. ## Author Information diff --git a/Management-Utilities/auto_set_fsxn_auto_grow/cloudformation.yaml b/Management-Utilities/auto_set_fsxn_auto_grow/cloudformation.yaml new file mode 100644 index 0000000..a18770e --- /dev/null +++ b/Management-Utilities/auto_set_fsxn_auto_grow/cloudformation.yaml @@ -0,0 +1,594 @@ +Description: "Deploy auto-set-fsxn-auto-grow" +# +# This just formats the page that prompts for the parameters when using the AWS Console to deploy your stack. +Metadata: + AWS::CloudFormation::Interface: + ParameterGroups: + - Label: + default: "Deployment Parameters" + Parameters: + - subNetIds + - vpcId + - securityGroupIds + - dynamoDbSecretsTableName + - dynamoDbRegion + - secretsManagerRegion + - createWatchdogAlarm + - snsTopicArn + - createSecretManagerEndpoint + - createFSxEndpoint + - createDynamoDbEndpoint + - routeTableIds + - endpointSecurityGroupIds + - Label: + default: "Auto Size Parameters" + Parameters: + - autoSizeMode + - growThresholdPrecentage + - maxGrowSizePercentage + - shrinkThresholdPrecentage + - minShrinkSizePercentage + - maxWaitTime + +Parameters: + subNetIds: + Description: "The subnet IDs where you want the Lambda function to run from. These subnets must have access to management end points of all the FSxNs you want it to manage." + Type: "List" + + securityGroupIds: + Description: "The security group IDs to associate with the Lambda function. Just needs to allow outbound traffic to port 443 to the AWS API endpoints as well as the FSxNs." + Type: "List" + + vpcId: + Description: "The VPC ID of the subnets where the Lambda function is run from. This is only needed if you are creating an AWS service VPC endpoint." + Type: String + Default: "" + + dynamoDbSecretsTableName: + Description: "The name of the DynamoDB table that contains the list of AWS secrets use to authenicate to the FSxNs with." + Type: String + Default: "" + + dynamoDbRegion: + Description: "The region where the DynamoDB table is located." + Type: String + Default: "" + + secretsManagerRegion: + Description: "The region where the AWS Secrets are located." + Type: String + Default: "" + + createWatchdogAlarm: + Description: "Create a CloudWatch alarm to monitor the this Lambda function." + Type: String + Default: "true" + AllowedValues: ["true", "false"] + + snsTopicArn: + Description: "The ARN of the SNS topic to send watchdog alerts to. Only needed if createWatchdogAlarm is set to 'true'." + Type: String + Default: "" + + createSecretManagerEndpoint: + Description: "Create a secret manager endpoint." + Type: String + Default: "false" + AllowedValues: ["true", "false"] + + createFSxEndpoint: + Description: "Create an FSx endpoint." + Type: String + Default: "false" + AllowedValues: ["true", "false"] + + createDynamoDbEndpoint: + Description: "Create a DynamoDB endpoint." + Type: String + Default: "false" + AllowedValues: ["true", "false"] + + endpointSecurityGroupIds: + Description: "The security group IDs to associate with the AWS service VPC endpoints. Must allow traffic from from the Lambda function over TCP port 443. This parameter is only needed if you are creating the AWS service VPC endpoints." + Type: CommaDelimitedList + Default: "" + + routeTableIds: + Description: "The route table IDs to update to use the DynamoDB endpoint. Since the DynamoDB endpoint is of type 'Gateway' route tables must be updated to use it. This parameter is only needed if createDynamoDbEndpoint is set to 'true'." + Type: CommaDelimitedList + Default: "" + + autoSizeMode: + Description: "Defines the mode you want auto size run in. Allowed values are grow, grow and shrink, off." + Type: String + Default: "grow" + AllowedValues: ["grow", "grow_and_shrink", "off"] + + growThresholdPrecentage: + Description: "The percentage of the volume that must be used before the volume will grow." + Type: Number + Default: 85 + MinValue: 1 + MaxValue: 100 + + shrinkThresholdPrecentage: + Description: "The percentage of the volume that must be in use before the volume will shrink." + Type: Number + Default: 50 + MinValue: 1 + MaxValue: 100 + + maxGrowSizePercentage: + Description: "The maximum size the volume can auto grow to expressed in terms of a percentage of the volume size." + Type: Number + Default: 200 + MinValue: 1 + + minShrinkSizePercentage: + Description: "The minimum size the volume can auto shrink to expressed in terms of a percentage of the volume size." + Type: Number + Default: 50 + MinValue: 1 + + maxWaitTime: + Description: "The maximum time, in minutes, to wait for a volume to be created before it will give up and exit." + Type: Number + Default: 60 + MinValue: 10 + +Conditions: + CreateSecretManagerEndpoint: !Equals [!Ref createSecretManagerEndpoint, "true"] + CreateFSxEndpoint: !Equals [!Ref createFSxEndpoint, "true"] + CreateDynamoDbEndpoint: !Equals [!Ref createDynamoDbEndpoint, "true"] + CreateWatchdogAlarm: !Equals [!Ref createWatchdogAlarm, "true"] + +Resources: + SecretManagerEndpoint: + Type: AWS::EC2::VPCEndpoint + Condition: CreateSecretManagerEndpoint + Properties: + VpcId: !Ref vpcId + ServiceName: !Sub "com.amazonaws.${AWS::Region}.secretsmanager" + VpcEndpointType: 'Interface' + PrivateDnsEnabled: true + SubnetIds: !Ref subNetIds + SecurityGroupIds: !Ref endpointSecurityGroupIds + + FSxEndpoint: + Type: AWS::EC2::VPCEndpoint + Condition: CreateFSxEndpoint + Properties: + VpcId: !Ref vpcId + ServiceName: !Sub "com.amazonaws.${AWS::Region}.fsx" + VpcEndpointType: 'Interface' + PrivateDnsEnabled: true + SubnetIds: !Ref subNetIds + SecurityGroupIds: !Ref endpointSecurityGroupIds + + DynamoDbEndpoint: + Type: AWS::EC2::VPCEndpoint + Condition: CreateDynamoDbEndpoint + Properties: + VpcId: !Ref vpcId + ServiceName: !Sub "com.amazonaws.${AWS::Region}.dynamodb" + VpcEndpointType: 'Gateway' + RouteTableIds: !Ref routeTableIds + + watchDogAlarm: + Type: "AWS::CloudWatch::Alarm" + Condition: CreateWatchdogAlarm + Properties: + AlarmName: !Sub "auto-set-auto-grow-${AWS::StackName}" + AlarmDescription: !Sub "Watchdog alarm for the auto-set-auto-grow-${AWS::StackName} Lambda function." + Namespace: "AWS/Lambda" + MetricName: "Errors" + Dimensions: + - Name: "FunctionName" + Value: !Sub "auto-set-fsxn-auto-grow-${AWS::StackName}" + Statistic: "Maximum" + Period: 300 + EvaluationPeriods: 1 + TreatMissingData: "ignore" + Threshold: 0.5 + ComparisonOperator: "GreaterThanThreshold" + AlarmActions: + - !Ref snsTopicArn + + LambdaRole: + Type: "AWS::IAM::Role" + Properties: + RoleName: !Sub "auto-set-fsxn-auto-grow-${AWS::StackName}" + AssumeRolePolicyDocument: + Version: "2012-10-17" + Statement: + - Effect: "Allow" + Principal: + Service: "lambda.amazonaws.com" + Action: "sts:AssumeRole" + + ManagedPolicyArns: + - "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole" + + Policies: + - PolicyName: "LambdaPolicy" + PolicyDocument: + Version: "2012-10-17" + Statement: + - Effect: "Allow" + Action: + - "secretsManager:GetSecretValue" + Resource: + - !Sub "arn:aws:secretsmanager:${secretsManagerRegion}:${AWS::AccountId}:secret:*" + - Effect: "Allow" + Action: + - "dynamodb:Scan" + Resource: + - !Sub "arn:aws:dynamodb:${AWS::Region}:${AWS::AccountId}:table/${dynamoDbSecretsTableName}" + - Effect: "Allow" + Action: + - "fsx:DescribeFileSystems" + - "fsx:DescribeVolumes" + Resource: "*" + - Effect: "Allow" + Action: + - "ec2:CreateNetworkInterface" + - "ec2:DeleteNetworkInterface" + - "ec2:DescribeNetworkInterfaces" + Resource: "*" + + LambdaInvokePermission: + Type: AWS::Lambda::Permission + Properties: + FunctionName: !GetAtt LambdaFunction.Arn + Action: "lambda:InvokeFunction" + Principal: "events.amazonaws.com" + SourceArn: !GetAtt EventBridgeCreateVolumeRule.Arn + + EventBridgeCreateVolumeRule: + Type: "AWS::Events::Rule" + Properties: + Description: "Triggered when a FSx volume is created." + Name: !Sub "auto-set-fsxn-auto-grow-${AWS::StackName}" + EventBusName: "default" + EventPattern: + detail-type: + - "AWS API Call via CloudTrail" + detail: + eventSource: + - "fsx.amazonaws.com" + eventName: + - "CreateVolume" + Targets: + - Arn: !GetAtt LambdaFunction.Arn + Id: "Target_1" + + LambdaFunction: + Type: "AWS::Lambda::Function" + Properties: + FunctionName: !Sub "auto-set-fsxn-auto-grow-${AWS::StackName}" + Role: !GetAtt LambdaRole.Arn + VpcConfig: + SecurityGroupIds: !Ref securityGroupIds + SubnetIds: !Ref subNetIds + PackageType: "Zip" + Runtime: "python3.12" + Handler: "index.lambda_handler" + Timeout: !Ref maxWaitTime + Environment: + Variables: + dynamoDbRegion: !Ref dynamoDbRegion + dynamoDbSecretsTableName: !Ref dynamoDbSecretsTableName + secretsManagerRegion: !Ref secretsManagerRegion + + autoSizeMode: !Ref autoSizeMode + growThresholdPrecentage: !Ref growThresholdPrecentage + maxGrowSizePercentage: !Ref maxGrowSizePercentage + shrinkThresholdPrecentage: !Ref shrinkThresholdPrecentage + minShrinkSizePercentage: !Ref minShrinkSizePercentage + maxWaitTime: !Ref maxWaitTime + + Code: + ZipFile: | + ################################################################################ + # THIS SOFTWARE IS PROVIDED BY NETAPP "AS IS" AND ANY EXPRESS OR IMPLIED + # WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF + # MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO + # EVENT SHALL NETAPP BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, + # EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, + # PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; + # OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, + # WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR' + # OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF + # ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + ################################################################################ + + ################################################################################ + # This Lambda function is used to set the auto size feature to 'grow' on a + # volume that was created in an AWS FSx for NetApp ONTAP file system. It is + # expected to be triggered by a CloudWatch event that is generated when a + # volume is created. The function uses the ONTAP API to set the auto size + # mode to 'grow' on the volume therefore it most run within the VPC where the + # FSx for ONTAP file system is located. + # + # Version: v1.1 + # Date: 2024-12-06-20:30:19 + ################################################################################ + + import json + import time + import urllib3 + from urllib3.util import Retry + import logging + import boto3 + import os + # + ################################################################################ + # Configuration settings. + # + # You can either set the variables in the code below, or set environment variables + # with the same name as the variables below. Except for the secretsTable, you must + # set that in the code if you are not going to use DynamoDb to hold your secrets + # table. + # + # NOTE: The environment variables will take precedence over the variables + # set in the code. + # + ################################################################################ + # + # Create a table of secret names and keys for the username and password for + # each of the FSxIds. In the example below, it shows using the same + # secret for four different FSxIds, but you can set it up to use + # a different secret and/or keys for the username and password for each + # of the FSxId. + #secretsTable = [ + # {"fsxId": "fs-0e8d9172fa5XXXXXX", "secretName": "fsxn-credentials", "usernameKey": "username", "passwordKey": "password"}, + # {"fsxId": "fs-020de2687bdXXXXXX", "secretName": "fsxn-credentials", "usernameKey": "username", "passwordKey": "password"}, + # {"fsxId": "fs-07bcb7ad84aXXXXXX", "secretName": "fsxn-credentials", "usernameKey": "username", "passwordKey": "password"}, + # {"fsxId": "fs-077b5ff4195XXXXXX", "secretName": "fsxn-credentials", "usernameKey": "username", "passwordKey": "password"} + # ] + # + # If you don't want to define the secretsTable in this script, you can + # define the following variables to use a DynamoDB table to get the + # secret information. + # + # NOTE: If both the secretsTable, and dynamodbSecretsTableName are defined, + # then the secretsTable will be used. + dynamoDbRegion=None + dynamoDbSecretsTableName=None + # + # Set the region where the secrets are stored. + secretsManagerRegion=None + # + # Set the auto size mode. Supported values are "grow", "grow_shrink", and "off". + autoSizeMode = "grow" + # + # Set the grow-threshold-percentage for the volume. This is the percentage of the volume that must be used before it grows. + growThresholdPercentage = 85 + # + # Set the maximum grow size for the volume in terms of the percentage of the provisioned size. + maxGrowSizePercentage = 200 + # + # Set the shrink-threshold-percentage for the volume. This is the percentage of the volume that must be free before it shrinks. + shrinkThresholdPercentage = 50 + # + # Set the minimum shirtk size for the volume in terms of the percentage of the provisioned size. + minShrinkSizePercentage = 100 + # + # Set the time to wait for a volume to get created. This Lambda function will + # loop waiting for the volume to be created on the ONTAP side so it can set + # the auto size parameters. It will wait up to the number of seconds specified + # below before giving up. NOTE: You must set the timeout of this function + # to at least the number of seconds specified here, and probably two times + # the number to account for the time it takes to do the API calls, + # otherwise the Lambda timeout feature will kill it before it is able to + # wait as long you want it to. Also note that the main reason for + # it to take a while for a volume to get created is when multiple are being + # created at the same time. So, if you have automation that might create a lot of + # volumes at the same time, you might need to either adjust this number really + # high, or come up with another way to get the auto size mode. + maxWaitTime=60 + + ################################################################################ + # This function sets the configuration variables from environment variables if + # they are defined. + ################################################################################ + def setConfigurationVariables(): + global secretsTable, secretsManagerRegion, autoSizeMode, growThresholdPercentage + global maxGrowSizePercentage, shrinkThresholdPercentage, minShrinkSizePercentage + global dynamoDbRegion, dynamoDbSecretsTableName, secretsTable, maxWaitTime, logger + + if os.environ.get('dynamoDbRegion') != None: + dynamoDbRegion = os.environ['dynamoDbRegion'] + if os.environ.get('dynamoDbSecretsTableName') != None: + dynamoDbSecretsTableName = os.environ['dynamoDbSecretsTableName'] + if os.environ.get('secretsManagerRegion') != None: + secretsManagerRegion = os.environ['secretsManagerRegion'] + if os.environ.get('autoSizeMode') != None: + autoSizeMode = os.environ['autoSizeMode'] + if os.environ.get('growThresholdPercentage') != None: + growThresholdPercentage = int(os.environ['growThresholdPercentage']) + if os.environ.get('maxGrowSizePercentage') != None: + maxGrowSizePercentage = int(os.environ['maxGrowSizePercentage']) + if os.environ.get('shrinkThresholdPercentage') != None: + shrinkThresholdPercentage = int(os.environ['shrinkThresholdPercentage']) + if os.environ.get('minShrinkSizePercentage') != None: + minShrinkSizePercentage = int(os.environ['minShrinkSizePercentage']) + if os.environ.get('maxWaitTime') != None: + maxWaitTime = int(os.environ['maxWaitTime']) + # + # Check that all the required variables are set. + message = "" + if dynamoDbRegion == None and dynamoDbSecretsTableName == None and 'secretsTable' not in globals(): + message += 'Error, you must either define the secretsTable array at the top of this script, or define dynamodbRegion and dynamoDbSecretsTableName environment variables.\n' + + if secretsManagerRegion == None: + message += 'Error, you must define the secretsManagerRegion environment variable.\n' + + if autoSizeMode == None or autoSizeMode not in ['grow', 'grow_shrink', 'off']: + message += 'Error, you must define the autoSizeMode environment variable to either "grow", "grow_shrink", or "off".\n' + + if growThresholdPercentage == None or isinstance(growThresholdPercentage, int) == False or growThresholdPercentage < 0 or growThresholdPercentage > 100: + message += 'Error, you must define the growThresholdPercentage environment variable between 0 and 100.\n' + + if maxGrowSizePercentage == None or isinstance(maxGrowSizePercentage, int) == False or maxGrowSizePercentage < 0 or maxGrowSizePercentage > 1000: + message += 'Error, you must define the maxGrowSizePercentage environment variable between 0 and 1000.\n' + + if shrinkThresholdPercentage == None or isinstance(shrinkThresholdPercentage, int) == False or shrinkThresholdPercentage < 0 or shrinkThresholdPercentage > 100: + message += 'Error, you must define the shrinkThresholdPercentage environment variable between 0 and 100.\n' + + if minShrinkSizePercentage == None or isinstance(minShrinkSizePercentage, int) == False or minShrinkSizePercentage < 0 or minShrinkSizePercentage > 100: + message += 'Error, you must define the minShrinkSizePercentage environment variable between 0 and 100.\n' + + if maxWaitTime == None or isinstance(maxWaitTime, int) == False or maxWaitTime < 0 or maxWaitTime > 600: + message += 'Error, you must define the maxWaitTime environment variable between 0 and 600.\n' + + if message != "": + logger.critical(message) + raise Exception(message) + + ################################################################################ + # This function is used to obtain the username and password from AWS's Secrets + # Manager for the fsxnId passed in. It returns empty strings if it can't + # find the credentials. + ################################################################################ + def getCredentials(secretsManagerClient, fsxnId): + global secretsTable + + for secretItem in secretsTable: + if secretItem['fsxId'] == fsxnId: + secretsInfo = secretsManagerClient.get_secret_value(SecretId=secretItem['secretName']) + secrets = json.loads(secretsInfo['SecretString']) + username = secrets[secretItem['usernameKey']] + password = secrets[secretItem['passwordKey']] + return (username, password) + return ("", "") + + ################################################################################ + # This function returns the AWS structure for a FSxN volume based on the + # volumeId passed it. It confirms that the volume has been created on the ONTAP + # side by checking that the ResourceARN field equals the volumeARN passed in + # that came from the volume creation event and that the UUID field has been + # populated. It returns None if it can't find the volume. + ################################################################################ + def getVolumeData(fsxClient, volumeId, volumeARN): + + global logger + + cnt = 0 + while cnt < maxWaitTime: + awsVolume = fsxClient.describe_volumes(VolumeIds=[volumeId])['Volumes'][0] + if awsVolume['ResourceARN'] == volumeARN and awsVolume['OntapConfiguration'].get("UUID") != None: + return awsVolume + logger.debug(f'Looping, getting the UUID {cnt}') + cnt += 1 + time.sleep(1) + + return None + + ################################################################################ + ################################################################################ + def lambda_handler(event, context): + + global logger, secretsTable + # + # Set up "logging" to appropriately display messages. It can be set it up + # to send messages to a syslog server. + logging.basicConfig(datefmt='%Y-%m-%d_%H:%M:%S', format='%(asctime)s:%(name)s:%(levelname)s:%(message)s', encoding='utf-8') + logger = logging.getLogger("set_fsxn_volume_auto_size") + # logger.setLevel(logging.DEBUG) + logger.setLevel(logging.INFO) + # + # Set the logging level higher for these noisy modules to mute thier messages. + logging.getLogger("botocore").setLevel(logging.WARNING) + logging.getLogger("boto3").setLevel(logging.WARNING) + logging.getLogger("urllib3").setLevel(logging.WARNING) + # + # Set the configuration variables from environment variables if they are defined. + setConfigurationVariables() + # + # If this is an event from a failed call. Report that and return. + if event['detail'].get('errorCode') != None: + logger.warning(f"This is reporting on a error event. Error Code: {event['detail']['errorCode']}. Error Message: {event['detail']['errorMessage']}.") + return + # + # Create a Secrets Manager client. + session = boto3.session.Session() + secretsManagerClient = session.client(service_name='secretsmanager', region_name=secretsManagerRegion) + # + # Disable warning about connecting to servers with self-signed SSL certificates. + urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning) + # + # Set the https retries to 1. + retries = Retry(total=None, connect=1, read=1, redirect=10, status=0, other=0) + http = urllib3.PoolManager(cert_reqs='CERT_NONE', retries=retries) + # + # Read in the secretTable if it is not already defined. + # It is safe to assume dynamoDbRegion and dynamoDbSecretsTableName are defined + # because the setConfigurationVariables function would have errored out if they + # were not. + if 'secretsTable' not in globals(): + dynamodbClient = boto3.resource("dynamodb", region_name=dynamoDbRegion) + table = dynamodbClient.Table(dynamoDbSecretsTableName) + response = table.scan() + secretsTable = response["Items"] + # + # Get the FSxN ID, region, volume name, volume ID, and volume ARN from the CloudWatch event. + fsxId = event['detail']['responseElements']['volume']['fileSystemId'] + regionName = event['detail']['awsRegion'] + volumeName = event['detail']['requestParameters']['name'] + volumeId = event['detail']['responseElements']['volume']['volumeId'] + volumeARN = event['detail']['responseElements']['volume']['resourceARN'] + if fsxId == "" or regionName == "" or volumeId == "" or volumeName == "" or volumeARN == "": + message = "Couldn't obtain the fsxId, region, volume name, volume ID or volume ARN from the CloudWatch evevnt." + logger.critical(message) + return + + logger.debug(f'Data from CloudWatch event: FSxID={fsxId}, Region={regionName}, VolumeName={volumeName}, volumeId={volumeId}.') + # + # Get the username and password for the FSxN ID. + (username, password) = getCredentials(secretsManagerClient, fsxId) + if username == "" or password == "": + message = f'No credentials for FSxN ID: {fsxId}.' + logger.critical(message) + return + # + # Build a header that is used for all the ONTAP API requests. + auth = urllib3.make_headers(basic_auth=f'{username}:{password}') + headers = { **auth } + # + # Get the management IP of the FSxN file system. + fsxClient = boto3.client('fsx', region_name = regionName) + fs = fsxClient.describe_file_systems(FileSystemIds = [fsxId])['FileSystems'][0] + fsxnIp = fs['OntapConfiguration']['Endpoints']['Management']['IpAddresses'][0] + if fsxnIp == "": + message = f"Can't find management IP for FSxN file system with an ID of '{fsxId}'." + logger.critical(message) + return + # + # Get the volume UUID and volume size based on the volume ID. + volumeData = getVolumeData(fsxClient, volumeId, volumeARN) + if volumeData == None: + message=f'Failed to get volume information for volumeID: {volumeId}.' + logger.critical(message) + return + volumeUUID = volumeData["OntapConfiguration"]["UUID"] + volumeSizeInMegabytes = volumeData["OntapConfiguration"]["SizeInMegabytes"] + # + # Set the auto grow feature. + try: + endpoint = f'https://{fsxnIp}/api/storage/volumes/{volumeUUID}' + maximum = volumeSizeInMegabytes * maxGrowSizePercentage / 100 * 1024 * 1024 + minimum = volumeSizeInMegabytes * minShrinkSizePercentage / 100 * 1024 * 1024 + # + # Make sure the minimum is at least 20MB. + if minimum < 20 * 1024 * 1024: + minimum = 20 * 1024 * 1024 + data = json.dumps({"autosize": {"mode": autoSizeMode, "grow_threshold": growThresholdPercentage, "maximum": maximum, "minimum": minimum, "shrink_threshold": shrinkThresholdPercentage}}) + logger.debug(f'Trying {endpoint} with {data}.') + response = http.request('PATCH', endpoint, headers=headers, timeout=5.0, body=data) + if response.status >= 200 and response.status <= 299: + logger.info(f"Updated the auto size parameters for volume name {volumeName}.") + else: + logger.error(f'API call to {endpoint} failed. HTTP status code: {response.status}. Error message: {response.data}.') + except Exception as err: + logger.critical(f'Failed to issue API against {fsxnIp}. The error messages received: "{err}".') diff --git a/Management-Utilities/auto_set_fsxn_auto_grow/set_fsxn_volume_auto_grow.py b/Management-Utilities/auto_set_fsxn_auto_grow/set_fsxn_volume_auto_grow.py index c4767ec..bd5ed2e 100644 --- a/Management-Utilities/auto_set_fsxn_auto_grow/set_fsxn_volume_auto_grow.py +++ b/Management-Utilities/auto_set_fsxn_auto_grow/set_fsxn_volume_auto_grow.py @@ -18,6 +18,9 @@ # volume is created. The function uses the ONTAP API to set the auto size # mode to 'grow' on the volume therefore it most run within the VPC where the # FSx for ONTAP file system is located. +# +# Version: %%VERSION%% +# Date: %%DATE%% ################################################################################ import json @@ -26,18 +29,32 @@ from urllib3.util import Retry import logging import boto3 +import os +# +################################################################################ +# Configuration settings. +# +# You can either set the variables in the code below, or set environment variables +# with the same name as the variables below. Except for the secretsTable, you must +# set that in the code if you are not going to use DynamoDb to hold your secrets +# table. +# +# NOTE: The environment variables will take precedence over the variables +# set in the code. +# +################################################################################ # # Create a table of secret names and keys for the username and password for # each of the FSxIds. In the example below, it shows using the same # secret for four different FSxIds, but you can set it up to use # a different secret and/or keys for the username and password for each # of the FSxId. -secretsTable = [ - {"fsxId": "fs-0e8d9172fa5XXXXXX", "secretName": "fsxn-credentials", "usernameKey": "fsxn-username", "passwordKey": "fsxn-password"}, - {"fsxId": "fs-020de2687bdXXXXXX", "secretName": "fsxn-credentials", "usernameKey": "fsxn-username", "passwordKey": "fsxn-password"}, - {"fsxId": "fs-07bcb7ad84aXXXXXX", "secretName": "fsxn-credentials", "usernameKey": "fsxn-username", "passwordKey": "fsxn-password"}, - {"fsxId": "fs-077b5ff4195XXXXXX", "secretName": "fsxn-credentials", "usernameKey": "fsxn-username", "passwordKey": "fsxn-password"} - ] +#secretsTable = [ +# {"fsxId": "fs-0e8d9172fa5XXXXXX", "secretName": "fsxn-credentials", "usernameKey": "username", "passwordKey": "password"}, +# {"fsxId": "fs-020de2687bdXXXXXX", "secretName": "fsxn-credentials", "usernameKey": "username", "passwordKey": "password"}, +# {"fsxId": "fs-07bcb7ad84aXXXXXX", "secretName": "fsxn-credentials", "usernameKey": "username", "passwordKey": "password"}, +# {"fsxId": "fs-077b5ff4195XXXXXX", "secretName": "fsxn-credentials", "usernameKey": "username", "passwordKey": "password"} +# ] # # If you don't want to define the secretsTable in this script, you can # define the following variables to use a DynamoDB table to get the @@ -45,11 +62,11 @@ # # NOTE: If both the secretsTable, and dynamodbSecretsTableName are defined, # then the secretsTable will be used. -#dynamodbRegion="us-west-2" -#dynamodbSecretsTableName="fsx_secrets" +dynamoDbRegion=None +dynamoDbSecretsTableName=None # # Set the region where the secrets are stored. -secretsManagerRegion="us-west-2" +secretsManagerRegion=None # # Set the auto size mode. Supported values are "grow", "grow_shrink", and "off". autoSizeMode = "grow" @@ -80,6 +97,64 @@ # high, or come up with another way to get the auto size mode. maxWaitTime=60 +################################################################################ +# This function sets the configuration variables from environment variables if +# they are defined. +################################################################################ +def setConfigurationVariables(): + global secretsTable, secretsManagerRegion, autoSizeMode, growThresholdPercentage + global maxGrowSizePercentage, shrinkThresholdPercentage, minShrinkSizePercentage + global dynamoDbRegion, dynamoDbSecretsTableName, secretsTable, maxWaitTime, logger + + if os.environ.get('dynamoDbRegion') != None: + dynamoDbRegion = os.environ['dynamoDbRegion'] + if os.environ.get('dynamoDbSecretsTableName') != None: + dynamoDbSecretsTableName = os.environ['dynamoDbSecretsTableName'] + if os.environ.get('secretsManagerRegion') != None: + secretsManagerRegion = os.environ['secretsManagerRegion'] + if os.environ.get('autoSizeMode') != None: + autoSizeMode = os.environ['autoSizeMode'] + if os.environ.get('growThresholdPercentage') != None: + growThresholdPercentage = int(os.environ['growThresholdPercentage']) + if os.environ.get('maxGrowSizePercentage') != None: + maxGrowSizePercentage = int(os.environ['maxGrowSizePercentage']) + if os.environ.get('shrinkThresholdPercentage') != None: + shrinkThresholdPercentage = int(os.environ['shrinkThresholdPercentage']) + if os.environ.get('minShrinkSizePercentage') != None: + minShrinkSizePercentage = int(os.environ['minShrinkSizePercentage']) + if os.environ.get('maxWaitTime') != None: + maxWaitTime = int(os.environ['maxWaitTime']) + # + # Check that all the required variables are set. + message = "" + if dynamoDbRegion == None and dynamoDbSecretsTableName == None and 'secretsTable' not in globals(): + message += 'Error, you must either define the secretsTable array at the top of this script, or define dynamodbRegion and dynamoDbSecretsTableName environment variables.\n' + + if secretsManagerRegion == None: + message += 'Error, you must define the secretsManagerRegion environment variable.\n' + + if autoSizeMode == None or autoSizeMode not in ['grow', 'grow_shrink', 'off']: + message += 'Error, you must define the autoSizeMode environment variable to either "grow", "grow_shrink", or "off".\n' + + if growThresholdPercentage == None or isinstance(growThresholdPercentage, int) == False or growThresholdPercentage < 0 or growThresholdPercentage > 100: + message += 'Error, you must define the growThresholdPercentage environment variable between 0 and 100.\n' + + if maxGrowSizePercentage == None or isinstance(maxGrowSizePercentage, int) == False or maxGrowSizePercentage < 0 or maxGrowSizePercentage > 1000: + message += 'Error, you must define the maxGrowSizePercentage environment variable between 0 and 1000.\n' + + if shrinkThresholdPercentage == None or isinstance(shrinkThresholdPercentage, int) == False or shrinkThresholdPercentage < 0 or shrinkThresholdPercentage > 100: + message += 'Error, you must define the shrinkThresholdPercentage environment variable between 0 and 100.\n' + + if minShrinkSizePercentage == None or isinstance(minShrinkSizePercentage, int) == False or minShrinkSizePercentage < 0 or minShrinkSizePercentage > 100: + message += 'Error, you must define the minShrinkSizePercentage environment variable between 0 and 100.\n' + + if maxWaitTime == None or isinstance(maxWaitTime, int) == False or maxWaitTime < 0 or maxWaitTime > 600: + message += 'Error, you must define the maxWaitTime environment variable between 0 and 600.\n' + + if message != "": + logger.critical(message) + raise Exception(message) + ################################################################################ # This function is used to obtain the username and password from AWS's Secrets # Manager for the fsxnId passed in. It returns empty strings if it can't @@ -137,6 +212,9 @@ def lambda_handler(event, context): logging.getLogger("boto3").setLevel(logging.WARNING) logging.getLogger("urllib3").setLevel(logging.WARNING) # + # Set the configuration variables from environment variables if they are defined. + setConfigurationVariables() + # # If this is an event from a failed call. Report that and return. if event['detail'].get('errorCode') != None: logger.warning(f"This is reporting on a error event. Error Code: {event['detail']['errorCode']}. Error Message: {event['detail']['errorMessage']}.") @@ -150,21 +228,16 @@ def lambda_handler(event, context): urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning) # # Set the https retries to 1. - retries = Retry(total=None, connect=1, read=1, redirect=10, status=0, other=0) # pylint: disable=E1123 + retries = Retry(total=None, connect=1, read=1, redirect=10, status=0, other=0) http = urllib3.PoolManager(cert_reqs='CERT_NONE', retries=retries) # # Read in the secretTable if it is not already defined. - if 'dynamodbRegion' in globals(): - dynamodbClient = boto3.resource("dynamodb", region_name=dynamodbRegion) # pylint: disable=E0602 - + # It is safe to assume dynamoDbRegion and dynamoDbSecretsTableName are defined + # because the setConfigurationVariables function would have errored out if they + # were not. if 'secretsTable' not in globals(): - if 'dynamodbRegion' not in globals() or 'dynamodbSecretsTableName' not in globals(): - message = 'Error, you must either define the secretsTable array at the top of this script, or define dynamodbRegion and dynamodbSecretsTableName.' - logger.critcal(message) - raise Exception(message) - - table = dynamodbClient.Table(dynamodbSecretsTableName) # pylint: disable=E0602 - + dynamodbClient = boto3.resource("dynamodb", region_name=dynamoDbRegion) + table = dynamodbClient.Table(dynamoDbSecretsTableName) response = table.scan() secretsTable = response["Items"] # @@ -176,8 +249,7 @@ def lambda_handler(event, context): volumeARN = event['detail']['responseElements']['volume']['resourceARN'] if fsxId == "" or regionName == "" or volumeId == "" or volumeName == "" or volumeARN == "": message = "Couldn't obtain the fsxId, region, volume name, volume ID or volume ARN from the CloudWatch evevnt." - logger.critcal(message) -# raise Exception(message) + logger.critical(message) return logger.debug(f'Data from CloudWatch event: FSxID={fsxId}, Region={regionName}, VolumeName={volumeName}, volumeId={volumeId}.') @@ -187,7 +259,6 @@ def lambda_handler(event, context): if username == "" or password == "": message = f'No credentials for FSxN ID: {fsxId}.' logger.critical(message) -# raise Exception(message) return # # Build a header that is used for all the ONTAP API requests. @@ -201,7 +272,6 @@ def lambda_handler(event, context): if fsxnIp == "": message = f"Can't find management IP for FSxN file system with an ID of '{fsxId}'." logger.critical(message) -# raise Exception(message) return # # Get the volume UUID and volume size based on the volume ID. @@ -209,7 +279,6 @@ def lambda_handler(event, context): if volumeData == None: message=f'Failed to get volume information for volumeID: {volumeId}.' logger.critical(message) -# raise Exception(message) return volumeUUID = volumeData["OntapConfiguration"]["UUID"] volumeSizeInMegabytes = volumeData["OntapConfiguration"]["SizeInMegabytes"] @@ -219,6 +288,10 @@ def lambda_handler(event, context): endpoint = f'https://{fsxnIp}/api/storage/volumes/{volumeUUID}' maximum = volumeSizeInMegabytes * maxGrowSizePercentage / 100 * 1024 * 1024 minimum = volumeSizeInMegabytes * minShrinkSizePercentage / 100 * 1024 * 1024 + # + # Make sure the minimum is at least 20MB. + if minimum < 20 * 1024 * 1024: + minimum = 20 * 1024 * 1024 data = json.dumps({"autosize": {"mode": autoSizeMode, "grow_threshold": growThresholdPercentage, "maximum": maximum, "minimum": minimum, "shrink_threshold": shrinkThresholdPercentage}}) logger.debug(f'Trying {endpoint} with {data}.') response = http.request('PATCH', endpoint, headers=headers, timeout=5.0, body=data) diff --git a/Management-Utilities/auto_set_fsxn_auto_grow/update_auto_set_fsxn_auto_grow_CF_Template b/Management-Utilities/auto_set_fsxn_auto_grow/update_auto_set_fsxn_auto_grow_CF_Template new file mode 100755 index 0000000..cff237d --- /dev/null +++ b/Management-Utilities/auto_set_fsxn_auto_grow/update_auto_set_fsxn_auto_grow_CF_Template @@ -0,0 +1,46 @@ +#!/bin/bash +# +# This script is used to update the CloudFormation template with the latest +# version of the Lambda function. It will also update the version number in +# the template. +################################################################################# + +majorVersionNum=1 +file="set_fsxn_volume_auto_grow.py" + +tmpfile1=$(mktemp /tmp/tmpfile1.XXXXXX) +tmpfile2=$(mktemp /tmp/tmpfile2.XXXXXX) +trap "rm -f $tmpfile1 $tmpfile2" exit +# +# First get the existing code out of the CF template. +sed -e '1,/ZipFile/d' cloudformation.yaml > cloudformation.yaml.tmp +# +# Now get the Date and Version lines out of both files. +egrep -v '^ # Date:|^ # Version' cloudformation.yaml.tmp > $tmpfile1 +egrep -v '^# Date:|^# Version:' $file > $tmpfile2 + +if diff -w $tmpfile1 $tmpfile2 > /dev/null; then + echo "No changes to the monitor code." + rm -f cloudformation.yaml.tmp + rm -f $tmpfile1 $tmpfile2 + exit 0 +fi +# +# Get the number of commits in the git history for the file to calculate the minor version number. +minorVersionNum="$(git log "$file" | egrep '^commit' | wc -l)" +if [ -z "$minorVersionNum" ]; then + echo "Failed to calculate version number." 1>&2 + exit 1 +fi + +version="v${majorVersionNum}.${minorVersionNum}" +# +# Strip out the monitoring code. +sed -e '/ZipFile/,$d' cloudformation.yaml > cloudformation.yaml.tmp +echo " ZipFile: |" >> cloudformation.yaml.tmp +# +# Add the monitoring code to the CF template while updating the version and date. +cat "$file" | sed -e 's/^/ /' -e "s/%%VERSION%%/${version}/" -e "s/%%DATE%%/$(date +%Y-%m-%d-%H:%M:%S)/" >> cloudformation.yaml.tmp + +echo "Updating cloudformation.yaml" +mv cloudformation.yaml.tmp cloudformation.yaml