v47.1.0

47.1.0 (2022-08-20)

Features

• riff-raff.yaml Generate a riff-raff.yaml from a CDK app (#1372)

At the Guardian, we use Riff-Raff for deployment, configured via a riff-raff.yaml file. This file describes the steps to take during a deployment. For example, to deploy a lambda, we'll typically have:

regions:
  - eu-west-1
stacks:
  - playground
allowedStages:
  - CODE
  - PROD
deployments:
  cloudformation:
    type: cloud-formation
    app: my-application
    parameters:
      templateStagePaths:
        CODE: MyApplication-CODE.template.json
        PROD: MyApplication-PROD.template.json
  my-application:
    type: aws-lambda
    dependencies:
      - cloudformation
    parameters:
      bucketSsmLookup: true
      fileName: my-app.zip
      lookupByTags: true

Creating this file can be a little complicated, and when creating a new service we usually copy the file from an existing repository, and then spend a couple of hours massaging it to fit. Even then, it's not uncommon for us to manually provision the infrastructure first, then use Riff-Raff from the second deployment onwards.

What does this change and how to use?

CDK knows[^3] everything to create this file. In this change, we auto-generate the riff-raff.yaml file, saving developer's time. The resulting file will be structured such that it can be used to provision a new stack from scratch via a Riff-Raff deployment, removing the need to manually provision things first.

Note
These changes are marked as experimental.

In order to make this feature easy to adopt, a new GuRootExperimental construct has been created. Adoption means going from this:

// bin/cdk.ts
import { App } from "aws-cdk-lib";

const app = new App();
new MyStack(app, "some-id", {});

To this:

// bin/cdk.ts
import { GuRootExperimental } from "@guardian/cdk/lib/experimental/constructs/root";

const app = new GuRootExperimental();
new MyStack(app, "some-id", {});

The auto-generated version of the above would be:

allowedStages:
  - CODE
  - PROD
deployments:
  lambda-upload-eu-west-1-playground-my-application:
    type: aws-lambda
    stacks:
      - playground
    regions:
      - eu-west-1
    app: my-application
    contentDirectory: my-application
    parameters:
      bucketSsmLookup: true
      lookupByTags: true
      fileName: my-app.zip
    actions:
      - uploadLambda
  cfn-eu-west-1-playground-my-application-stack:
    type: cloud-formation
    regions:
      - eu-west-1
    stacks:
      - playground
    app: my-application-stack
    contentDirectory: cdk.out
    parameters:
      templateStagePaths:
        CODE: MyApplication-CODE.template.json
        PROD: MyApplication-PROD.template.json
    dependencies:
      - lambda-upload-eu-west-1-playground-my-application
  lambda-update-eu-west-1-playground-my-application:
    type: aws-lambda
    stacks:
      - playground
    regions:
      - eu-west-1
    app: my-application
    contentDirectory: my-application
    parameters:
      bucketSsmLookup: true
      lookupByTags: true
      fileName: my-app.zip
    actions:
      - updateLambda
    dependencies:
      - cfn-eu-west-1-playground-my-application-stack

Crucially, the dependencies between deployments will be configured such that the order of events is:

1. Upload artifact to S3
2. Deploy infrastructure
3. Update the application

This will allow us to deploy applications from scratch using Riff-Raff - no more confusing first deployments!

Supported Riff-Raff deployment types

The following deployment types are currently supported:

• cloud-formation
• aws-lambda
• autoscaling

It also supports deploying across multiple regions and multiple stacks.

The added tests can be used to see more examples of the generated file.

Caveats

Directory structure

It's worth noting the addition of contentDirectory and app to each deployment. This impacts the directory structure of the files uploaded to Riff-Raff during CI. The files should match this structure:

.
├── cdk.out
│   └── MyApplication-CODE.template.json
│   └── MyApplication-PROD.template.json
└── my-application
    └── my-app.zip

Where all the CloudFormation templates are in cdk.out and application artifacts are in <app>. The file names are down to the user; a .zip file is used for illustration purposes only.

Use of tags

The generated riff-raff.yaml file will tell Riff-Raff to perform service discovery by tags.

If a service currently hardcodes the CloudFormation stack name, or lambda function names in it's riff-raff.yaml file, then it cannot switch to this experimental process. We'd need to think about if it's correct to support these stacks, and if yes, how. I'd suggest this is future work too.

Riff-Raff templates

It's worth noting that if you're migrating a service that uses templates in riff-raff.yaml, the snapshot will not exactly match the manually crafted riff-raff.yaml file, as RiffRaffYamlFileExperimental doesn't use templates, favouring instead to expand all the deployments. However, Riff-Raff will perform the same steps.

You might want to consider adding a snapshot test against the generated riff-raff.yaml file, to ensure it looks as you expect.