(bootstrap): Least privilege deployments with CDK V2

[chrisjgray]

General Issue

Least privilege deployments with CDK V2

The Question

When migrating to AWS CDK v2, there was a significant change in the way roles are used to deploy the cloudformation stack updates. Previously, a role could be assumed which has least privilege access to be able to deploy aws cdk resources within an account. With the upgrade, now we are responsible for creating a role that can assume a CDK Deploy role which then uses an Exec role which has full administrative privileges.

Why is granting anyone who can assume a role, full access to the entire account the primary usage pattern? What is the recommended approach to control role access within the accounts as to not be granting over permissive roles?

To mitigate this issue, we modified the cdk.json to disable the new-style synthesis.
"@aws-cdk/core:newStyleStackSynthesis": false

Additionally, after performing the steps to disable the new-style synthesis, we still have errors around cdk attempting to assume the deploy roles and file publishing role.

CDK CLI Version

2.7.0 (build cfb09d5)

Framework Version

No response

Node.js Version

v16.13.2

OS

macOS Big Sur

Language

Typescript

Language Version

Typescript (4.3.5)

Other information

No response

[kaprah]

Also as apart of this, Cloudformation stacks that were deployed before the cdk.json flag of "@aws-cdk/core:newStyleStackSynthesis": false was set, gets assigned the Service execution role created by CDK in V2. If you then change the newStyleStackSynthesis option to be false this still keeps the IAM service role that is assigned to the stack and it seems the only way to remove that role is to assign a new Service Role to the particular cloudformation stack.

Makes it difficult for deployments that have happened with CDK V2 before the above option was known, since even after it is set, they are still managed and updated by the service role that is assigned to the stack unless we are to change it after the fact.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-iam-servicerole.html

[kaprah]

With modifying the bootstrap template we are able to add in a iam:PassRole permission that corresponds with deployment roles that we do have, we can then run through cdk deploy --role-arn <roleArn> with the stacks that have the original bootstrap ones to update those particular stacks with our deployment roles that we want, the downside is we need to go through cdk bootstrap --show-template modify the template with our policy and deploy it out to each location, and if/when the bootstrap changes we need to consistently add in our unique policy.

[spssmn-aws]

+1 my customer is affected by this issue

[rix0rrr]

If the default permissions of the CloudFormation execution role are too broad for you, you can configure a different policy for them using the --cloudformation-execution-policies argument to cdk bootstrap.

[erikerikson]

Note that any existing stacks will need to be destroyed if this wasn't done prior to overriding the default.

Meanwhile such stacks will, again by default, allow using that execution role with CloudFormation to arbitrarily escalate their rights.

[kornicameister]

@erikerikson but if execution policy disallows changing cdk related policies it won't be possible to escalate them, will it?

[erikerikson]

@kornicameister The default rights given to the CDK are AdminstratorAccess. Perhaps there are mechanisms to limit what can be accomplished with that AWS defined and controlled policy (that the CDK bootstrap uses by default) but such policies would definitely not be in place by default. You may want to validate your expectations by attempting an escalation as a proof of concept.

[kornicameister]

Well that's why I have been using limited policies to allow deployments to operate. Even IAM actions are limited in a way that cdk related roles cannot be changed using them. It is far from ideal but at least something.

[chrisjgray]

That doesn't solve an issue for us.
If you have one app in the account needing to have least privilege to deploy just IAM Roles, then we would update the execution policies to just have IAM permissions.
If I have a second app that needs to deploy lambdas in that same account, now the execution policy needs to have both IAM permissions and Lambda permissions. That now means, someone can deploy lambdas from the first app which we wanted restricted to just be able to deploy IAM permissions.

[froilan]

@chrisjgray
Were you able to work around this limitation?

[rix0rrr]

Am I correct in understanding that you are going to evaluate, before each deployment, what resources are going to be changed and then update the permissions of the role to only have the permissions that the CloudFormation deployment would need?

Or do you group different types of resources in different apps that get deployed separately, by different people? As in, there is a team over here that only deploys IAM roles, and another team over there that only deploys Lambdas?

And you used to have your own mechanism for users to assume roles with appropriate permissions for the deployments?

In the mean time, you can use the LegacyStackSynthesizer to fall back to the old behavior where the CLI will use the "current CLI credentials" for all operations:

new MyStack(this, 'MyStack', {
  synthesizer: new cdk.LegacyStackSynthesizer()
});

[souravsaraf123]

this worked , thanks a lot :)

[kaprah]

Or do you group different types of resources in different apps that get deployed separately, by different people? As in, there is a team over here that only deploys IAM roles, and another team over there that only deploys Lambdas?

So we have it separated by particular services, so one particular service might have IAM, lambda, sqs, and s3 permissions and that CICD process will use that particular role based off of the repo they are working out of.

Another service will have its own deploy role with its individual permissions such as s3, and IAM.

These are tied into the CICD process and the repo with where these services exist and teams can't specify other deploy roles and essentially tie them to the repo they are working in.

[kaprah]

In the mean time, you can use the LegacyStackSynthesizer to fall back to the old behavior where the CLI will use the "current CLI credentials" for all operations:

We are also validating this option and it seems to be a potential solution for stacks that have not been deployed. For those that have been deployed it seems like the only option is updating the bootstrap with additional IAM pass role permissions, to allow us to specify a role arn on the deployment, then switch them over to using the legacy stack synthesizer since there is not a way to un-associate an IAM role from being the service role on the cloudformation stack.

[tmokmss]

fyi you can specify CDK bootstrap stack name by --toolkit-stack-name for each bootstrap and deploy. Does it help?

cdk bootstrap --toolkit-stack-name cdkbootstrap --qualifier test
cdk deploy TestStack --toolkit-stack-name cdkbootstrap

[chrisjgray]

Unfortunately, no this doesn't help out at the grand scale.

[Ophirr33]

Just chiming in here as a data point. Thanks @rix0rrr for pointing out that the LegacySynthesizer exists, as it was the only way for me to get deployments working for our v1 -> v2 migration. The DefaultSynthesizer makes a lot of assumptions around what resources can be created as part of the bootstrap stack, and it requires a large amount of customization to get them to work within a locked down aws environment (ultimately, I was unable to figure out how to get this working correctly).

[alexpulver]

AWS CDK 2.13.0 (2022-02-18) introduced CliCredentialsStackSynthesizer that might help to address this. From the docs:

A synthesizer that uses conventional asset locations, but not conventional deployment roles.

Instead of assuming the bootstrapped deployment roles, all stack operations will be performed using the CLI's current credentials.

• This synthesizer does not support deploying to accounts to which the CLI does not have credentials. It also does not support deploying using CDK Pipelines. For either of those features, use DefaultStackSynthesizer.
• This synthesizer requires an S3 bucket and ECR repository with well-known names. To not depend on those, use LegacyStackSynthesizer.

[emmamoinat]

Interesting points and great to see some ideas of how to approach this. I have written a blog explaining more about the permissions you need in cdk which has a section on the exec role and its issues.

https://dev.to/aws-builders/cdk-principle-of-least-privilege-45i4

Thinking of writing up something around using IAM Access Analyzer to help generate the least privileges for the exec role. Thoughts?

[rix0rrr]

We also have a Wiki page about this topic: https://github.com/aws/aws-cdk/wiki/Security-And-Safety-Dev-Guide

[emmamoinat]

Ahh nice!