1 Introduction

1.1 IaC

Infrastructure as code(IaC) is the managing and provisioning of infrastructure through code instead of manual processes, for example, clicking button, adding or editing roles in AWS console.

1.2 AWS CloudFormation

AWS CloudFormation is the original IaC tool for AWS, released in 2011, which uses template files to automate and mange the setup of AWS resources.

1.3 AWS CDK

AWS Cloud Development Kit(CDK) is a product provided by AWS that makes it easier for developers to manage their infrastructure with familiar programming languages like TypeScript, Python, Java, etc.

And, CDK is standing on the shoulder of Cloudformation, providing tools for developers by leveraging Cloudformation.

A stack is a collection of AWS resources that you can manage as a single unit, like a box.

For instance, this box could include all the resources required to run an application or Lambda service, such as S3 Buckets (storage), Roles (authorization), Lambda Function (computing), API Gateway (access point), Alarm, Monitoring, etc.

2 Problem

I am currently working on a project which requires to set up two stacks, one stack( GlueStack ) for defining a list of AWS Glue tables and the other stack( ServiceStack ) for definition of Lambda service and associated resources.

In fact, S3 bucket names have to be globally unique within a partition, which means crossing the whole AWS customer base.

You are unable to create a S3 bucket with bucket name which is in use by another AWS customer or your own account.

So it’s safer to let CloudFormation generate a random bucket name for a developer when he need to initialize a S3 bucket.

However, there is new a problem I face: since the S3 bucket name is randomly generated characters, if GlueStack need to read the bucket created by ServiceStack, how could I share the bucket name between two stacks?

While these two stacks are isolated and separated, resources collection.

3 Solution

Fortunately, CDK offers a facility named CfnOutput to export a deployed resource, so that the consumer of the resource is able to Import required resource.

  1. Define the required resource in ServiceStack (producer), for instance, a S3 bucket:
    import { Bucket } from 'aws-cdk-lib/aws-s3';

const s3Bucket = new Bucket(this, 'MyBucketId', {});
  2. Export the resource by specifying the value and exportName:
    import { CfnOutput } from 'aws-cdk-lib';

// export the generated bucket name to other stack
new CfnOutput(this, 'exportRequiredS3Bucket', {
    value: s3Bucket.bucketName,
    exportName: 'exportRequiredS3Bucket',
});
  3. Import the required resource in GlueStack (consumer):
    import { Fn} from 'aws-cdk-lib';

const requiredS3BucketName = Fn.importValue('exportRequiredS3Bucket');

If we take a closer look at the synthesized CFN template for ServiceStack, we could find:

"Outputs": {
    "exportRequiredS3Bucket": {
        "Value": {
            "Ref": "MyBucketId737FC949"
        },
        "Export": {
            "Name": "exportRequiredS3Bucket"
        }
    }

The synthesized CFN template for GlueStack:

{
    "Fn::ImportValue": "exportRequiredS3Bucket"
}

This is the way about how to share value between two stacks.

4 Loose couping solution

Updated on 2023-12-02

People learn from mistake.

After applying this practice in my project, I recently learn that it’s not good practice to share resource across stack.

With using export/import, I tightly couple my stacks with a commitment that I can never update that unless I remove that couping later on.

It means it will become a disaster1 whenever I need to update/delete the S3Bucket, CloudFormation will raise an error, complaining something like: “ServiceStack cannot be deleted as it’s in use by GlueStack”.

A better practice I learnt is adding a loose couping between ServiceStack and GlueStack by sharing a constant variable:

  1. Define a constant variable somewhere:

    export const Constants = {
    MyBucketName: 'TestBucket'
}

  2. Refine the definition of s3Bucket

    import { Bucket } from 'aws-cdk-lib/aws-s3';

const s3Bucket = new Bucket(this, 'MyBucketId', {
    bucketName: Constants.MyBucketName,
});

  3. Refer the s3Bucket in GlueStack by MyBucketName instead of CDK exported reference

    const requiredS3BucketName = Constants.MyBucketName;

Therefore, these two stacks are not directly coupled, but they are referencing the same constant variable.

Then, CloudFormation won’t prevent you from updating the S3Bucket as there is not direct relation between these two stacks anymore.

This is the benefit of loose couping.

5 Reference

  1. https://stackoverflow.com/questions/63350346/delete-resource-with-references ↩︎