# Leveraging Cloud Custodian to shift left tag compliance

Tagging is one of the capability that included in Establishing Your Cloud Foundation on AWS [whitepaper](https://docs.aws.amazon.com/whitepapers/latest/establishing-your-cloud-foundation-on-aws/governance.html) under category Governance, Risk Management, and Compliance.

> **Tagging** enables you to group sets of resources by assigning metadata to cloud resources for a variety of purposes. These purposes include access control (such as ABAC), cost reporting, and automation (such as patching for select tagged instances). Tagging can also be used to create new resource constructs for visibility or control (such as grouping together resources that make up a microservice, application, or workload). Tagging is fundamental to providing enterprise-level visibility and control.

There are several use cases why we need tagging as explained in this document “[Tagging use cases](https://docs.aws.amazon.com/whitepapers/latest/tagging-best-practices/tagging-use-cases.html)“, among them:

1. Tags for cost allocation and financial management
    
2. Tags for operations and support
    
3. Tags for data security, risk management, and access control
    

In this blog, I will explain how to shift left tag compliance check inside CI/CD. I leverage Cloud Custodian package called c7n-left to run policy directly against Infrastructure as Code which in this case is Terraform. If tag compliance check is passed then Terraform core workflow which consist plan and apply can be continued. Otherwise, if tag compliance check is failed then Terraform core workflow cannot be continued. CI/CD tool that I used in this blog is Amazon CodeCatalyst.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1737996890235/5bb19798-2ff1-41b2-8710-0532a21347f8.png align="center")

> Disclaimer: All contents written in this blog are solely my personal opinion and doesn’t represent my employer opinion

## Prerequisites

Here are the prerequisites that needed to follow configurations discussed in this blog:

1. AWS account
    
2. CodeCatalyst space and project that has been connected with AWS account and source repository. In this blog, I use GitHub as source repository. For configuration reference, please check my other blog "[**Build and Release Container Image to Amazon Elastic Container Registry (ECR) via Amazon CodeCatalyst**](https://medium.com/aws-in-plain-english/build-and-release-container-image-to-amazon-elastic-container-registry-ecr-via-amazon-ae7f9f07a135)"
    
3. S3 bucket and DynamoDB table that will be used for Terraform backend
    

## **Configuration steps**

### **Directory structure**

Below is directory structure used in this blog:

```bash
.
├── .codecatalyst/
│   └── workflows/
│       └── tf-sandbox-sbx0-ec2.yml
├── custodian/
│   └── tf-tag-compliance.yml
└── sandbox/
    └── sbx0/
        └── ec2/
            ├── backend.tf
            ├── main.tf
            ├── terraform.tfvars
            └── variables.tf
```

### Custodian policy

We will start discussion with policy that will be used to evaluate Infrastructure as Code by Cloud Custodian

1. Under directory `custodian`, I created file called `tf-tag-compliance.yml`. Inside that file there is a policy named `check-tag` that will check all Terraform codes that related to AWS resources. Policy will check if tag called `lanandra:environment` is absent or not on all resources that taggable. I also added metadata to that policy
    

```yaml
policies:
  - name: check-tag
    resource: "terraform.aws*"
    metadata:
      category: [tag]
      severity: low
    filters:
      - taggable
      - or:
        - type: value
          key: "tag:lanandra:environment"
          value: absent
```

### Sample Terraform code

As an example, I created two EC2 instances, one that has tag `lanandra:environment` and the other doesn’t

1. Create Terraform files inside directory `sandbox/sbx0/ec2` , main resources such as EC2 are defined inside `main.tf` file
    
2. EC2 that has tag `lanandra:environment` , this tag is got by merging local tags.
    

```go
locals {
  azs = slice(data.aws_availability_zones.available.names, 0, 3)

  tags = {
    "lanandra:environment" = "${var.env_name}"
    "lanandra:managedBy"   = "terraform"
  }
}

module "lab_ec2_0" {
  source  = "terraform-aws-modules/ec2-instance/aws"
  version = "5.6.1"

  name = "${var.env_name}-lab-0"

  ami                         = data.aws_ami.ubuntu_22_04.id
  instance_type               = "t3a.nano"
  availability_zone           = element(local.azs, 0)
  subnet_id                   = element(var.public_subnet_id, 0)
  vpc_security_group_ids      = [module.lab_sg.security_group_id]
  associate_public_ip_address = true
  key_name                    = var.env_name
  iam_instance_profile        = module.lab_instance_profile.iam_instance_profile_name

  enable_volume_tags = false
  root_block_device = [
    {
      encrypted   = true
      volume_type = "gp3"
      volume_size = 10
      tags = merge(
        local.tags,
        {
          Name = "${var.env_name}-lab-0"
        }
      )
    }
  ]

  tags = merge(
    local.tags,
    {
      Name = "${var.env_name}-lab-0"
    }
  )
}
```

2. EC2 that hasn’t got tag `lanandra:environment`
    

```go
module "lab_ec2_1" {
  source  = "terraform-aws-modules/ec2-instance/aws"
  version = "5.6.1"

  name = "${var.env_name}-lab-1"

  ami                         = data.aws_ami.ubuntu_22_04.id
  instance_type               = "t3a.nano"
  availability_zone           = element(local.azs, 0)
  subnet_id                   = element(var.public_subnet_id, 0)
  vpc_security_group_ids      = [module.lab_sg.security_group_id]
  associate_public_ip_address = true
  key_name                    = var.env_name
  iam_instance_profile        = module.lab_instance_profile.iam_instance_profile_name

  enable_volume_tags = false
  root_block_device = [
    {
      encrypted   = true
      volume_type = "gp3"
      volume_size = 10
      tags = {
        Name = "${var.env_name}-lab-1"
      }
    }
  ]

  tags = {
    Name = "${var.env_name}-lab-1"
  }
}
```

3. For full context of Terraform codes/files, please refer to this [**directory**](https://github.com/lanandra/mock-aws-infrastructure/tree/master/sandbox/sbx0/ec2)
    

### Run tag compliance in CI/CD pipeline

As mentioned in the beginning of this blog, I use Amazon CodeCatalyst as tool for CI/CD pipeline.

1. I created a workflow file called `tf-sandbox-sbx0-ec2.yml` and I put that inside `.codecatalyst/workflow` directory. Inside workflow/pipeline, I created action called `tag-compliance`. This action use container image `python:3.12.8-bookworm` that being pulled from Dockerhub. First, it will redirect working directory to ec2 directory (`sandbox/sbx0/ec2`). Then it will run several steps that needed to run tag compliance check using Cloud Custodian. In this case, I use specific package called `c7n-left` that designed to evaluate policies directly against infrastructure as code source assets (Please see [Custodian policies for Infrastructure Code](https://cloudcustodian.io/docs/tools/c7n-left.html#custodian-policies-for-infrastructure-code) for more details). Then it will run command that will find custodian policy inside `custodian` directory with category value filtered to tag
    

```yaml
  tag-compliance:
    Identifier: aws/build@v1
    Inputs:
      Sources:
        - WorkflowSource
    Environment:
      Name: sandbox
      Connections:
        - Name: lanandra-sandbox
          Role: tf-codecatalyst-admin-sandbox
    Configuration:
      Container:
        Registry: DockerHub
        Image: python:3.12.8-bookworm
      Steps:
      - Run: cd sandbox/sbx0/ec2
      - Run: pip install c7n
      - Run: pip install c7n-left
      - Run: c7n-left run -p ../../../custodian/ -d . --filters="category=tag"
    Compute:
      Type: EC2
```

2. In workflow file, tag compliance action will be merged with other actions that used by Terraform core workflow. For full context, please see this [file](https://github.com/lanandra/mock-aws-infrastructure/blob/master/.codecatalyst/workflows/tf-sandbox-sbx0-ec2.yml):
    

```yaml
Name: tf-sandbox-sbx0-ec2
SchemaVersion: "1.0"

Triggers:
  - Type: PUSH
    Branches:
      - master
    FilesChanged:
      - sandbox\/sbx0\/ec2\/.*

Actions:
  tag-compliance:
    Identifier: aws/build@v1
    Inputs:
      Sources:
        - WorkflowSource
    Environment:
      Name: sandbox
      Connections:
        - Name: lanandra-sandbox
          Role: tf-codecatalyst-admin-sandbox
    Configuration:
      Container:
        Registry: DockerHub
        Image: python:3.12.8-bookworm
      Steps:
      - Run: cd sandbox/sbx0/ec2
      - Run: pip install c7n
      - Run: pip install c7n-left
      - Run: c7n-left run -p ../../../custodian/ -d . --filters="category=tag"
    Compute:
      Type: EC2
  terraform-plan:
    Identifier: aws/build@v1
    DependsOn:
      - tag-compliance
    Inputs:
      Sources:
        - WorkflowSource
    Environment:
      Name: sandbox
      Connections:
        - Name: lanandra-sandbox
          Role: tf-codecatalyst-admin-sandbox
    Configuration:
      Container:
        Registry: DockerHub
        Image: hashicorp/terraform:1.8.2
      Steps:
      - Run: cd sandbox/sbx0/ec2
      - Run: terraform fmt -check -no-color
      - Run: terraform init -no-color
      - Run: terraform validate -no-color
      - Run: terraform plan -no-color -input=false
    Compute:
      Type: EC2
  wait-for-approval:
    Identifier: aws/approval@v1
    DependsOn:
      - terraform-plan
    Configuration:
      ApprovalsRequired: 1
  terraform-apply:
    DependsOn:
      - wait-for-approval
    Identifier: aws/build@v1
    Inputs:
      Sources:
        - WorkflowSource
    Environment:
      Name: sandbox
      Connections:
        - Name: lanandra-sandbox
          Role: tf-codecatalyst-admin-sandbox
    Configuration:
      Container:
        Registry: DockerHub
        Image: hashicorp/terraform:1.8.2
      Steps:
      - Run: cd sandbox/sbx0/ec2
      - Run: terraform init -no-color
      - Run: terraform apply -auto-approve -no-color -input=false
    Compute:
      Type: EC2
```

3. We will simulate to provision EC2 with tag that needed by tag compliance first. Tag compliance action will run first as defined in workflow. Cause tag compliance is passed, then Terraform core workflow can be continued. Please see detailed logs of tag compliance to see the result
    

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1738390760380/9f555db9-9dce-42ee-9f70-296dc2b79b5b.png align="center")

4. Here is the example result of successful tag compliance check using c7n-left. It said no failed result
    

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1738390915813/7de58c09-649d-4ea9-80de-e32afd85dc87.png align="center")

5. As explained in point number 3, if tag compliance action is passed, then Terraform core workflow can be continued. So that it can run terraform apply to provision EC2
    

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1738406329729/30283eb8-b599-4572-b58f-0f870913e5a3.png align="center")

6. For next example, let’s see what if we want to provision EC2 that hasn’t got tag that needed by tag compliance action. In CI/CD, tag compliance action will be failed and the rest of Terraform core workflow cannot be continued. Please see detailed logs of tag compliance action for further details
    

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1738406416295/8feb1956-0228-4016-83b9-0d56a62809a3.png align="center")

7. Custodian will inform which file(s) and line(s) that has got policy violation
    

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1738406612303/7dfa0563-e338-4874-a918-3dea7fe8bea2.png align="center")

8. Custodian will also inform how many resources that failed and got policy violation and how many resources that passed in evaluation summary
    

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1738410774220/82064f8b-e52b-41ab-9fb0-725ebc489ecc.png align="center")

## Summary

We have reached the last section of this blog. Here are some key takeaways that can be summarized:

1. We can shift left our tag compliance check inside CI/CD
    
    1. Cloud Custodian is one of the tool that can be leveraged to run tag compliance check
        
    2. Cloud Custodian has specific package called c7n-left to run policy check directly against Infrastructure code
        

Please comment if you have any suggestions, critiques, or thoughts.

Hope this article will benefit you. Thank you!