GCP Artifact Registry dynamic Clean-up policies using Terraform

Artifact Registry is an important service within GCP that is used to store and manage your packaged and docker container images. 

There are 3 components based on which Artifact registry is billed for the associated billing account:

• Storage
• Data Transfer
• Vulnerability Scanning 

Read more about this here.

Objective of this post is to go through a use case of optimizing the cost of storage component of Artifact Registry by applying the clean-up policies.

An Artifact Registry cleanup policy defines criteria for automatically deleting artifact versions that you no longer need or keeping artifacts that you want to store indefinitely.

Cleanup policies are useful if you store many versions of your artifacts, but only need to keep specific versions that you release to production. You can define delete policies with criteria for deleting artifacts and keep policies with criteria for retaining artifacts.

If an artifact version matches criteria in both a delete policy and a keep policy, Artifact Registry applies the keep policy.

Deletions triggered by delete policies count against your Artifact Registry per project delete request quota and are limited to 300,000 deletions per repository, per day.

Let us now consider a scenario where we have below requirement for storage of docker images within Artifact Registry across different projects:

We will make use of terraform resource "google_artifact_registry_repository" to apply these policies:

resource "google_artifact_registry_repository" "artifact-repo" {
  project = var.project
  location      = var.region
  repository_id = "your-repository-id"
  description   = "Description about your reporsitory"
  format        = "DOCKER"

# setting this to true prevents all tags from being modified, moved or deleted  
docker_config {
    immutable_tags = false
  }

# this property is essentially used to test the clean-up policies before actually applying it
  cleanup_policy_dry_run = false
  
 # Policy number 1 to delete untagged images from the repository
 cleanup_policies {
    id     = "delete-untagged"
    action = "DELETE"
    condition {
      tag_state    = "UNTAGGED" 
    }
  }

 # Policy number 2 to keep images tagged with "recent"
  cleanup_policies {
    id     = "keep-recent-tagged"
    action = "KEEP"
    condition {
      tag_state             = "TAGGED"
      tag_prefixes            = ["recent"]    
    }
  }

  # Policy number 3 to delete tagged images except "recent" after 7 days but only for projects ending with test. We have used terraform dynamic module to set up this conditionally
  dynamic "cleanup_policies" {
    content {
      id     = "delete-other-tagged-after-7-days"
      action = "DELETE"
      condition {
        tag_state         = "TAGGED"
        older_than        = "604800s"      
      }
    }
   
   # Ensure the policy is applied only on the projects  containing test
   check_flag = contains(project, "test") # you can use any other terraform feature like for each to derive the flag indicating if your project names contains test
    for_each = check_flag ? [1] : [])
  }
}
        

Summary:

Effective use of clean up policies can save a significant amount of cost for artifact registry and sing terraform lets you seamlessly set up and manage configuration of multiple projects at once.