CI/CD Without Internet: ECR Pull Through Cache for Private Builds

In today’s cloud-native development landscape, CI/CD pipelines heavily rely on container images, many sourced from public registries like Docker Hub or Quay. But what happens when your workloads reside in private subnets, isolated VPCs, or even air-gapped environments without internet access?

Do you compromise security to fetch base images?

You don’t have to.

Enter AWS ECR Pull Through Cache, a native solution that allows you to securely access both public and private container images without direct internet connectivity.

📚 This guide is at an beginner level difficulty. 📚

The Challenge: Accessing Images in Isolated Networks

Imagine your CI/CD runners or ECS tasks operating within private subnets—a common scenario in high-security or regulated environments. These instances:

• Lack internet gateways
• Cannot directly access external registries like Docker Hub or private ECR repositories
• Reached external Docker image registry pull rate limit
• Shouldn’t rely on NAT gateways solely for image retrieval

Yet, your pipelines need base images such as:

• node:18-alpine
• python:3.12-slim
• ubuntu:22.04

This creates a conflict between maintaining strict security and ensuring efficient pipeline operations.

The Solution: AWS ECR Pull Through Cache

ECR Pull Through Cache enables your AWS account to act as a private mirror for both public and private registries, including Docker Hub, Quay, and even other ECR repositories. When a private VPC attempts to pull an image, it retrieves it from your ECR repository, eliminating the need for direct internet access.

Key Benefits:

• 🔒 Enhanced Security: No public internet access required
• 🔑 No External Registry Rate Limits: Use images from external registries without exhausting pull limits
• 🚀 Improved Performance: Cache frequently used images close to your workloads
• 💸 Cost Efficiency: Reduce or eliminate NAT gateway usage
• 🛡️ Centralized Control: Audit, restrict, and manage image usage effectively

How It Works

1. Create a Pull Through Cache Rule: Define a rule in ECR that specifies the upstream registry (e.g., Docker Hub, Quay, or another ECR repository) and a namespace prefix.
2. Configure Authentication: For private registries, store credentials in AWS Secrets Manager with the prefix ecr-pullthroughcache/. For ECR-to-ECR caching, set up appropriate IAM roles and permissions.
3. Pull Images Using ECR URI: Use the ECR URI with the specified namespace to pull images. For example:

docker pull <account-id>.dkr.ecr.<region>.amazonaws.com/<namespace>/node:18-alpine

On the first pull, ECR fetches and caches the image. Subsequent pulls retrieve the image directly from your ECR cache.

Accessing ECR in Private VPCs

To enable access to ECR from isolated environments:

• Set Up VPC Interface Endpoints: Configure the following interface endpoints in your VPC:
  • com.amazonaws.<region>.ecr.api
  • com.amazonaws.<region>.ecr.dkr
  • com.amazonaws.<region>.s3 (required for image layers)

This setup allows services like EC2, ECS, or CodeBuild within private subnets to interact with ECR without internet access.

Example: CodeBuild in a Private Subnet

phases:
  build:
    commands:
      - docker pull <account-id>.dkr.ecr.<region>.amazonaws.com/<namespace>/python:3.12-slim
      - docker build -t myapp .
      - docker push <your-private-ecr>

With the pull through cache and VPC endpoints configured, this process operates seamlessly within an isolated subnet.

Supporting Private Registries

ECR Pull Through Cache now supports private registries, including:

• Docker Hub
• Quay
• GitHub Container Registry
• GitLab Container Registry
• Azure Container Registry
• Amazon ECR (cross-account and cross-region)

For private registries requiring authentication:

• Store Credentials in AWS Secrets Manager: Create a secret with the prefix ecr-pullthroughcache/ containing your registry credentials.
• Configure IAM Roles for ECR-to-ECR Caching: Set up IAM roles and permissions to allow ECR to access upstream private ECR repositories.

Bonus: Creating a Pull Through Cache Rule in Terraform

resource "aws_ecr_pull_through_cache_rule" "dockerhub" {
  ecr_repository_prefix = "dockerhub"
  upstream_registry_url = "https://registry-1.docker.io"
  credential_arn        = "arn:aws:secretsmanager:us-east-1:123456789:secret:ecr-pullthroughcache/dockerhub"
}

Conclusion

For teams operating in secure, isolated AWS environments, AWS ECR Pull Through Cache offers a robust solution to access necessary container images without compromising security or efficiency. By caching images from both public and private registries within your AWS account, you ensure faster builds, reduced costs, and enhanced control over your CI/CD pipelines.

If you found this guide helpful, there’s plenty more on the way—ranging from beginner-friendly walkthroughs to deep-dive DevOps strategies.
Subscribe to my newsletter below to get notified whenever a new post drops—no spam, just practical cloud insights.

Need hands-on help with your Cloud, DevOps, or AWS projects?
👉 Visit my homepage to learn how I can support your next initiative and bring real value to your team.