Container Layer Caching Strategies for Test Performance

A slow Docker build is a tax on every developer and every CI run. When building a Docker image takes four minutes and your test pipeline runs it twenty times a day across ten engineers, that is 13 hours of waiting — every day. Most of that time is unnecessary. Docker's layer caching mechanism can eliminate the majority of it, but only if you structure your Dockerfile to take advantage of it.

This post explains how layer caching works, how to structure Dockerfiles for maximum cache hits, how BuildKit cache mounts go beyond standard caching, and how to configure GitHub Actions and GitLab CI for cache sharing across runners.

How Docker Layer Caching Works

Docker builds images in layers. Each instruction in a Dockerfile — FROM, RUN, COPY, ADD — creates a new layer. Docker computes a cache key for each layer based on the instruction itself and the layer below it. If the cache key matches a previously built layer, Docker reuses it. If any layer's cache key changes, every subsequent layer is rebuilt.

This cascade behavior is the fundamental constraint you must design around. The moment a layer is invalidated, everything after it is rebuilt from scratch. This is why instruction order in the Dockerfile matters enormously.

Consider this naive Dockerfile:

FROM node:20-alpine
WORKDIR /app
COPY . .
RUN npm ci
CMD ["node", "server.js"]

Every time any file in the project changes — including a comment in a markdown file — the COPY . . layer is invalidated, which invalidates RUN npm ci, which forces a full package reinstall. On a project with 500 npm dependencies, this takes 60–90 seconds every time.

Dockerfile Ordering for Maximum Cache Hits

The fix is to copy only what each step needs, in the order that changes least frequently to most frequently:

FROM node:20-alpine
WORKDIR /app

# 1. Copy only package files — changes rarely
COPY package.json package-lock.json ./

# 2. Install dependencies — only reruns when package files change
RUN npm ci

# 3. Copy source code — changes frequently, but only rebuilds the last layer
COPY . .

RUN npm run build
CMD ["node", "dist/server.js"]

Now when you change a source file, only the COPY . . and RUN npm run build layers rebuild. The npm ci layer hits cache every time. For a project with a 90-second install, this saves 90 seconds on every build where you did not change a dependency.

Java Maven Example

FROM eclipse-temurin:21-jdk-alpine AS build
WORKDIR /app

# Copy POM first — changes less often than source
COPY pom.xml ./
COPY .mvn .mvn
COPY mvnw ./

# Download dependencies — cached until pom.xml changes
RUN ./mvnw dependency:go-offline -q

# Copy source and build
COPY src ./src
RUN ./mvnw package -DskipTests -q

FROM eclipse-temurin:21-jre-alpine
WORKDIR /app
COPY --from=build /app/target/*.jar app.jar
CMD ["java", "-jar", "app.jar"]

dependency:go-offline downloads all dependencies declared in pom.xml into the local Maven repository inside the container. This layer only invalidates when pom.xml changes. On subsequent builds where only Java source files changed, dependency download is skipped entirely.

Gradle Example

FROM eclipse-temurin:21-jdk-alpine AS build
WORKDIR /app

COPY build.gradle settings.gradle gradle.properties ./
COPY gradle ./gradle
COPY gradlew ./

# Cache dependencies
RUN ./gradlew dependencies --no-daemon -q

COPY src ./src
RUN ./gradlew build --no-daemon -x test -q

FROM eclipse-temurin:21-jre-alpine
COPY --from=build /app/build/libs/*.jar app.jar
CMD ["java", "-jar", "app.jar"]

BuildKit Cache Mounts

Standard Docker layer caching is an all-or-nothing affair: the layer either hits cache or it does not. BuildKit cache mounts are more sophisticated. They mount a persistent cache directory during the RUN instruction that is not included in the resulting image layer. The cache persists across builds even when the layer's inputs change.

Enable BuildKit:

export DOCKER_BUILDKIT=1

Or set it permanently in /etc/docker/daemon.json:

{"features": {"buildkit": true}}

npm with BuildKit Cache Mount

FROM node:20-alpine
WORKDIR /app

COPY package.json package-lock.json ./

RUN --mount=type=cache,target=/root/.npm \
    npm ci --cache /root/.npm

COPY . .
RUN npm run build

The --mount=type=cache,target=/root/.npm mounts a persistent cache at /root/.npm. When npm downloads packages, they are stored there. On the next build — even if package-lock.json changed and the layer itself invalidates — npm finds the downloaded tarballs in the cache and only downloads what is new. For a project that added two dependencies, only two packages are fetched instead of reinstalling all 500.

Maven with BuildKit Cache Mount

FROM eclipse-temurin:21-jdk-alpine AS build
WORKDIR /app

COPY pom.xml ./
COPY mvnw ./
COPY .mvn .mvn

RUN --mount=type=cache,target=/root/.m2 \
    ./mvnw dependency:go-offline -q

COPY src ./src

RUN --mount=type=cache,target=/root/.m2 \
    ./mvnw package -DskipTests -q

The Maven local repository at /root/.m2 persists between builds. Even if pom.xml changes completely, all previously downloaded JARs remain available. Only truly new dependencies trigger network downloads.

pip with BuildKit Cache Mount

FROM python:3.12-slim
WORKDIR /app

COPY requirements.txt ./

RUN --mount=type=cache,target=/root/.cache/pip \
    pip install -r requirements.txt

COPY . .
CMD ["python", "app.py"]

GitHub Actions Cache Strategies

GitHub Actions runners are ephemeral — each job starts with a clean machine and no Docker layer cache. Without explicit cache configuration, every build is a cold build. Two approaches fix this.

Registry Cache (Recommended)

Use a container registry as the cache store. GitHub Actions can push and pull cache layers to GitHub Container Registry (ghcr.io):

name: Build and Test

on: [push, pull_request]

jobs:
  build:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      packages: write

    steps:
      - uses: actions/checkout@v4

      - name: Log in to GitHub Container Registry
        uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Build image with cache
        uses: docker/build-push-action@v5
        with:
          context: .
          push: false
          load: true
          tags: myapp:${{ github.sha }}
          cache-from: type=registry,ref=ghcr.io/${{ github.repository }}/cache:buildcache
          cache-to: type=registry,ref=ghcr.io/${{ github.repository }}/cache:buildcache,mode=max

      - name: Run tests
        run: docker run --rm myapp:${{ github.sha }} npm test

mode=max exports all intermediate layers to the cache, not just the final image. This maximizes cache hits for subsequent builds.

The cache-from reference points to the same cache tag that cache-to writes. On the first run, the cache does not exist and the build is cold. On every subsequent run, Docker pulls the cached layers from the registry and reuses them.

GitHub Actions Cache (Alternative)

For projects where pushing to a registry adds complexity, the actions/cache approach works with local BuildKit cache:

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Cache Docker layers
        uses: actions/cache@v4
        with:
          path: /tmp/.buildx-cache
          key: buildx-${{ github.ref }}-${{ github.sha }}
          restore-keys: |
            buildx-${{ github.ref }}-
            buildx-

      - name: Build image
        uses: docker/build-push-action@v5
        with:
          context: .
          load: true
          tags: myapp:${{ github.sha }}
          cache-from: type=local,src=/tmp/.buildx-cache
          cache-to: type=local,dest=/tmp/.buildx-cache-new,mode=max

      # Prevent cache from growing unbounded
      - name: Move cache
        run: |
          rm -rf /tmp/.buildx-cache
          mv /tmp/.buildx-cache-new /tmp/.buildx-cache

The cache key hierarchy (buildx-$ref-$sha, buildx-$ref-, buildx-) means a branch first tries to restore its own previous cache, then falls back to the same branch without a specific SHA, then falls back to any cache. Pull requests restore from the main branch cache, so they start warm.

GitLab CI Cache Strategy

GitLab CI uses a different caching mechanism. Docker layer caching integrates through the registry:

build:
  image: docker:24
  services:
    - docker:24-dind
  variables:
    DOCKER_BUILDKIT: "1"
    IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
    CACHE_TAG: $CI_REGISTRY_IMAGE:cache
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  script:
    - docker buildx create --use
    - docker buildx build
        --cache-from type=registry,ref=$CACHE_TAG
        --cache-to type=registry,ref=$CACHE_TAG,mode=max
        --tag $IMAGE_TAG
        --load
        .
    - docker run --rm $IMAGE_TAG npm test

GitLab's built-in container registry serves as the cache store with no additional configuration.

Measuring Cache Effectiveness

Docker outputs cache hit information during builds. To see it clearly, use --progress=plain:

DOCKER_BUILDKIT=1 docker build --progress=plain -t myapp:latest . 2>&1 | grep -E "CACHED|RUN"

Output with good caching:

#4 CACHED
#5 CACHED
#6 CACHED
#7 0.312s RUN npm run build

Output with poor caching (pom.xml changed):

#4 CACHED
#5 12.4s RUN ./mvnw dependency:go-offline
#6 34.7s RUN ./mvnw package

Track build times per layer to identify which instructions are cache misses and cost the most time. The goal is to ensure that the most expensive steps — dependency installation — are cached on every build that does not change dependencies.

For CI pipelines, add timing around the build step:

time docker buildx build --cache-from ... --cache-to ... -t myapp:latest .

Baseline a cold build, then measure warm builds. A well-configured pipeline should achieve 80–90% cache hit rate on typical development commits, reducing a 4-minute build to under 60 seconds.

Common Cache Pitfalls

Secrets in build args invalidate cache. If you pass a secret as a build arg — even if it does not change — Docker includes it in the cache key. Use BuildKit secrets instead:

RUN --mount=type=secret,id=npm_token \
    NPM_TOKEN=$(cat /run/secrets/npm_token) npm ci

Cache grows unbounded with local caching. The /tmp/.buildx-cache approach accumulates layers over time. The rm -rf and mv pattern shown earlier resets the cache to only what the latest build wrote, preventing unbounded growth.

Different base image digests miss cache. If you use FROM node:20-alpine without pinning the digest, a registry update to that tag (for a security patch) invalidates your entire cache. Consider pinning to a digest for production builds: FROM node:20-alpine@sha256:abc123.... For test images, this trade-off may not be worth it.

Wrapping Up

Layer caching is the highest-leverage optimization available in a Docker-based test pipeline. Ordering Dockerfile instructions correctly — dependencies before source code — alone eliminates the most expensive cache misses. Adding BuildKit cache mounts eliminates network downloads for incremental dependency changes. Registry-based cache sharing in CI means every build starts warm instead of cold. Together these strategies can reduce a four-minute build to under one minute for the vast majority of commits.