CI Build Time Optimization: Caching, Parallelism & Profiling

Dependency Caching with actions/cache

Caching node_modules or the npm cache directory eliminates the most time-consuming step in most Node.js pipelines. Key the cache on your lockfile checksum so it invalidates automatically when dependencies change.

# GitHub Actions: npm dependency caching
- name: Cache npm dependencies
  uses: actions/cache@v4
  with:
    path: ~/.npm
    key: npm-${{ runner.os }}-${{ hashFiles('**/package-lock.json') }}
    restore-keys: |
      npm-${{ runner.os }}-

- name: Install dependencies
  run: npm ci

# Python pip caching
- name: Cache pip packages
  uses: actions/cache@v4
  with:
    path: ~/.cache/pip
    key: pip-${{ runner.os }}-${{ hashFiles('**/requirements*.txt') }}
    restore-keys: pip-${{ runner.os }}-

# Go module caching
- name: Cache Go modules
  uses: actions/cache@v4
  with:
    path: |
      ~/go/pkg/mod
      ~/.cache/go-build
    key: go-${{ runner.os }}-${{ hashFiles('**/go.sum') }}

Docker Layer Caching

Order your Dockerfile statements to maximize layer cache hits. Dependency installation layers (slow, rarely change) must come before application code layers (fast, change every commit). With cache-to/cache-from, GitHub Actions can reuse layers from previous runs via the registry.

# Optimized Dockerfile: COPY package files BEFORE source
FROM node:20-alpine AS deps
WORKDIR /app
COPY package.json package-lock.json ./   # cached unless package.json changes
RUN npm ci --only=production

FROM node:20-alpine AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .                                  # source code (cache miss on every commit)
RUN npm run build

# GitHub Actions: registry-based layer caching
- name: Build with layer cache
  uses: docker/build-push-action@v5
  with:
    push: true
    tags: myorg/myapp:${{ github.sha }}
    cache-from: type=registry,ref=myorg/myapp:buildcache
    cache-to: type=registry,ref=myorg/myapp:buildcache,mode=max

Test Splitting with Matrix Strategy

Split your test suite across multiple runners using GitHub Actions matrix. Combined with Jest's --shard flag or pytest-split, you can run 4x test workers in parallel, cutting test time from 12 minutes to 3 minutes with zero code changes.

jobs:
  test:
    strategy:
      matrix:
        shard: [1, 2, 3, 4]   # 4 parallel test runners
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: '20' }
      - uses: actions/cache@v4
        with:
          path: ~/.npm
          key: npm-${{ hashFiles('package-lock.json') }}
      - run: npm ci
      - name: Run shard ${{ matrix.shard }} of 4
        run: npx jest --shard=${{ matrix.shard }}/4 --ci

  # Merge sharded results into one report
  coverage:
    needs: test
    runs-on: ubuntu-latest
    steps:
      - run: npx jest --coverage --merge-coverage

Build Time Benchmark: Before vs After Optimization