Security in CI/CD: Pipeline Hardening
Your CI/CD pipeline is probably the most privileged system in your organization. It has access to your source code, production credentials, deployment infrastructure, and package registries. When...
Key Insights
- CI/CD pipelines are high-value targets because they have privileged access to production systems, secrets, and source code—a single compromised pipeline can lead to supply chain attacks affecting thousands of downstream users.
- Defense in depth requires layering multiple security controls: OIDC authentication over static credentials, pinned dependencies with hash verification, least-privilege permissions, and cryptographic signing of artifacts.
- Most pipeline breaches exploit misconfigurations rather than zero-days—hardening your pipeline is primarily about discipline and defaults, not exotic security tools.
The Attack Surface of Modern Pipelines
Your CI/CD pipeline is probably the most privileged system in your organization. It has access to your source code, production credentials, deployment infrastructure, and package registries. When attackers compromised SolarWinds, they didn’t exploit a runtime vulnerability—they poisoned the build pipeline. The same pattern repeated with Codecov, ua-parser-js, and countless other supply chain attacks.
Modern pipelines face three primary attack vectors. Dependency poisoning injects malicious code through compromised packages or typosquatting. Secret exfiltration steals credentials through malicious pull requests or compromised build steps. Build tampering modifies artifacts between build and deployment to inject backdoors.
The uncomfortable truth is that most organizations treat their pipelines as trusted infrastructure when they should treat them as hostile environments. Every build step, every dependency, every external action is a potential attack vector. Let’s fix that.
Secrets Management Best Practices
Hardcoded secrets in repositories remain embarrassingly common. Even when teams use environment variables, they often rely on long-lived credentials that become liability the moment they’re exposed.
The gold standard is eliminating static credentials entirely using OIDC (OpenID Connect) federation. Instead of storing AWS access keys, your pipeline authenticates directly with your cloud provider using short-lived tokens.
# .github/workflows/deploy.yml
name: Deploy to AWS
on:
push:
branches: [main]
permissions:
id-token: write # Required for OIDC
contents: read
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::123456789012:role/GitHubActionsDeployRole
role-session-name: github-actions-deploy
aws-region: us-east-1
# No access keys! OIDC handles authentication
- name: Deploy
run: aws s3 sync ./dist s3://my-bucket
For secrets that must exist, follow these rules: store them in your platform’s native secrets manager (GitHub Secrets, GitLab CI Variables with masking), never echo them in logs, rotate them on a schedule, and scope them to specific environments. Treat any secret that appears in a log as compromised.
Dependency Security and Supply Chain Integrity
Pinning dependencies to exact versions isn’t paranoia—it’s basic hygiene. A floating version tag means you’re trusting every maintainer of every transitive dependency to never make a mistake or get compromised.
# Dockerfile with pinned base image and hash verification
FROM node:20.10.0-alpine3.18@sha256:aad0f326d4e6e6a5e4b8d5f7e8f9a0b1c2d3e4f5a6b7c8d9e0f1a2b3c4d5e6f7
WORKDIR /app
# Copy lockfile first for better caching
COPY package.json package-lock.json ./
# Use ci for reproducible installs, ignore-scripts blocks postinstall attacks
RUN npm ci --ignore-scripts
# Verify no unexpected changes to lockfile
RUN npm audit --audit-level=high
COPY . .
RUN npm run build
# Non-root user
USER node
CMD ["node", "dist/server.js"]
For JavaScript projects, enforce lockfile integrity in your pipeline:
# In your CI workflow
- name: Verify lockfile integrity
run: |
# Ensure lockfile exists and matches package.json
npm ci --ignore-scripts
# Fail if lockfile would change (indicates tampering or drift)
git diff --exit-code package-lock.json
- name: Check for known vulnerabilities
run: npm audit --audit-level=high
- name: Generate SBOM
run: npx @cyclonedx/cyclonedx-npm --output-file sbom.json
Generate a Software Bill of Materials (SBOM) for every build. When the next Log4Shell happens, you’ll know within minutes whether you’re affected.
Least Privilege and Pipeline Isolation
Most pipelines run with far more permissions than they need. GitHub Actions workflows default to broad contents: write permissions. GitLab runners often share credentials across projects. This violates the principle of least privilege.
Explicitly declare minimal permissions for every workflow:
# .github/workflows/ci.yml
name: CI
on:
pull_request:
branches: [main]
# Default to no permissions
permissions: {}
jobs:
test:
runs-on: ubuntu-latest
permissions:
contents: read # Only what's needed to checkout
steps:
- uses: actions/checkout@v4
- run: npm test
lint:
runs-on: ubuntu-latest
permissions:
contents: read
checks: write # Only if writing check annotations
steps:
- uses: actions/checkout@v4
- run: npm run lint
For GitLab, use rules to restrict when jobs run and what they can access:
# .gitlab-ci.yml
stages:
- test
- deploy
variables:
# Disable features you don't need
FF_SCRIPT_SECTIONS: "false"
test:
stage: test
image: node:20-alpine
script:
- npm ci --ignore-scripts
- npm test
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
deploy:
stage: deploy
image: alpine:3.18
script:
- ./deploy.sh
rules:
# Only deploy from main, never from forks or MRs
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH && $CI_PIPELINE_SOURCE != "merge_request_event"
environment:
name: production
# Require manual approval
when: manual
Network segmentation matters too. Your build runners shouldn’t have direct access to production databases. Use separate runners for different security contexts, and firewall them appropriately.
Artifact Signing and Verification
Building a secure artifact means nothing if an attacker can swap it before deployment. Cryptographic signing creates a chain of custody from build to production.
Cosign, part of the Sigstore project, makes container signing straightforward:
# .github/workflows/build-and-sign.yml
name: Build and Sign
on:
push:
branches: [main]
permissions:
contents: read
id-token: write # For OIDC signing
packages: write # For pushing to GHCR
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to GHCR
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push
id: build
uses: docker/build-push-action@v5
with:
push: true
tags: ghcr.io/${{ github.repository }}:${{ github.sha }}
- name: Install Cosign
uses: sigstore/cosign-installer@v3
- name: Sign container image
run: |
cosign sign --yes \
ghcr.io/${{ github.repository }}@${{ steps.build.outputs.digest }}
At deployment time, verify the signature before running anything:
# In your deployment workflow
- name: Verify image signature
run: |
cosign verify \
--certificate-identity-regexp="https://github.com/${{ github.repository }}/*" \
--certificate-oidc-issuer="https://token.actions.githubusercontent.com" \
ghcr.io/${{ github.repository }}@${{ env.IMAGE_DIGEST }}
Audit Logging and Anomaly Detection
You can’t detect breaches you don’t log. Every pipeline execution should produce an audit trail: who triggered it, what changed, what secrets were accessed, what artifacts were produced.
Configure webhooks to stream CI events to your logging infrastructure:
# .github/workflows/audit.yml
name: Audit Events
on:
workflow_run:
workflows: ["*"]
types: [completed]
permissions:
actions: read
jobs:
audit:
runs-on: ubuntu-latest
steps:
- name: Send audit event
run: |
curl -X POST "${{ secrets.SIEM_WEBHOOK_URL }}" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${{ secrets.SIEM_TOKEN }}" \
-d '{
"event_type": "ci_workflow_completed",
"repository": "${{ github.repository }}",
"workflow": "${{ github.event.workflow_run.name }}",
"conclusion": "${{ github.event.workflow_run.conclusion }}",
"actor": "${{ github.event.workflow_run.actor.login }}",
"sha": "${{ github.event.workflow_run.head_sha }}",
"run_id": "${{ github.event.workflow_run.id }}",
"timestamp": "${{ github.event.workflow_run.updated_at }}"
}'
Look for anomalies: builds triggered at unusual hours, workflows accessing secrets they don’t normally use, sudden increases in artifact sizes, or builds from unexpected branches.
Putting It Together: A Hardened Pipeline Template
Here’s a complete GitHub Actions workflow combining all hardening techniques:
# .github/workflows/secure-pipeline.yml
name: Secure Pipeline
on:
push:
branches: [main]
pull_request:
branches: [main]
# Explicit minimal permissions at workflow level
permissions:
contents: read
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
jobs:
security-scan:
runs-on: ubuntu-latest
permissions:
contents: read
security-events: write
steps:
- uses: actions/checkout@v4
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@0.16.0
with:
scan-type: 'fs'
scan-ref: '.'
format: 'sarif'
output: 'trivy-results.sarif'
- name: Upload scan results
uses: github/codeql-action/upload-sarif@v2
with:
sarif_file: 'trivy-results.sarif'
build:
needs: security-scan
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
id-token: write
outputs:
digest: ${{ steps.build.outputs.digest }}
steps:
- uses: actions/checkout@v4
- name: Verify dependency integrity
run: |
npm ci --ignore-scripts
git diff --exit-code package-lock.json
- name: Generate SBOM
run: npx @cyclonedx/cyclonedx-npm --output-file sbom.json
- name: Upload SBOM
uses: actions/upload-artifact@v4
with:
name: sbom
path: sbom.json
- uses: docker/setup-buildx-action@v3
- name: Login to GHCR
if: github.event_name == 'push'
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push
id: build
uses: docker/build-push-action@v5
with:
context: .
push: ${{ github.event_name == 'push' }}
tags: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}
sbom: true
provenance: true
- name: Install Cosign
if: github.event_name == 'push'
uses: sigstore/cosign-installer@v3
- name: Sign image
if: github.event_name == 'push'
run: |
cosign sign --yes \
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}@${{ steps.build.outputs.digest }}
deploy:
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
needs: build
runs-on: ubuntu-latest
permissions:
id-token: write
environment: production
steps:
- name: Install Cosign
uses: sigstore/cosign-installer@v3
- name: Verify image signature
run: |
cosign verify \
--certificate-identity-regexp="https://github.com/${{ github.repository }}/*" \
--certificate-oidc-issuer="https://token.actions.githubusercontent.com" \
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}@${{ needs.build.outputs.digest }}
- name: Configure AWS credentials (OIDC)
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::123456789012:role/GitHubActionsDeployRole
aws-region: us-east-1
- name: Deploy
run: |
# Your deployment commands here
echo "Deploying verified image: ${{ needs.build.outputs.digest }}"
This template enforces security at every stage: vulnerability scanning before build, dependency verification, SBOM generation, minimal permissions per job, OIDC authentication, image signing, and signature verification before deployment.
Pipeline security isn’t a feature you add—it’s a discipline you practice. Start with the highest-impact changes: eliminate long-lived credentials, pin your dependencies, and restrict permissions. Then layer on signing and verification. Your future self, investigating a 2 AM security incident, will thank you.