05-production-hardening

Secrets and Configuration: Two Problems, Two Solutions

Build-time and runtime credentials leak in different ways. Learn where each one ends up, why ENV isn't as private as it looks, and what to do when one escapes.

September 13, 2026
dockersecretsbuildkitenv-varscredential-leakscompose-secretsrotation

Two Different Problems

"How do I handle secrets in Docker?" is really two questions with two different answers, and conflating them is why the topic feels confusing.

  • Build-time secrets — a private registry token, an SSH key to clone a dependency. Needed while the image is built, never after. The risk is that they become part of the image.
  • Runtime secrets — a database password, an API key. Needed by the running container. The risk is where they're stored and who can read them.

Build-Time: Why ARG and COPY Fail

Phase 2 established the layer rule. It's worth restating in security terms:

A file added in one layer and deleted in a later one is still in the image.

So this leaks:

dockerfile
COPY .npmrc /root/.npmrc
RUN npm ci
RUN rm /root/.npmrc          # too late — it's in the COPY layer

And so does this, differently:

dockerfile
ARG NPM_TOKEN
RUN npm ci

Build arguments are recorded in image metadata. Anyone who pulls the image can read them:

bash
docker history --no-trunc myapp:latest | grep -i token
docker inspect myapp:latest | jq '.[0].Config'

This is the scenario from the roadmap: a Dockerfile that built cleanly, passed review, shipped — and published a token that docker history hands to anyone who asks. Nothing errored, because nothing was wrong from Docker's point of view.

The fix: secret mounts

dockerfile
# syntax=docker/dockerfile:1
FROM node:22 AS build
WORKDIR /app
COPY package.json package-lock.json ./
RUN --mount=type=secret,id=npmrc,target=/root/.npmrc \
    npm ci
bash
docker build --secret id=npmrc,src=$HOME/.npmrc -t myapp .

The file exists for the duration of that one RUN step and nowhere else — not in a layer, not in metadata, not in history. Secrets mount at /run/secrets/<id> unless target= says otherwise.

For private Git dependencies there's a dedicated SSH mount:

dockerfile
RUN --mount=type=ssh git clone git@github.com:me/private-repo.git
bash
docker buildx build --ssh default -t myapp .
💡

ARG still has a legitimate job — build-time configuration like a version number, a target environment, or a base image tag. The rule is simply: if publishing the value would hurt you, it is not an ARG.

Check yourself

A Dockerfile copies a credentials file, uses it, then deletes it — all inside a single RUN instruction. A colleague says it's still a leak. Who's right?

Runtime: Environment Variables Are Not Private

Environment variables are the most common way to pass runtime configuration, and they're fine for most of it. For secrets they have real weaknesses worth knowing rather than ignoring:

  • They're visible in docker inspect to anyone who can talk to the daemon.
  • They're inherited by every child process the application spawns.
  • They're routinely captured by crash reporters and error-tracking SDKs, which serialise the environment into bug reports.
  • On Linux they can often be read from /proc/<pid>/environ.
  • They end up in shell history and CI logs when passed on a command line.

None of this makes them unusable — plenty of production systems pass secrets this way. It means you should know what you're accepting, and prefer files when you have the choice.

Files are better

A secret mounted as a file is read by the process that needs it, at the moment it needs it, without being broadcast to every child process and crash report:

yaml
services:
  api:
    image: myapp:1.4.0
    environment:
      DB_PASSWORD_FILE: /run/secrets/db_password
    secrets:
      - db_password
 
secrets:
  db_password:
    file: ./secrets/db_password.txt

The container reads /run/secrets/db_password. Many images support this convention directly — Postgres accepts POSTGRES_PASSWORD_FILE, and a great many others follow the _FILE suffix pattern.

⚠️

In Compose, secrets: with a file: source reads from a path on your machine — so the secret's real security is whatever protects that file. Keep it out of Git, restrict its permissions, and understand that this is a development convenience, not a secret-management system. In production the source should be your platform's secret store.

Where Secrets Should Actually Come From

The progression, roughly in order of maturity:

  1. .env files, gitignored — fine for local development, nothing more.
  2. Orchestrator secrets — Kubernetes Secrets, Swarm secrets, or your platform's equivalent, injected at deploy time.
  3. A dedicated secret manager — Vault, AWS Secrets Manager, GCP Secret Manager. Centralised access control, audit logging, and rotation.

The properties that distinguish a real secret manager are worth naming: access control (who can read this), audit (who did read it), and rotation (change it without redeploying everything). A .env file has none of the three.

🚨

.env belongs in .gitignore and .dockerignore. The first prevents committing it; the second prevents COPY . . baking it into an image, where phase 2's layer rule makes it permanent. Missing the .dockerignore half is a genuinely common way credentials reach a registry. Commit a .env.example with placeholders instead.

Never Bake Secrets Into an Image

Worth stating plainly, because it's tempting and it's final:

dockerfile
ENV DATABASE_URL=postgres://user:realpassword@db:5432/app   # never

An image is a distributable artifact. It gets pushed to registries, pulled by CI, cached on hosts, and shared. A secret inside it is a secret you have published — and every copy already pulled keeps it.

The same image should run in development, staging and production, with configuration supplied at runtime. That's the whole point of the build/run separation, and it's what makes promoting a tested artifact between environments meaningful.

When a Secret Leaks

It will happen. What matters is the response, and the order:

  1. Rotate first. Revoke the credential and issue a new one. This is the only step that actually reduces risk.
  2. Then clean up. Remove the image tag, purge the layer, rewrite the Git history.
  3. Then prevent. Add a secret scanner to CI, fix the Dockerfile, fix the .dockerignore.

The order matters because steps 2 and 3 do nothing about copies already pulled. A registry deletion doesn't reach the CI runner that cached the image last week.

Assume a leaked credential is compromised the moment it's published, not when someone proves it was used. Rotation is cheap. The alternative is finding out from your logs.

Check yourself

An API key was baked into an image with ENV three weeks ago, and the image has been pushed to a private registry and pulled by several CI runners. The team deletes the image tag. Is the problem solved?

What's Next

Your credentials are handled. The last guide of this roadmap addresses the remaining question about anything you ship: what's actually inside it. Vulnerability scanning with Docker Scout, SBOMs, build provenance, and how to triage findings without chasing an impossible zero.