Scenario: exec format error

The Symptom

A hotfix needs to go out quickly. CI is backed up, so an engineer builds and pushes from their laptop — a reasonable shortcut under pressure:

docker build -t myregistry/api:1.4.1 .
docker push myregistry/api:1.4.1

The image pushes cleanly. Deployment succeeds. Then every container on every production host dies immediately, and the logs contain exactly one line:

exec format error

No stack trace. No application output. Nothing from the runtime at all.

Meanwhile the same tag runs perfectly on the engineer's machine, and on the two other laptops they ask to test it.

Reading the Error

exec format error comes from the kernel, not from your application. It means the kernel was asked to execute a binary it cannot recognise as runnable on this CPU.

Which narrows things down usefully, because it means:

• The image pulled fine — this isn't a registry or network problem.
• The container was created fine — this isn't a Docker configuration problem.
• Your code never ran at all — so nothing in the application is at fault.

The binary is intact. It's just compiled for a different processor architecture.

The Investigation

The laptop was an Apple Silicon Mac (arm64). The production hosts are x86 (amd64). A plain docker build produces an image for the architecture of the machine doing the building — so the pushed image was arm64-only.

Confirm it by asking the registry what's actually under the tag:

docker buildx imagetools inspect myregistry/api:1.4.1

Name:      myregistry/api:1.4.1
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform:  linux/arm64

A single manifest, single platform. Compare that with a properly built multi-platform image:

docker buildx imagetools inspect alpine

Name:      docker.io/library/alpine:latest
MediaType: application/vnd.docker.distribution.manifest.list.v2+json
Manifests:
  Name:     docker.io/library/alpine:latest@sha256:e7d88de73db3...
  Platform: linux/amd64

  Name:     docker.io/library/alpine:latest@sha256:e047bc2af179...
  Platform: linux/arm/v6
  ...

Note the MediaType. The working image is a manifest list — an index pointing at one image per platform. The broken one is a bare single-platform manifest.

That's the entire bug, visible in one command.

The Mechanism

When a host pulls a tag backed by a manifest list, it selects the entry matching its own CPU automatically. One tag, correct binary everywhere, no branching in deploy scripts.

When the tag points at a single-platform manifest, every host gets that one build regardless of what it can run.

This failure became common the moment ARM laptops became standard developer hardware while servers stayed predominantly x86. It's a mismatch between where code is built and where it runs — and CI usually hides it, because CI runners are typically x86 and therefore accidentally produce the right thing.

The Fix

Build for both architectures and push a manifest list:

docker buildx build \
  --platform linux/amd64,linux/arm64 \
  -t myregistry/api:1.4.1 \
  --push .

Two details that trip people up:

--push is required, not optional. A multi-platform build can't be loaded into the local image store as a single image, because that store holds one image per tag. The manifest list only makes sense in a registry.

Emulation is slow. Building a foreign architecture uses QEMU emulation unless you have native builders, and emulated builds can be dramatically slower — enough to reshape a pipeline's runtime. If cross-architecture builds become a bottleneck, the answer is a builder with native nodes per architecture rather than more emulation.

Catching it before production

The real fix is making the mismatch impossible to ship. Assert the platforms in CI before a tag is promoted:

docker buildx imagetools inspect myregistry/api:1.4.1 | grep -q "linux/amd64" \
  || { echo "ERROR: image is missing a linux/amd64 build"; exit 1; }

In an Interview

This one comes up as a debugging question — "a container works locally and fails in production with exec format error, what's going on?" — and it rewards reading the error precisely.

What's being tested

• Can you interpret an unfamiliar error? The reasoning (kernel-level, so execution never started, so it isn't application code) matters more than recognising the string.
• Do you understand images as platform-specific artifacts? Many people think of an image as universally portable. It's portable across runtimes, not across architectures.
• Do you think about prevention? Fixing the build is table stakes; gating it in CI is the better answer.

How to answer

Lead with the interpretation: "That error is from the kernel saying it can't execute the binary format, so the application never started — this is almost always an architecture mismatch, typically an arm64 image built on an Apple Silicon machine running on x86 hosts."

Then the confirmation: docker buildx imagetools inspect on the tag, checking whether it's a manifest list and which platforms it covers.

Then the fix: buildx with --platform and --push to publish a manifest list, plus a CI assertion so it can't recur.

Follow-ups to expect

"Why does CI usually get this right by accident?" CI runners are typically x86 and match production, so a single-platform build happens to be correct. That's precisely why it surfaces during an out-of-band build from a laptop — exactly when you're under time pressure.

"What's the cost of multi-platform builds?" Emulated builds are significantly slower. Native builder nodes per architecture are the usual answer when it matters.

"How does this relate to Kubernetes?" Identically — nodes pull by tag and get whatever the manifest points at. On a mixed-architecture cluster (ARM nodes for cost, x86 for compatibility), a single-platform image will schedule onto a node that can't run it, and you get exec format error in a CrashLoopBackOff instead.

Next Scenario

This failure was consistent and reproducible once you knew where to look. The last scenario is the hardest kind — a test suite that passed locally every time and failed one CI run in four, which the team retried for weeks before anyone treated it as a real bug.