11-tooling-cloud-ai-awareness

Docker and CI/CD for Backend Services

Build efficient multi-stage Dockerfiles, run local dev stacks with docker-compose, and understand CI/CD pipeline fundamentals.

August 14, 2026
backend-engineerdockerci-cdcontainersdockerfiledocker-compose

Docker and CI/CD

Containers and pipelines are the delivery mechanism for everything a backend engineer builds. You can write a flawless Spring Boot service, but if the image is 900 MB, rebuilds take four minutes on every code change, and there's no automated gate before deploy, you've shipped a maintenance burden along with the feature. This guide covers building lean Dockerfiles for real Spring Boot services, running a full local dev stack, and the CI/CD concepts that turn a Dockerfile into a repeatable, trustworthy release process.


1. Why Containers, Briefly

A container is a process running on the host's kernel, isolated by two Linux kernel features:

MechanismWhat it isolates
NamespacesWhat the process can see — its own PID tree, network interfaces, mounts, hostname
cgroupsWhat the process can use — CPU, memory, I/O limits

An image is a read-only, layered filesystem snapshot plus metadata (entrypoint, exposed ports, env vars). A container is a running instance of an image with a writable layer on top. This distinction matters operationally: you rebuild images, but you throw away and recreate containers — never patch a running container and expect it to survive a redeploy.

💡

Docker doesn't run a full VM — it shares the host kernel. That's why containers start in milliseconds instead of the minutes a VM takes, and why a Linux container image can't run natively on a non-Linux kernel without a lightweight VM layer (which is exactly what Docker Desktop provides on macOS/Windows).


2. Anatomy of a Multi-Stage Dockerfile

The single highest-leverage Docker skill for a backend engineer is a correct multi-stage build: compile in a heavyweight builder stage, then copy only the compiled artifact into a minimal runtime stage.

dockerfile
# syntax=docker/dockerfile:1
 
# ---- Stage 1: build ----
FROM eclipse-temurin:21-jdk-jammy AS builder
WORKDIR /build
 
# Copy dependency descriptors first — maximizes layer cache reuse
COPY pom.xml .
COPY .mvn/ .mvn/
COPY mvnw .
RUN ./mvnw dependency:go-offline -B
 
# Now copy source and build — this layer invalidates on code change,
# but the dependency layer above stays cached
COPY src/ src/
RUN ./mvnw clean package -DskipTests -B
 
# ---- Stage 2: runtime ----
FROM eclipse-temurin:21-jre-jammy AS runtime
WORKDIR /app
 
# Run as a non-root user
RUN groupadd -r appgroup && useradd -r -g appgroup appuser
 
COPY --from=builder /build/target/order-service-*.jar app.jar
RUN chown appuser:appgroup app.jar
USER appuser
 
EXPOSE 8080
 
HEALTHCHECK --interval=30s --timeout=3s --start-period=40s --retries=3 \
  CMD curl -f http://localhost:8080/actuator/health || exit 1
 
ENTRYPOINT ["java", "-XX:+UseContainerSupport", "-XX:MaxRAMPercentage=75.0", "-jar", "app.jar"]

Why each choice matters

Line/techniqueWhy
FROM ... AS builder / AS runtimeOnly the runtime stage ships — the JDK, Maven cache, and source tree never reach production
eclipse-temurin:21-jre-jammy for runtimeJRE-only base is far smaller than a JDK image; no compiler needed to run a jar
COPY pom.xml before COPY src/Docker caches layers by content hash — dependency resolution only reruns when pom.xml changes, not on every code edit
RUN groupadd ... && USER appuserContainers should never run as root; a container breakout as root is a much bigger blast radius
HEALTHCHECKLets docker ps and orchestrators (Compose, ECS, Kubernetes) know the app is actually serving traffic, not just that the process exists
-XX:+UseContainerSupport -XX:MaxRAMPercentage=75.0The JVM must respect the container's cgroup memory limit, not the host's total RAM, or it will OOM-kill itself under the container's cap
⚠️

Without MaxRAMPercentage (or an explicit -Xmx), older JVMs (pre-Java 10) size the heap based on host memory, not the container's cgroup limit — a JVM in a 512 MB container on a 64 GB host could try to allocate a multi-GB heap and get OOM-killed by the kernel. Modern JDKs (17+) detect cgroup limits by default, but explicitly setting MaxRAMPercentage remains best practice for predictable sizing.

Layer caching order — general rule

Order layers from least-frequently-changed to most-frequently-changed. Every instruction that comes after a changed layer is invalidated and re-run — put source code copies last.

.dockerignore

text
target/
.git/
.idea/
*.log
.env
Dockerfile
docker-compose*.yml

A missing .dockerignore is a top cause of slow, bloated builds — without it, COPY . . sends your entire .git history and local build artifacts into the Docker build context on every single build.


3. docker-compose for Local Development

A full local stack — app, database, cache, message broker — should start with one command. docker-compose.yml is how you make that reproducible across the team.

yaml
# docker-compose.yml
version: "3.9"
 
services:
  app:
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - "8080:8080"
    environment:
      SPRING_PROFILES_ACTIVE: docker
      SPRING_DATASOURCE_URL: jdbc:mysql://mysql:3306/orders
      SPRING_DATASOURCE_USERNAME: app
      SPRING_DATASOURCE_PASSWORD: app_password
      SPRING_REDIS_HOST: redis
      SPRING_KAFKA_BOOTSTRAP_SERVERS: kafka:9092
    depends_on:
      mysql:
        condition: service_healthy
      redis:
        condition: service_started
      kafka:
        condition: service_started
    networks:
      - backend-net
 
  mysql:
    image: mysql:8.0
    environment:
      MYSQL_DATABASE: orders
      MYSQL_USER: app
      MYSQL_PASSWORD: app_password
      MYSQL_ROOT_PASSWORD: root_password
    ports:
      - "3306:3306"
    volumes:
      - mysql-data:/var/lib/mysql
    healthcheck:
      test: ["CMD", "mysqladmin", "ping", "-h", "localhost", "-uroot", "-proot_password"]
      interval: 5s
      timeout: 5s
      retries: 10
    networks:
      - backend-net
 
  redis:
    image: redis:7-alpine
    ports:
      - "6379:6379"
    networks:
      - backend-net
 
  kafka:
    image: bitnami/kafka:3.6
    ports:
      - "9092:9092"
    environment:
      KAFKA_CFG_NODE_ID: 0
      KAFKA_CFG_PROCESS_ROLES: controller,broker
      KAFKA_CFG_LISTENERS: PLAINTEXT://:9092,CONTROLLER://:9093
      KAFKA_CFG_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092
      KAFKA_CFG_CONTROLLER_QUORUM_VOTERS: 0@kafka:9093
      KAFKA_CFG_CONTROLLER_LISTENER_NAMES: CONTROLLER
    networks:
      - backend-net
 
volumes:
  mysql-data:
 
networks:
  backend-net:
    driver: bridge
bash
docker compose up -d              # start the full stack in the background
docker compose logs -f app        # follow just the app service's logs
docker compose ps                 # see status + health of every service
docker compose exec mysql mysql -uapp -papp_password orders   # shell into a running service
docker compose down               # stop and remove containers (keeps named volumes)
docker compose down -v            # also remove volumes — full reset
💡

depends_on with condition: service_healthy waits for the dependency's HEALTHCHECK to pass, not just for the container to start. Without this, your app container often starts before MySQL is actually accepting connections, causing spurious startup failures — especially on a cold docker compose up.


4. CI/CD Pipeline Concepts

CI/CD turns "it works on my machine" into "it's verified and deployed the same way every time." The canonical pipeline has four stages.

StagePurposeTypical tools
BuildCompile source, resolve dependencies, produce an artifactMaven, Gradle
TestRun unit + integration tests, fail fast on regressionsJUnit, Testcontainers
ImagePackage the artifact into a versioned, immutable container imageDocker, Buildkit
DeployPush the image to a registry and roll it out to an environmentECS, Kubernetes, Argo CD

CI vs CD, precisely: Continuous Integration is the build+test stages — every merge is automatically verified. Continuous Delivery means every passing build is deployable on demand. Continuous Deployment goes further and deploys automatically without a manual gate. Most teams practice continuous delivery with a manual promotion to production, not full continuous deployment.

A realistic GitHub Actions pipeline

yaml
# .github/workflows/ci.yml
name: CI
 
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
 
jobs:
  build-and-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
 
      - name: Set up JDK 21
        uses: actions/setup-java@v4
        with:
          java-version: "21"
          distribution: "temurin"
          cache: "maven"
 
      - name: Build
        run: ./mvnw -B clean compile
 
      - name: Run tests
        run: ./mvnw -B test
 
      - name: Package
        run: ./mvnw -B package -DskipTests
 
      - name: Upload jar artifact
        uses: actions/upload-artifact@v4
        with:
          name: app-jar
          path: target/*.jar
 
  build-and-push-image:
    needs: build-and-test
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
 
      - name: Log in to registry
        uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}
 
      - name: Build and push image
        uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: |
            ghcr.io/acme/order-service:${{ github.sha }}
            ghcr.io/acme/order-service:latest
          cache-from: type=gha
          cache-to: type=gha,mode=max

5. Image Tagging and Registry Strategy

Tag strategyExampleUse
Git SHAorder-service:a1b2c3dImmutable, traceable to an exact commit — always deploy by SHA, not latest
Semantic versionorder-service:2.4.1Public/versioned releases
latestorder-service:latestConvenience only — never deploy production by latest, it's mutable and non-reproducible
Environment tagorder-service:stagingMoving pointer updated by CD, useful for dashboards, not for rollback safety
🚨

Deploying :latest to production is a reproducibility bug waiting to happen. If two engineers deploy at different times, they may get different actual code despite using the identical tag. Always deploy by immutable digest or Git SHA, and treat latest as a development convenience only.


6. Health, Graceful Shutdown, and Resource Limits

A container that starts is not the same as a container that's ready to serve traffic, and a container that receives SIGTERM needs time to drain in-flight requests.

yaml
# docker-compose resource limits (mirrors what you'd set in Kubernetes requests/limits)
services:
  app:
    deploy:
      resources:
        limits:
          cpus: "1.0"
          memory: 768M
        reservations:
          cpus: "0.5"
          memory: 512M
    stop_grace_period: 30s   # time given to the process between SIGTERM and SIGKILL
java
// Spring Boot: graceful shutdown, drains in-flight requests before stopping
// application.yml
// server:
//   shutdown: graceful
// spring:
//   lifecycle:
//     timeout-per-shutdown-phase: 25s
⚠️

Without graceful shutdown configured, a rolling deploy or autoscaler scale-down sends SIGTERM, and the JVM (or the orchestrator's grace period) may kill in-flight requests mid-response — users see connection resets during every deploy. server.shutdown: graceful in Spring Boot, paired with a stop_grace_period / Kubernetes terminationGracePeriodSeconds longer than your slowest request, fixes this.


7. Common Pitfalls

PitfallConsequenceFix
Running as root in the containerLarger blast radius on container breakoutUSER appuser in the Dockerfile
Baking secrets into image layers (ENV DB_PASSWORD=...)Secrets visible in docker history, image layers, and registries foreverInject secrets at runtime via env vars / secret managers, never ARG/ENV with real values
COPY . . before installing dependenciesCache invalidated on every source change, slow buildsCopy dependency manifests first, install, then copy source
No .dockerignoreHuge build context, slow docker build, accidental secret inclusionAdd .dockerignore mirroring .gitignore plus build artifacts
Single-stage build with the full JDK in production400-900 MB images, slower pulls, larger attack surfaceMulti-stage build, JRE-only runtime base
No HEALTHCHECK / readiness probeOrchestrator routes traffic to a container that's still startingAdd HEALTHCHECK or a Kubernetes readiness probe tied to /actuator/health

Key takeaways

  • Multi-stage builds keep production images small by discarding the compiler, build cache, and source tree — only the runtime and artifact ship.
  • Layer caching rewards ordering instructions from least-to-most frequently changed; dependency manifests before source code is the single biggest win.
  • Always run containers as a non-root user; root inside a container is still a meaningfully larger risk than a scoped service account.
  • Set explicit JVM memory flags (MaxRAMPercentage or -Xmx) so the heap respects the container's cgroup limit, not the host's total RAM.
  • docker-compose with healthcheck + depends_on: condition: service_healthy eliminates the classic "app started before the database was ready" local-dev flake.
  • Never deploy :latest to production — deploy by immutable Git SHA or digest so every environment is traceably reproducible.
  • Graceful shutdown (server.shutdown: graceful + a grace period longer than your slowest request) prevents dropped connections on every rolling deploy.
  • CI should block the merge on build/test failure before an image is ever built — don't let a broken image reach a registry.

Interview Questions

  • What is the difference between a Docker image and a Docker container?
  • Why use a multi-stage Dockerfile instead of a single FROM stage? What specifically gets left behind?
  • How does Docker's layer caching work, and how should you order Dockerfile instructions to exploit it?
  • Why should a container run as a non-root user? What's the actual risk of running as root?
  • How does the JVM's heap sizing interact with a container's memory limit? What can go wrong if you don't configure it?
  • What's the difference between Continuous Integration, Continuous Delivery, and Continuous Deployment?
  • Why is deploying an image tagged :latest to production considered risky?
  • What does depends_on: condition: service_healthy do in docker-compose, and why is plain depends_on often insufficient?
  • Walk through what happens, step by step, when a container receives SIGTERM during a rolling deployment without graceful shutdown configured.
  • What belongs in a .dockerignore file, and why does it matter for both build speed and security?
  • How would you avoid baking a database password into a Docker image's layers?
  • Describe the four stages of a typical CI/CD pipeline for a backend service and what each one verifies.
  • What's the difference between an ECS/Kubernetes readiness probe and a liveness probe, and how does Docker's HEALTHCHECK relate to them?
  • Why might you use cache-from/cache-to with type=gha in a GitHub Actions Docker build?