04-state-storage-resources

Volumes, PersistentVolumes, PVCs and StorageClasses

The three-object dance behind persistent storage, and the access mode that means something different from what almost everyone assumes.

September 13, 2026
kubernetespersistent-volumespvcstorageclassaccess-modescsireclaim-policy

Three Objects, Three Owners

Kubernetes splits storage across three objects, and the split is about who decides what:

A PersistentVolumeClaim (PVC) is a request written by an application team: I need 20Gi of this class of storage. It says nothing about which disk, which cloud, or which zone.

A StorageClass is a template defined by the cluster operator describing a kind of storage — an SSD tier, a network filesystem, a cheap archival tier — and which provisioner creates it.

A PersistentVolume (PV) is the actual storage. In a modern cluster you rarely create one by hand: the PVC triggers dynamic provisioning through the StorageClass, and the PV appears automatically.

💡

This separation is why an application manifest is portable across clusters. The pod asks for storageClassName: fast-ssd; whether that's an EBS volume, a GCE persistent disk, or something on-prem is the operator's business. The application never encodes cloud specifics.

Ephemeral Volumes First

Not every volume needs to persist. emptyDir lives and dies with the pod:

yaml
spec:
  containers:
    - name: api
      volumeMounts:
        - name: cache
          mountPath: /tmp/cache
  volumes:
    - name: cache
      emptyDir: {}

It survives a container restart but not pod deletion or rescheduling. Useful for scratch space and for sharing files between containers in a pod — the log-shipping sidecar pattern from phase 1 uses exactly this.

emptyDir: { medium: Memory } backs it with tmpfs, so nothing touches disk. Note that it then counts against the container's memory limit, which surprises people when a large write triggers an OOM kill.

Using a PVC

yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: postgres-data
spec:
  accessModes: [ReadWriteOnce]
  storageClassName: fast-ssd
  resources:
    requests:
      storage: 20Gi
---
apiVersion: apps/v1
kind: Deployment
# ...
      containers:
        - name: postgres
          volumeMounts:
            - name: data
              mountPath: /var/lib/postgresql/data
      volumes:
        - name: data
          persistentVolumeClaim:
            claimName: postgres-data
⚠️

Note the mount path. As the Docker roadmap's data-loss scenario showed, a volume mounted at a path the application never writes to gives you all the reassurance of persistence with none of the substance. Verify with kubectl exec <pod> -- df -h /var/lib/postgresql/data — if it reports the container's root filesystem, your data is ephemeral.

Access Modes: the One Everyone Gets Wrong

This is the most consequential misunderstanding in Kubernetes storage.

ModeShortWhat it actually means
ReadWriteOnceRWOMounted read-write by pods on a single node
ReadOnlyManyROXMounted read-only by many nodes
ReadWriteManyRWXMounted read-write by many nodes
ReadWriteOncePodRWOPMounted read-write by exactly one pod

Read ReadWriteOnce again. It is once per node, not once per pod.

Two consequences people get backwards:

  • Multiple pods on the same node can share an RWO volume. It's not a single-pod lock — that's what ReadWriteOncePod is for.
  • An RWO volume cannot be attached to pods on different nodes. So a Deployment with replicas: 3 sharing one RWO PVC will schedule one pod successfully while the others hang with a multi-attach error.
🚨

This is the roadmap's "three replicas, one disk" scenario. The manifest is valid, kubectl apply succeeds, and two pods sit in ContainerCreating forever. kubectl describe pod shows Multi-Attach error for volume. The fix is either a StatefulSet with volumeClaimTemplates — which gives each replica its own volume — or a storage class that genuinely supports RWX.

And RWX isn't free: most cloud block storage (EBS, GCE PD, Azure Disk) supports only RWO. RWX generally requires a network filesystem — EFS, Filestore, Azure Files, NFS, CephFS. Asking for RWX on a block-storage class leaves your PVC Pending indefinitely.

Check yourself

A Deployment with replicas: 3 mounts a single PVC with accessModes: [ReadWriteOnce] on a cluster with 3 nodes. What happens?

StorageClasses and Binding Mode

yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: fast-ssd
provisioner: ebs.csi.aws.com
parameters:
  type: gp3
reclaimPolicy: Delete
allowVolumeExpansion: true
volumeBindingMode: WaitForFirstConsumer

Two fields deserve attention.

reclaimPolicy decides what happens to the underlying storage when the PVC is deleted:

  • Delete (typical default) — the real disk is destroyed with the PVC.
  • Retain — the PV and its data survive; an administrator reclaims it manually.
🚨

reclaimPolicy: Delete means deleting a PVC destroys real data, immediately and irreversibly. For anything you'd be sorry to lose, use Retain — it converts an accidental deletion from data loss into a manual cleanup task.

volumeBindingMode decides when the volume is created:

  • Immediate — the PV is provisioned as soon as the PVC exists, before any pod is scheduled. In a multi-zone cluster the volume can land in zone A while the scheduler later wants to place the pod in zone B — and a zonal disk can't cross zones. The pod is then permanently unschedulable.
  • WaitForFirstConsumer — provisioning waits until a pod actually needs the volume, so the storage is created in the zone where the pod will run.

WaitForFirstConsumer is almost always the right choice on any cluster spanning zones.

Expanding a Volume

If the StorageClass sets allowVolumeExpansion: true, edit the PVC:

bash
kubectl patch pvc postgres-data -p '{"spec":{"resources":{"requests":{"storage":"50Gi"}}}}'

Growing works; shrinking is not supported. Some drivers need a pod restart to complete the filesystem resize. Check status with kubectl get pvc postgres-data -o yaml and look at the conditions.

CSI: Why Any of This Works

Storage is implemented through the Container Storage Interface (CSI) — a standard contract that storage vendors implement as drivers running in the cluster. It's the same pattern as the CRI for runtimes and Gateway API for traffic: Kubernetes defines an interface, vendors implement it, and your manifests don't change when the implementation does.

CSI is also what enables features beyond basic provisioning — volume snapshots, cloning, and topology-aware placement.

Diagnosing a Pending PVC

bash
kubectl get pvc
kubectl describe pvc postgres-data      # the events explain WHY it's Pending
kubectl get storageclass                # does the requested class exist?
kubectl get pv                           # any available volumes?

Common causes, all visible in describe:

  • No StorageClass with that name — a typo, or a class that exists on another cluster.
  • No default StorageClass and none specified, so nothing provisions it.
  • An access mode the backend doesn't support — RWX on block storage.
  • WaitForFirstConsumer — Pending is correct here until a pod is scheduled. Not a bug.

Check yourself

Why is volumeBindingMode: WaitForFirstConsumer recommended on a multi-zone cluster?

What's Next

You can attach storage. The next guide covers the workload type built for stateful applications — StatefulSets, which give each replica a stable identity and its own volume, and what they still don't do for you.