Services, Endpoints and Cluster DNS

The Problem Services Solve

Every pod gets its own IP address. That IP is disposable — it changes when a pod is rescheduled, replaced during a rollout, or scaled away. Nothing can reasonably connect to a pod by IP.

A Service provides a stable virtual IP and DNS name in front of a set of pods, selected by label:

apiVersion: v1
kind: Service
metadata:
  name: api
spec:
  selector:
    app: api          # which pods
  ports:
    - port: 80        # the Service's port
      targetPort: 8080  # the container's port

Now anything in the cluster can talk to api and reach a healthy pod, whichever ones currently exist.

The Four Types

There's also a fifth variant that isn't a type but matters a great deal:

spec:
  clusterIP: None      # headless

A headless Service has no virtual IP. DNS returns the pod IPs directly rather than load balancing. This is how StatefulSets get per-pod addressable identity, covered in phase 4, and it's also what clients that do their own load balancing (many database drivers, gRPC) often want.

Endpoints: Where Traffic Actually Goes

A Service doesn't route to pods matching its selector. It routes to pods that match and are Ready — the readiness probe from phase 2, doing the job it exists for.

That membership is tracked in EndpointSlices:

kubectl get endpointslices -l kubernetes.io/service-name=api
# NAME        ADDRESSTYPE   PORTS   ENDPOINTS               AGE
# api-abc12   IPv4          8080    10.1.2.3,10.1.2.4       5m

kubectl get endpointslices -l kubernetes.io/service-name=api   # any backends?
kubectl get pods -l app=api                                    # does the selector match?
kubectl get pods -l app=api -o wide                            # are they READY 1/1?

Cluster DNS

CoreDNS gives every Service a name. The full form:

<service>.<namespace>.svc.cluster.local

Short forms work through DNS search domains:

curl http://api                    # same namespace
curl http://api.production         # different namespace
curl http://api.production.svc.cluster.local   # fully qualified

Debugging resolution from inside a pod:

kubectl exec -it api-7d4f8b9c5-x2kfp -- nslookup db
kubectl exec -it api-7d4f8b9c5-x2kfp -- getent hosts db.production
kubectl run tmp --rm -it --image=busybox --restart=Never -- nslookup api.production

How Traffic Actually Reaches a Pod

A Service's ClusterIP is virtual — no interface holds it. kube-proxy watches Services and EndpointSlices and programs each node's packet-handling rules to rewrite traffic destined for that IP toward a real pod IP.

Historically kube-proxy used iptables; modern clusters may run its nftables mode, which scales better with many Services. Some clusters replace kube-proxy entirely with an eBPF dataplane such as Cilium.

The practical consequence: load balancing happens at the connection level, not per request. A client holding one long-lived connection — a gRPC channel, a keep-alive HTTP pool — keeps hitting the same pod. This is why gRPC traffic often distributes unevenly across replicas, and why the answers are client-side load balancing (often via a headless Service) or a service mesh.

Service-to-Service, End to End

apiVersion: v1
kind: Service
metadata:
  name: db
  namespace: production
spec:
  selector:
    app: postgres
  ports:
    - port: 5432
      targetPort: 5432
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
  namespace: production
spec:
  selector:
    matchLabels: { app: api }
  template:
    metadata:
      labels: { app: api }
    spec:
      containers:
        - name: api
          image: myapp:1.4.0
          env:
            - name: DATABASE_URL
              value: postgres://db.production:5432/myapp

db.production resolves to the Service's ClusterIP; kube-proxy rewrites the traffic to a ready Postgres pod. No IPs anywhere in the configuration.

A Diagnostic Order for Connectivity

When service-to-service traffic fails, work in this order:

# 1. Does the Service exist and have endpoints?
kubectl get svc db
kubectl get endpointslices -l kubernetes.io/service-name=db
 
# 2. Are the backing pods ready?
kubectl get pods -l app=postgres
 
# 3. Does the name resolve from the client pod?
kubectl exec -it <client-pod> -- getent hosts db.production
 
# 4. Is the port reachable?
kubectl exec -it <client-pod> -- nc -zv db.production 5432
 
# 5. Is a NetworkPolicy blocking it?
kubectl get networkpolicy -n production

Each step eliminates a layer. Most failures resolve at step 1 or 2.

What's Next

Internal traffic works. The next guide covers getting traffic in from outside — and it's the area where following an older tutorial will now actively cost you, because the controller most of them recommend has been archived.