Services, Endpoints and DNS Inside the Cluster

Why this matters

An application reaches http://api:8080 and gets connection refused, while the API pods run green. The team restarts everything twice before someone reads the Service: its selector names app=api-v2, the pods carry app=api, and the EndpointSlice is empty. Nothing ever selected the pods, so the ClusterIP had nowhere to send traffic. Twenty minutes of restarts for a one-line label gap. Service failures are wiring failures, and the wiring has exactly three links to check in a fixed order.

Concepts

A Service declares a selector (which pods), ports (port exposed inside the cluster, targetPort on the container) and a type (ClusterIP by default). The endpoints controller watches matching pods and writes EndpointSlices; kube-proxy (or the CNI's datapath) programs the forwarding. Selector matches nothing and the slice stays empty: callers get refused or timeout depending on the datapath. targetPort names the wrong container port and traffic arrives where nothing listens. Port versus targetPort confusion causes half of all first-week Service incidents: port is what callers dial, targetPort is where the application actually listens.

Cluster DNS turns service names into addresses: my-svc.my-ns expands to a ClusterIP, and pod hostnames resolve under a predictable pattern. DNS troubleshooting starts inside a pod: resolve the name, then dial the IP directly. Name fails but IP works and the fault sits in DNS or search domains; both fail and the fault sits in selectors, ports or probes removing every endpoint. Headless Services skip the virtual IP and return pod addresses, which stateful callers and debugging both rely on.

EndpointSlices are the ground truth, not the pod list. A pod can run while excluded from endpoints by a failing readiness probe (lesson dp09l3) or by a label drift. Read the slice first: empty slice means selection or readiness, populated slice with failures means the path past the Service.

Worked example

A demo namespace runs an API with two pods and a Service whose selector carries a one-character typo. The learner resolves the name (works, DNS only needs the Service object), dials the ClusterIP (refused, no endpoints), reads the empty EndpointSlice, fixes the selector character, and watches endpoints populate. Then a targetPort mismatch is staged the same way: populated slice, refused connections, one corrected port.

Common wrong move

Debugging the application when the Service wiring is broken. Logs show nothing because traffic never arrived; restarts change nothing because the selector still matches nothing. Read the EndpointSlice before the logs. The L28 lab drills this order until it is reflex.