Apply Observe, Narrow, Verify Under Time Pressure

Why this matters

Under the clock, engineers either flail (restarting things at random) or freeze (reading every log). The method beats both by bounding each step: one symptom sentence, one split test, one proof. Most CKA failures are not knowledge gaps but method collapses: the candidate knew the commands and applied them to the wrong layer.

Concepts

Observe means describing the failure precisely: which pods, since when, what changed, what the events say. Write it down; memory edits itself under stress. Narrow means binary splits: app versus platform (exec plus logs versus events), DNS versus network (name versus raw IP), one pod versus all pods (single versus fleet). Each test must eliminate a whole branch, never confirm a hunch.

Verify means proving causation: the fix applied, the symptom gone, the fix reverted in staging (or reasoned through) with the symptom returning. Recovery without proof is superstition that fails the next identical incident. The incident record is four lines: symptom, tests run with results, cause, fix plus prevention. Future you reads it at 3 AM.

Worked example

Checkout errors since 14:05, deploys at 14:00, one suspect:

# observe: symptom sentence
# "POST /pay returns 500 from all checkout pods since 14:05, after the 14:00 deploy; events show no kills."
# narrow 1: platform or app?
kubectl get events --sort-by=.lastTimestamp | tail -10   # quiet: app side
# narrow 2: new code or new config?
kubectl rollout history deployment/checkout              # revision 7 at 14:00
kubectl exec deploy/checkout -- printenv | grep -i PAY_  # new var PAY_API_V2=1
# narrow 3: does the old revision still serve?
kubectl rollout undo deployment/checkout --to-revision=6
# errors stop: cause proven, not coincidental
# verify: re-apply revision 7 in staging, errors return -> causation confirmed

The record writes itself from the commands: symptom, three splits with outputs, cause (unmigrated flag in revision 7), fix (migration plus re-roll) and prevention (flag-gated deploys).

Exam notes

The symptom sentence is the highest-value minute: vague symptoms breed vague tests. Never skip the verify step under time pressure; the exam grades the fix, and fixes without proof are reverted guesses. And change one variable per test: parallel changes make every result unreadable.