DevOps Practitioner · Module 8: GitOps and incident response
Desired State and Reconciliation
GitOps closes the loop M09 opened: the desired state lives in git, an agent watches both sides, and reality converges to the commit. Every change is a reviewed merge; every cluster state answers to a SHA.
10 min reading
Objectives
- Explain the GitOps promise: git holds desired state, agents converge reality
- Trace a git commit to a cluster change through the reconcile loop
- Separate the declared state from the observed state in any drift dispute
- Name what GitOps does not solve: bad config still ships fast
Why this matters
A cluster accumulates hand edits for a year: hotfixes applied live, experiments never cleaned, manifests in the repo describing a system that no longer exists. Nobody can say what runs or why, and rebuilding from the repo produces a stranger. The drift grew because nothing reconciled: the repo described wishes, the cluster lived its own life. GitOps makes divergence visible within minutes and convergence automatic, so the repo stays the truth by machinery rather than by discipline alone.
Concepts
Two repositories of state, one agent between them. Git holds the declared desired state: manifests, charts, values, reviewed and versioned. The cluster holds the live state. The agent (Argo CD, Flux and kin) compares continuously and acts: apply what git adds, prune what git removes (when pruning is enabled, a deliberate choice), and report drift when live state disagrees. The L46 lab applies a git change through reconciliation on a lightweight setup and watches the loop close.
Reconciliation is pull-based and boring on purpose. No pipeline pushes credentials into the cluster; the agent inside reads git and applies. Every change carries its audit trail for free: author, review, commit, timestamp. Rollback is git revert followed by the same loop, which replays the recorded good state instead of inventing a fix under pressure.
The honest limit is stated early: GitOps ships bad config faster and more reliably than hand edits did. A wrong value merged with approvals still breaks traffic; reconciliation only guarantees the breakage matches the commit. Gates, previews and canaries from M14 still apply; GitOps changes how config travels, not whether it is correct.
Worked example
A demo app runs from a git directory through a local agent. The learner merges a replica change, watches the agent detect it, apply it, and report healthy. Then a hand edit to the live object is made: the agent reports drift within its interval, and with self-heal enabled reverts it to the git state. Both motions are quoted from the agent's view.
Common wrong move
Treating the git repo as documentation while still applying by hand. Two writers (humans and the agent) fight over the same objects, and the agent usually wins at 3 AM, reverting the hotfix nobody committed. One writer: everything through git, no exceptions, or disable the agent.
Quick check
An optional 4-question self-check. Answers never leave your device, are not stored, and never count toward any assessment.
Lesson feedback
No published feedback yet.
Log in and complete the lesson to leave feedback.
Exercise
On a lightweight local setup, merge a declared change to git, watch reconciliation apply it, then hand-edit live state and quote the drift report.
Pass criteria
The record shows the commit, the reconciled change with healthy status, and the quoted drift report on the hand edit.