A rebuilt carburettor can be torqued to spec and still fail the aircraft if one cotter pin is missing. Software setups fail the same way: the visible parts run, the demo works, and an unfastened detail waits further down the airframe. Teams that adopt GitOps inherit exactly this risk, because a reconciliation system that is misconfigured still reconciles — confidently, endlessly, and against the wrong instructions. The checklist below exists to catch those loose pins. Each item states what to inspect, what the evidence says, and what failure looks like when the item is skipped.
Why Inspect a Working GitOps Install
Working and trustworthy are different conditions. A reconciliation agent that syncs eagerly can also delete eagerly, escalate privileges quietly, and overwrite an operator’s emergency fix within minutes. None of that shows up in a demo. It shows up in an incident report. The eight checks below are ordered roughly the way a walkaround is: from the airframe outward, cheapest first, and none of them requires new tooling — only deliberate inspection of what is already installed.
The Eight GitOps Checks
1. Pin every reference before you ship
Scan the repository for anything that floats: an image tag reading latest, a chart version left unpinned, a remote base pointed at a moving branch head. Each one breaks the immutability promise at the heart of GitOps, because two renders of the same commit can produce different systems. The evidence is unambiguous — mutable references are the most commonly cited anti-pattern in the field. The fix costs minutes: resolve every dependency to a fixed digest or version, and let automation propose upgrades as reviewed changes instead of letting the cluster absorb them silently.
2. Environments live in folders, not branches
Lay out one repository with a directory per environment — clusters, shared infrastructure, tenant workloads, applications — and promote changes by moving rendered output between folders. Branch-per-environment looks tidier on a whiteboard and behaves badly in practice: long-lived branches accumulate merge conflicts, and promotion becomes a cherry-pick exercise nobody can audit. The directory layout also makes one discipline natural: a single value change per overlay, rendered in CI, with the resulting diff attached to the review. If your reviewers approve changes they cannot see rendered, this check fails.
3. Give the webhook a secret and a guard
A webhook only accelerates the pull — the agent still fetches on its own schedule — but the endpoint itself is exposed and, by default, unthrottled. Verify that a shared secret is configured and that junk requests cannot hammer the receiver. Then confirm the fallback behaviour: with the webhook switched off entirely, reconciliation should still converge within the normal polling window. Installations that treat the webhook as load-bearing, rather than as a convenience, go quiet the day a firewall rule changes, and nobody notices until someone asks why a merged change never shipped.
4. Shrink the agent’s permissions on purpose
Out of the box, reconciliation agents tend to run with authority across the whole cluster. That default is a gift to any attacker who reaches the repository, and an accident waiting for a typo. Argo CD answers with AppProjects and impersonated service accounts; Flux answers with lockdown flags that bound what its controllers may touch. Inspect which namespaces each agent can read and write, then narrow them to what the workload actually needs. The evidence for this check is indirect but consistent: every credible security write-up on the subject names over-privileged reconcilers among the top risks of the pull model.
5. Decide what prune may never touch
Pruning is the feature that removes objects deleted from the repository — and the source of the worst documented accidents in this field. A merge conflict once dropped a payments manifest from a production layout, and the agent, doing precisely what it was told, removed the live namespace along with its persistent volumes under a Delete reclaim policy. The data went with it. Set namespaces, PVCs, and CRDs to be protected from prune; prefer a Retain reclaim policy on anything holding state; and add a CI step that flags removed state-bearing resources for human eyes. Verify all three are present, not just intended.
6. Write the drift rules before drift happens
Every resource eventually diverges from its declaration, and the response should be a policy fixed in advance rather than a debate during an outage. Four dispositions cover the field: revert it, tolerate it, alert on it, or protect the object from pruning altogether. Autoscaler-managed replica counts and fields filled in by mutating webhooks belong in the tolerate bucket, handled through ignoreDifferences so the agent stops fighting the platform. Anything money touches — a payment namespace, a volume — belongs in the protect bucket. An installation with no written drift policy has simply not met its first serious divergence yet.
7. Keep a documented break-glass path
Self-healing is a virtue until it becomes an adversary. During an incident, an operator may need to scale something down, disable a feature, or pin a version in the live system — and an eager agent will undo each of those moves at the next cycle, fighting the responder in real time. A trustworthy setup documents the escape hatch in advance: suspension in Flux, or the equivalent pause mechanism in Argo CD, invoked through a known procedure with a named owner and a required follow-up change. Test the procedure during a calm quarter. An untested break-glass path is a document, not a capability.
8. Treat Git history as evidence, not the whole case
Chronicling every change in version control creates an audit story that finance and compliance teams adore — and the story has a gap. The repository records intent, not behaviour. What the agent observed, when it reconciled, what it corrected, and which errors it swallowed live in controller and event logs. Argo CD’s own documentation makes this point: reconstruction of an incident needs both sources. Confirm that reconciler events are retained, exportable, and actually read. Then, when an auditor asks how the system was configured on a given date, you can answer with commits plus events instead of a shrug.
Scoring Your Own Walkaround
Run the eight items and record one of three findings against each — nothing vaguer than that, and nothing spoken that is not written down:
- Pass: verified with evidence attached — a config export, a CI log line, a dated screenshot of the setting.
- Fail: scheduled for correction with a named owner and a date, the same way a deferred defect gets a work card.
- Unknown: treat as a fail until someone demonstrates otherwise. Optimism is not an inspection result.
Weight the unknowns heavily. In adoption surveys, teams reporting the strongest outcomes also report exactly these disciplines in place, though the survey evidence is correlational rather than experimental, so treat it as direction, not proof. What is not correlational is the failure record behind checks five and seven: the prune disasters and responder fights are documented case history, described in detail in a detailed GitOps guide on xOps School alongside the runbooks for recovering from each.
The mechanical mindset carries over from hardware, which is why parts people adapt well here. Nobody signs off an annual inspection by saying the aircraft “seemed fine”. The same standard — stated criteria, recorded results, dated sign-off — is what separates a GitOps installation you can trust from one you are merely fond of. Budget an afternoon for the first walkaround, an hour each quarter after that, and keep the completed checklist with the repository so the next reviewer inherits your findings rather than your optimism.
Costs deserve one honest line, because skipped inspections have a price the same way deferred maintenance does. The parallel is drawn neatly in this breakdown of where money quietly goes every month: small unwatched leaks compound. And when a check fails, resist the urge to fix and fly immediately — plan the correction the way a good trip starts, with the route written down first, as in this account of discovering Canada with a planned route and the right vehicle, and let the journey begin from a documented baseline, the way a well-prepared journey starts here.
