Data quality checks that catch problems early
Updated
Five check families, quarantine discipline, and the weekly report that keeps dirty data from becoming wrong decisions.
Bad data rarely announces itself. The checks below announce it for you, with the reason attached.
The short answer
Five check families at entry, quarantine with reasons, a weekly trend report, and dated rule changes. Dirty data becomes a managed queue, not a surprise.
The five families
| Family | Example rule | Trend to watch |
|---|---|---|
| Uniqueness | One row per order id plus version | Duplicate rate by source |
| Completeness | Identifier and timestamp present | Missing-field rate by feed |
| Range | Dates plausible, codes known | Out-of-range share weekly |
| Referential | Every line has a parent order | Orphan rate after joins |
| Freshness | Feed arrived within the window | Late arrivals per source |
Worked example: the report that changed the conversation
A fictional operations team (fictional) argues about data quality for months. The FDE ships the five checks with a one-page weekly report: quarantine share by source, top three reasons, oldest unowned failure. Week three shows one feed causing 80 percent of drops; the source team fixes its export. Arguments end where measurement starts. The ETL demo post shows the demo version; the readout post shows how the report earns its slot.
Checklist: quality discipline
- Checks run before load, with quarantine as the only alternative path.
- Every rule has an owner and a threshold in writing.
- Weekly trend report, one page, reviewed with the customer.
- Rule changes carry dates and reasons.
Related reading
Straight answers
Frequently asked questions
Which checks matter most?
Uniqueness, completeness, range, referential and freshness. Five families cover most customer data failures before they reach decisions.
Where should checks run?
At entry, before load: validate the batch, quarantine failures with reasons, and load only what passes. Post-load checks are audits, not gates.
How do you avoid alert fatigue?
Thresholds per check with owners, weekly review of quarantine trends, and rules that change only with a dated note. Noisy checks get tuned or cut.