Data Governance and Data Catalogs
Concept
Data governance is the set of policies, roles, and processes that decide who may create, access, modify, and delete an organization's data, and to what quality standard that data is held. It answers three questions for every dataset: who owns it, who can touch it, and how good does it have to be — and it does so before those questions get asked during an incident or an audit, not after. Three roles recur across every framework: the data owner (accountable for a dataset at the organizational level — usually a business role, not an engineer), the data steward (does the day-to-day work: resolving quality issues, maintaining the glossary, approving access requests, documenting lineage changes), and the data custodian (the technical role that implements access controls and storage — typically a platform/data engineering team). Segregation of these three matters: the person who can grant access (custodian) shouldn't be the same person who decides whether access should be granted (owner).
The data catalog is governance's operational backbone rather than a separate concern — it's the searchable inventory of what data exists, tagged with both technical metadata (author, created/modified dates, source system, schema, size) and business metadata (classification level, retention period, owning team, data-quality score). Without a catalog, governance policy lives in a document nobody consults; with one, "who owns the bookings table and what's its retention policy" is a lookup, not a Slack thread.
Tradeoffs
| Operating model | Benefit | Cost |
|---|---|---|
| Centralized (one governance team owns all policy + catalog upkeep) | Consistent standards, single source of truth, easy to audit | Becomes a bottleneck as the org grows — every new dataset waits on one team's bandwidth |
| Federated / hub-and-spoke (central standards, domain teams do the stewardship work) | Scales past one team's bandwidth; stewards sit closest to the data and know its real quality issues | Requires real investment in tooling and role-specific training so domain stewards apply the same standard, not their own interpretation of it |
| Fully decentralized (data-mesh style — each domain owns governance of its own data products) | Fastest for the owning team, no central bottleneck at all | Cross-domain consistency (naming, classification, retention) degrades without strong platform-level enforcement; discoverability suffers without a shared catalog |
Most organizations start centralized by necessity (one or two people, no catalog tooling yet) and move toward federated as the number of datasets and teams outgrows what a single governance team can review — the failure mode isn't picking the "wrong" model, it's staying centralized past the point where it's actually a bottleneck, or decentralizing before any shared catalog/classification standard exists to hold it together.
When to use / when not to
- Start writing down ownership — even informally, even before any tooling exists — the moment more than one person or team reads from or writes to the same core dataset. The AWS framing is blunt on this: without clear ownership, quality and access-control decisions default to whoever touched the table last, which is not a decision anyone actually made.
- Invest in a real catalog once "who owns X" or "can I use Y for this" becomes a recurring question that used to go to Slack/tribal knowledge — that's the signal the informal stage has stopped scaling.
- Don't build a governance framework or catalog speculatively for data that has one reader, one writer, and no compliance exposure — the overhead isn't justified until sharing (of the data, or of the responsibility for it) actually exists.
- Roll out governance incrementally: a pilot scoped to one high-impact, low-complexity data domain builds credibility and working process before expanding org-wide: attempting full-org governance on day one usually produces a policy document nobody follows.
Common pitfall
Measuring governance success by the number of policies written or datasets catalogued rather than by whether stewardship is actually happening day-to-day. A catalog with 400 entries and zero updates in six months is a liability, not an asset — stale ownership records and stale quality metadata are actively worse than "no catalog" because they're trusted by default. The sustainable version of governance is operating, not documenting: daily resolution of quality issues, glossary upkeep, and access-request approval, done by stewards who are trained on their specific slice (data engineers on lineage/technical metadata, analysts on catalog search, stewards on governance workflows) rather than one central team trying to do all of it.
Engineering Lens
The engineering trap in data governance is treating it as a compliance artifact — a policy doc that exists to be pointed at during an audit — rather than as an operational system with real owners and real SLAs of its own (how stale can catalog metadata get before it's untrusted? who's paged when a classification tag is wrong?). The strongest version of this discipline looks identical to on-call ownership for a service: a named owner, a documented escalation path, and metadata that's live because someone's job depends on it staying accurate, not because a policy said it should be. This matters most exactly where the stakes are highest — PII, payment data, anything with regulatory retention requirements — which is also where an undocumented owner is most expensive to discover during an incident rather than before one.