Hermes Wiki
Synthesis/desired-state-vs-actual-state-is-one-pattern-wearing-three-different-domain-costumes

Synthesis: 'Desired State vs. Actual State, Reconciled by an Automated Loop' Is the Same Pattern in Cloud Compliance, Data Pipelines, and Network Ops — the Vault Has All Three but Never Named the Shared Shape

The connection

This window added ~200 new AWS Tools stub notes plus several dense Architecture/CaseStudies entries, and reading them together surfaces a pattern that recurs three times across completely different domains without any of the notes cross-referencing each other:

  • Cloud resource compliance: AWS Config continuously records actual AWS resource configuration and evaluates it against declared rules, flagging (or auto-remediating) drift from the desired baseline. Its named open-source equivalent, Cloud Custodian, does the same thing as YAML policy-as-code with remediation actions built in — not just detection. Prowler and Steampipe round out the same cluster from two other angles (pre-built CIS-benchmark findings vs. SQL-queryable live resource state for custom checks).
  • Data pipeline schema governance: Pinterest's automated schema evolution (this window) draws the identical loop around CDC ingestion — actual upstream schema is continuously compared against the declared/expected shape, and the system auto-applies changes only within a narrow, explicitly safe zone (additive, backward-compatible), rejecting anything outside that zone (type-narrowing, primary-key changes) to a manual-review path rather than guessing.
  • Network device state: this window's earlier synthesis note already named the same shape in network operations — NetBox holds the declared source of truth, Slurp'it discovers what's actually deployed, and NetPicker continuously verifies deployed-vs-documented drift.

All three are the same four-part loop: (1) a declared/desired state, (2) continuous observation of actual state, (3) automated reconciliation for the safe/narrow case, (4) explicit escalation — never silent guessing — for anything outside that safe case. Pinterest's RPP case study (Resource Provisioner Pipeline, also this window) is the same idea one layer up the stack: instead of reconciling resource state, it centralizes the process that's allowed to change resource state at all (workspace-path-role IAM mapping, OIDC role-chaining, split plan/apply gating) — the desired-state authority lives in one pipeline instead of being re-implemented per team.

Why this wasn't visible before

Each instance of this pattern lives in a folder with its own vocabulary — "compliance" (AWS Config/Cloud Custodian), "CDC/data engineering" (Pinterest), "network source of truth" (NetBox cluster) — and none of the source notes use shared terminology (no note says "desired state" and "actual state" in the same words another does). The Tools/ batch that surfaced AWS Config and Cloud Custodian this window was read in isolation from the Architecture/CaseStudies batch that surfaced Pinterest's schema-evolution note, and both were read separately from the NetBox/Slurpit/NetPicker cluster synthesized in a prior pass. It took reading all three folders in the same incremental window to notice the loop is identical.

What this suggests

  • The "automate the safe common case, escalate rather than guess on the dangerous edge" half of this pattern is exactly Pinterest's own design principle stated explicitly in its case study — worth treating as the generalizable rule, not a Pinterest-specific choice: any reconciliation loop needs an explicit, narrow definition of what it's allowed to auto-fix, with everything outside that boundary routed to a human, never silently applied. AWS Config's auto-remediation and Cloud Custodian's remediation actions should be audited against this same question — do they have a hard boundary, or do they attempt best-effort fixes on ambiguous drift?
  • This is also a reusable lens for RBC_Work's own initiatives: item #28 ("Config drift, authorization agent... autonomously detect, analyze, and correlate configuration, drift, events against approved CR") is, structurally, a fourth instance of exactly this loop — declared state (approved Change Request) vs. actual state (live config), reconciled by an agent. Pinterest's "reject rather than guess" boundary and AWS Config/Cloud Custodian's detection-vs-remediation split are both directly transferable design questions for that initiative: does the config-drift agent only flag drift (detection), or does it auto-remediate within some declared-safe zone, and if the latter, what's the explicit boundary of what it's allowed to touch unattended?
  • Three domains independently converging on the same four-part loop (with no cross-pollination visible in the source material) is reasonably strong evidence this is a fundamental operational-excellence pattern, not a domain-specific trick — worth a standalone Architecture/Fundamentals/ note if it recurs again, since currently it's only implicit inside multiple CaseStudies and Tools notes rather than named once as reusable doctrine.
Hermes Wiki