AgentStack/Agents/hermes-harness-spec

Hermes Harness — Spec (Draft)

Status: core architecture reviewed and decided (see §9). One open item remains (§8 — compression rubric). Daily tier implemented 2026-07-02 — AgentStack/Hermes/dispatcher.py (DISPATCHER + mechanical LINT-AGENT) and AgentStack/Hermes/validator.py (VALIDATOR) are live in the nightly/weekly cron chain, writing to Logs/memory/YYYY-MM-DD.md. FIX-AGENT implemented 2026-07-04 — four .claude/skills/ (fix-frontmatter, fix-frontmatter-fields, fix-broken-wikilinks, setup-wikilinks) plus .claude/agents/fix-agent.md and AgentStack/Hermes/fix.md are live in the nightly cron line, between DISPATCHER and the LINT-AGENT step (lint.md) — it actually fixes what DISPATCHER only used to detect. Same deviation shape as LINT-AGENT's: only one VALIDATOR call actually exists at the end of the real nightly cron line, not one per arrow in §2's diagram. Monthly/yearly compaction (MEMORY-COMPACTOR, ARCHIVE-COMPACTOR), SKILL-SCOUT-AGENT's .drafts/ promotion, BRIEFING-AGENT, REVISER-AGENT, and WIKI-CONSOLIDATION-AGENT are still not implemented. One deviation from §2's table: LINT-AGENT runs as a deterministic Python script, not a Haiku 4.5 call — a bash prototype proved the mechanical checks don't need an LLM, and doing so was strictly cheaper and more reliable. Applies the harness principles from agent_harness_hands_on and harness-rtk-headroom-synthesis, and the memory taxonomy from LLM_Memory, to the Hermes nightly scheduler.

1. Why change anything

Current state: Hermes runs one pass, one model (Sonnet 5), one reasoning budget, for every job. Two problems, both stated in your own research:

No deterministic gate. agent_harness_hands_on's thesis: "the harness enforces correctness, the LLM provides reasoning." Hermes currently has no non-LLM enforcement point between steps.
No model/reasoning routing. Same doc, Next Step #4: "you don't run all tasks at rung 5... run a fast/cheap model first, escalate if it fails." A mechanical lint pass and a week of wiki synthesis don't need the same budget.

2. Nightly pipeline — agent roles

DISPATCHER (not an LLM — reads Logs/hermes-task-queue.md)
   │
   ▼
FIX-AGENT ──▶ [VALIDATOR] ──▶ LINT-AGENT ──▶ [VALIDATOR] ──▶ SKILL-SCOUT-AGENT ──▶ [VALIDATOR] ──▶ SYNTHESIS-AGENT ──▶ [VALIDATOR] ──▶ BRIEFING-AGENT ──▶ [VALIDATOR] ──▶ REVISER-AGENT
   │                │                    │                        │                              │                                  │
   └── all agents read/append ───────────┴────────────────────────┴──────────────────────────────┴── Logs/memory/YYYY-MM-DD.md ─────┘

Every arrow into [VALIDATOR] means: agent writes its section → validator checks structure → pipeline only advances on pass. Failure halts the run for morning review — no auto-retry in v1 (decided, §9).

Role	Cadence	Reads	Writes	Model / effort
DISPATCHER	every run	`Logs/hermes-task-queue.md`	`Logs/memory/YYYY-MM-DD.md` (init), `Logs/heartbeat.md`	no LLM — plain script
FIX-AGENT	nightly, always	tonight's `## LINT-AGENT` findings (frontmatter-block, broken-wikilinks) + fresh per-skill scans for what DISPATCHER doesn't check (missing frontmatter fields, plain-text→wikilink, `_index.md` drift)	vault `.md` files in place (frontmatter, wikilink targets, `## Related`, `_index.md`), memory file	Sonnet 5, mid effort — its 4 `.claude/skills/` bundle deterministic Python detection and zero-judgment fixes; the LLM only handles title/tag wording, ambiguous-match disambiguation, and plain-text-mention linking
LINT-AGENT	nightly, always	vault files	`Logs/2026.md` (prepend), memory file	Haiku 4.5, low effort
VALIDATOR	after every agent	memory file	pass/fail note, `Logs/heartbeat.md`	no LLM — schema/structure check script
SKILL-SCOUT-AGENT	nightly, skips quiet nights	flagged files, memory file	`AgentStack/Skills/.drafts/`, memory file	Sonnet 5, medium effort
SYNTHESIS-AGENT	nightly, skips quiet nights	`Projects/*.md`, GitHub API, wiki	`PKM/opportunities/`, memory file	Sonnet 5, high effort
BRIEFING-AGENT	nightly, always	full memory file so far	`daily/YYYY-MM-DD.md`	Sonnet 5, medium effort
REVISER-AGENT	nightly, always, last	full memory file	finalizes memory file, seeds tomorrow's queue	Sonnet 5, high effort

"Skips quiet nights" = only runs when there's something to act on (lint flagged procedural content / a project manifest changed) — decided in §9, not "always run at low effort."

3. Two separate log systems — don't conflate them

Resolved in review (§9): there is no log.md. There are two distinct, purpose-separated systems:

Logs/YYYY.md            — raw event log. Already exists as Logs/2026.md.
                           Append-only, general-purpose: lint findings, procedural
                           events, wiki updates, missing frontmatter — everything
                           that happened, in the order it happened. Not compacted,
                           not pruned. This is the ground truth.

Logs/memory/*.md         — compacted MEMORY hierarchy (below). Derived from the
                           daily pipeline runs, not a replacement for Logs/YYYY.md.
                           Bounded, self-pruning, built for agent handoff and
                           retrospective summarization — not for "what exactly
                           happened on file X."

If you need to know precisely what happened to a specific file on a specific day: Logs/YYYY.md. If you need "what mattered this month" or "what's recurred enough to be a pattern": Logs/memory/.

3.1 Three-tier memory hierarchy

Maps onto the taxonomy in LLM_Memory (short-term / episodic / semantic), applied as a time-based consolidation chain — each tier a lossy rollup of the tier below it.

Logs/memory/YYYY-MM-DD.md   (daily · episodic/session)   keep last 15, rest rolled up + cleared
        │  bi-weekly rollup — MEMORY-COMPACTOR
        ▼
Logs/memory/YYYY-MM.md      (monthly · short-term)        keep last 3 months, rest rolled up + cleared
        │  on-eviction rollup — ARCHIVE-COMPACTOR
        ▼
Logs/memory/YYYY.md         (yearly · long-term)           kept forever · VPS now, S3 later

Ordering constraint (new — falls out of §9's decision to keep MEMORY-COMPACTOR and WIKI-CONSOLIDATION-AGENT separate): WIKI-CONSOLIDATION-AGENT runs weekly and is the only source of truth for what graduates to wiki//AgentStack/Skills/. MEMORY-COMPACTOR runs bi-weekly and clears daily files after rolling them into the monthly file — pure storage hygiene, no graduation authority. Since bi-weekly ≈ every ~14-15 days and the daily retention window is 15 files, MEMORY-COMPACTOR must never evict a daily file that WIKI-CONSOLIDATION-AGENT hasn't had at least one weekly pass over yet. In practice this holds naturally (weekly < bi-weekly), but it's a real dependency, not a coincidence — worth asserting explicitly if this gets built as a hard check rather than just relying on timing.

`Logs/memory/YYYY-MM-DD.md` — daily / episodic / session memory

The per-run agent log (Task Queue, per-agent sections, Reviser Summary from §2), plus:

type: tag per finding — lint | skill-candidate | synthesis | contradiction | procedural-signal — lets MEMORY-COMPACTOR filter mechanically.
## Procedural Signals section — repeated behaviors/workflow friction, kept separate from other findings so WIKI-CONSOLIDATION-AGENT can read this section directly without wading through the rest.

Retention: last 15 files, count-based on files that actually exist (quiet nights don't write a file at all — see §7 gap 3).

`Logs/memory/YYYY-MM.md` — monthly / short-term memory

MEMORY-COMPACTOR, bi-weekly, Sonnet 5 / medium effort. Storage hygiene only — not a knowledge-graduation authority.

---
month: 2026-07
period: 2026-07-01 to 2026-07-15
source_daily_logs: [2026-07-01, 2026-07-02, ...]
tier: local
---

# Memory — July 2026 (1st half)

## Summary
2-5 bullets on what actually happened this half-month.

## Lint Trend
Aggregate counts across the period (frontmatter debt, orphan count) — trend, not snapshot.

## Carried-Forward / Open Items
Anything flagged but unresolved — rolls forward until closed.

## Source Daily Logs
Which YYYY-MM-DD.md files this entry consolidates — kept as a pointer even
after those files are cleared (the real recovery path is git history, §7 gap 1).

Note: no "Procedural Patterns" section here anymore — that's WIKI-CONSOLIDATION-AGENT's job, reading the daily files directly while they're still within their 15-file window, not something MEMORY-COMPACTOR needs to extract.

Retention: last 3 months; older ones fold into the yearly file before deletion.

`Logs/memory/YYYY.md` — yearly / long-term memory

ARCHIVE-COMPACTOR, triggered on eviction (not a fixed schedule) — rolls a monthly file up before deleting it. Opus 4.8 / medium effort.

---
year: 2026
months_archived: [2026-01, 2026-02, 2026-03, 2026-04]
storage: vps
tier: local
---

# Memory — 2026

## January 2026
One condensed paragraph — much shorter than the monthly file it replaces.

## Major Milestones
Cross-month significant events — doubles as a project timeline.

## Archive Log
Which months' raw YYYY-MM.md files have been folded in and deleted, and when.

Retention: forever. VPS now, S3 later once volume makes it worth it.

4. Deterministic validator — what it actually checks

Agent's section was appended (non-empty, expected heading present)
Claimed file writes actually exist / actually changed (git status/git diff)
New-note frontmatter parses as valid YAML
No unresolved placeholder text (TODO, <...>, PLACEHOLDER) in a file marked "done"

Failure: mark status: failed, halt, leave for morning review.

5. Scheduled consolidation jobs (not part of the nightly chain)

Job	Cadence	Reads	Writes	Model / effort	Authority
MEMORY-COMPACTOR	bi-weekly	daily memory files since last checkpoint	`Logs/memory/YYYY-MM.md`	Sonnet 5, medium	storage only — no graduation power
ARCHIVE-COMPACTOR	on-eviction (monthly file ages out of 3-month window)	the evicted `YYYY-MM.md`	`Logs/memory/YYYY.md`	Opus 4.8, medium	storage only
WIKI-CONSOLIDATION-AGENT	weekly	daily memory files (`## Procedural Signals`), `AgentStack/Skills/.drafts/`, `PKM/opportunities/`	`wiki/`, `AgentStack/Skills/`, `Logs/wiki-consolidation.md`	Opus 4.8, high	source of truth for knowledge graduation

Decided (§9): these stay separate jobs, different cadences, different authority. Not a merge.

6. Lifecycle of work-product folders

AgentStack/Skills/.drafts/ and PKM/opportunities/ are pipeline outputs, not logs, but have the same noise risk with no retention rule yet. WIKI-CONSOLIDATION-AGENT should either promote or explicitly discard (one-line reason, logged) every draft it touches each week — nothing sits in .drafts/ indefinitely without a recorded decision.

7. Gaps — design fixes already folded in above

Deletion isn't destructive — say so explicitly. Everything's git-tracked. Compactors should commit the roll-up, then commit the deletion as a separate commit, so git history is the actual recovery path and the roll-up is auditable against its source.
Retention clock vs. rollup clock. Rollup trigger is "everything since last checkpoint," not a fixed file count — decouples it from the 15-file retention number so a skipped night doesn't cause drift.
No writes on quiet nights. A night with nothing to report shouldn't produce a memory file — otherwise "last 15" can mean 15 nights of nothing during a quiet stretch, evicting genuinely useful older entries for no reason.
~~Logs/2026.md / log.md boundary~~ — resolved, §9.

Still unresolved: compression policy

The schemas above say what sections each tier has, but not the rule for what's signal (carries forward) vs. noise (dies at that tier). Proposed rubric, pending your confirmation:

Carry forward: anything that recurred ≥2×, anything still unresolved, anything that changed vault structure (new folder/agent/integration). Drop: routine "ran clean, 0 findings" entries.

This rubric is most of MEMORY-COMPACTOR's and ARCHIVE-COMPACTOR's actual job — worth confirming before either gets built.

8. Remaining open questions

Compression rubric (§7) — confirm, adjust, or replace the carry-forward/drop rule above.
Should DISPATCHER's scan itself stop flagging non-note files nightly? AgentStack/Hermes/*.md, **/system_prompt.md, and viewer/** have been reported as missing-frontmatter for several consecutive nights (see the 2026-07-03 Procedural Signal) and FIX-AGENT deliberately never touches them — it filters them out on its own end (fix_common.FIX_AGENT_EXCLUDE_PATTERNS). Whether DISPATCHER's own scan scope should narrow to match, stopping the nightly noise at the source instead of just downstream, is an open call, deferred here since it wasn't part of the FIX-AGENT ask.

9. Decisions from this review session (2026-07-01)

Question	Decision
`Logs/2026.md` / `log.md` boundary	No `log.md`. `Logs/YYYY.md` = raw event log (lint + procedural + wiki-update events), unbounded, ground truth. `Logs/memory/` = separate, bounded, compacted hierarchy.
MEMORY-COMPACTOR vs. WIKI-CONSOLIDATION-AGENT	Kept separate. WIKI-CONSOLIDATION-AGENT is weekly and the sole authority on knowledge graduation. MEMORY-COMPACTOR is bi-weekly and purely storage hygiene — it has no say in what graduates.
SKILL-SCOUT-AGENT / SYNTHESIS-AGENT on quiet nights	Skip — only run when there's something to act on.
Validator failure handling	Halt only, no auto-retry in v1.
Task-queue config location	`Logs/hermes-task-queue.md`.
Model/effort pinning	Pinned now (see tables in §2 and §5) — Haiku 4.5 for mechanical lint, Sonnet 5 for mid-tier reasoning agents, Opus 4.8 for the two agents with the highest judgment load (WIKI-CONSOLIDATION-AGENT, ARCHIVE-COMPACTOR).

Decisions from review session (2026-07-04)

Question	Decision
Where should Mihir's four auto-fix asks (frontmatter, frontmatter fields, broken links, wikilink/index setup) live?	Real Claude Code primitives: four `.claude/skills/` plus one `.claude/agents/fix-agent.md` combining them, wired into the existing nightly cron — not a separate scheduler, not skills without an orchestrating agent.
Where does FIX-AGENT sit in the nightly chain?	Between DISPATCHER and the LINT-AGENT step (`lint.md`) — fixes land before the nightly judgment pass reads the vault, so opportunity/procedural-signal detection sees post-fix state.
Autonomy level	Full autonomy, same as LINT-AGENT — applies fixes directly, relies on the existing validator + git-tracked-commit safety net, no new approval gate.
Broken-wikilink scope	Only exact-after-normalization typo/case/rename matches auto-fix; genuine gap-marker links (no matching note anywhere) stay untouched, per README's existing convention. No fuzzy/Levenshtein tier — under-fixing is cheap (re-flagged tomorrow), over-fixing silently repoints a link to the wrong note.
Scope of "insert wikilinks" (README step 3, "should have its own note")	FIX-AGENT only links plain-text mentions to notes that already exist; it does not invent gap-marker links for not-yet-written concepts — that stays a human/interactive judgment call.
`_index.md` creation for folders with none	Out of scope — FIX-AGENT updates existing `_index.md` files only; a missing index is flagged (`needs-index`) for Mihir, not auto-generated.
Bare-link-stub files (found live in `TechResearch/Harness_Engineering/`: several notes are a single bare URL, no prose)	`source: web`, filename-derived title only (no fabricated substance), and a `> [!todo]` bookmark callout — and `## Related` is skipped entirely for these, since inventing "conceptually adjacent" connections for an unprocessed bookmark would be fabrication, not documentation.
Detection vs. judgment split	Each skill bundles a `scripts/` helper (in `AgentStack/Hermes/fix_common.py`, reusing `dispatcher.py`'s own file-iteration/exclusion logic) that does the fix directly wherever it requires zero judgment (git-log dates, exact-normalized wikilink rewrites); the LLM handles only title/tag wording, ambiguous-match disambiguation, and plain-text-to-wikilink calls.
`.claude/` excluded from DISPATCHER's scan	Added to `EXCLUDE_DIRS` alongside `.obsidian` — without it, DISPATCHER would nightly-flag the new `.claude/skills//SKILL.md` and `.claude/agents/.md` files themselves as missing-frontmatter, pure self-inflicted noise from adding this feature.
`.gitignore` had `.claude/` blanket-ignored	Narrowed to just `.claude/settings.local.json` (genuinely machine-local). The whole directory being ignored predates this feature and would've meant `.claude/skills/`/`.claude/agents/` — durable automation infrastructure, same category as `dispatcher.py` — silently vanish on any fresh clone or VPS rebuild, breaking the nightly `fix.md` step with no warning.
`validator.py` changes	Added `"## FIX-AGENT"` to `RECOGNIZED_SECTIONS`; `infer_job()` unchanged — FIX-AGENT runs every nightly cadence alongside LINT-AGENT, adding no new cadence-disambiguating signal beyond `## WEEKLY-SYNTHESIS-AGENT`'s presence.
Logging (added 2026-07-04, same session)	FIX-AGENT originally only wrote to the memory file, missing `Logs/YYYY.md` entirely — invisible in the actual audit trail. Fixed by giving each of the four skills its own call to `AgentStack/Hermes/log_fix_activity.py` (backed by `fix_common.log_activity()`), so logging happens from whichever skill did the work, whether run standalone or orchestrated — not just when fix-agent remembers to write a summary afterward. Mirrors dispatcher.py's existing dual-write (`Logs/YYYY.md` raw + memory file section) exactly.