Hermes Wiki
Tools/Headroom

Headroom

Local-first context compression layer that sits between your agent and the LLM provider. Compresses everything an agent reads — RAG chunks, JSON tool outputs, source code, chat history, and shell output (via bundled RTK) — before it hits the LLM.

Claims 60–95% fewer tokens, same answers.

[!note] RTK is bundled Headroom ships RTK as a dependency. headroom wrap claude gives you Headroom's full pipeline plus RTK's shell compression. You're not stacking two tools — you're getting the complete two-layer stack.

Mental Model

RTK  =  surgical tool  (shell output layer only)
Headroom  =  the operating table  (all context types)

Agent task
│
├── git log --stat -2     ← RTK intercepts (shell layer)
│     8,388 raw → 108 RTK  (98.7% savings)
│
├── RAG retrieval (pgvector / JSON) ← Headroom SmartCrusher
│     10 user records, 12 fields → shared header + diffs
│     343 tokens removed, 9.2% reduction, zero semantic loss
│
├── Source code blobs   ← Headroom CodeCompressor (AST-aware)
│
├── Prose chunks / chat history  ← Headroom Kompress (ML, optional)
│
└── LLM call

Architecture

agent → CacheAligner → ContentRouter → {SmartCrusher | CodeCompressor | Kompress} → CCR → LLM
                                                                      ↑ originals cached locally

Four deployment modes:

  • Library: compress(messages, model="claude-...") — inline in your code
  • Drop-in proxy: headroom proxy --port 8787 — OpenAI-compatible, zero code changes
  • Agent wrapper: headroom wrap claude — wraps the entire agent process
  • MCP server: plug directly into the MCP layer

The Three Compressors

SmartCrusher — JSON arrays/objects

  • Detects homogeneous arrays, extracts shared header, writes per-record diffs
  • 10–40% reduction on repeated-schema JSON (user records, API responses, search results)
  • Semantic risk: near zero. Fully deterministic, fully reversible
  • Safe for regulated/compliance environments

CodeCompressor — Source code

  • AST-aware for Python, JS, Go, Rust, Java, C++
  • Keeps function signatures, class definitions, imports, docstrings — stubs or removes bodies
  • 15–50% reduction
  • protect_analysis_context automatically skips this compressor when the user message contains "analyze", "review", "debug", "explain"
  • Semantic risk: low–moderate depending on whether body logic is what the model needs

Kompress — Prose text

  • chopratejas/kompress-v2-base — small HuggingFace transformer (~180 MB, Apache 2.0)
  • Extractive sentence scoring — keeps top-k sentences at configured target_ratio
  • 40–85% reduction depending on ratio
  • Runs via ONNX runtime, CPU-only, local after one-time download
target_ratio Text kept Use case
0.7 70% Docs, specs, policies
0.4 40% General prose, summaries
0.2 20% Logs, verbose narratives

[!warning] Kompress CPU overhead Adds ~35% inference time on CPU. At <5,000 tokens, overhead exceeds savings. Enable only when context pressure is the binding constraint (>15,000 tokens), or when context window is near limit.

Key Design Decisions Worth Knowing

CCR — Reversible compression

Originals stored locally with configurable TTL. Model calls headroom_retrieve(content_id) to pull the full original when it decides it needs more detail. This is what separates Headroom from one-shot summarizers — lossy compression with an escape hatch.

CacheAligner — Preserves prompt cache hits

Naive compression changes the prompt prefix → breaks provider KV cache → you pay more, not less. CacheAligner stabilizes compressed prefixes so Anthropic/OpenAI prompt caching still hits after compression. Token savings and cache discount, not a trade-off.

Output token reduction

Newer feature: verbosity steering (appended terseness note) + effort routing (lower thinking effort on routine resume-after-tool-result turns). Reported as an estimate with confidence interval + 10% unshaped holdout control group — unusually rigorous self-reporting.

Real Numbers from Testing (LangGraph + Qwen2.5:7b, local)

Rule-based only (SmartCrusher, kompress_model="disabled"):

Baseline:   3,496 prompt tokens  |  11.57s inference
Compressed: 3,174 prompt tokens  |  11.06s inference
Reduction:  -9.2% tokens         |  -4.4% time
Transform:  router:tool_result:smart_crusher

With Kompress enabled:

3,496 tokens → 2,600 tokens  (25.6% reduction)
+35% inference time overhead on CPU
Break-even: ~5,000–15,000 tokens

Safe vs Dangerous Compression Zones

Chat session history

Safe: Early turns (N-4+), tool outputs from old turns, repeated system message JSON.

Dangerous:

  • Last 2–4 messages (protect_recent=4 default)
  • Messages containing commitments ("do X then Y then Z")
  • Numerical state ("retry count is currently 3")
compress(
    messages,
    compress_user_messages=False,  # never compress live user turns
    protect_recent=4,
    target_ratio=0.5,
    kompress_model="disabled",     # zero semantic risk: rule-based only
)

RAG payloads

compress(
    messages,
    compress_user_messages=True,   # tool results are in user messages
    protect_recent=0,              # all turns are context, not dialogue
    target_ratio=0.5,
    kompress_model="disabled",     # or HF model for prose-heavy retrieval
)

[!warning] Message format matters Headroom only processes tool_result blocks where block["content"] is a string, not a list of text blocks. A "content": [{"type": "text", "text": "..."}] format silently falls to router:noop. Ensure content is a string in tool_result blocks.

Enterprise Deployment

Rule-based only With Kompress
Semantic risk Near zero Low–moderate (tunable)
Token reduction 10–40% 40–85%
CPU overhead None ~35% on CPU
Deterministic ✅ ❌
External download ❌ One-time (then local)
GDPR / air-gap ✅ ✅ (after pre-provision)
Best for JSON/code + regulated Prose-heavy RAG, long sessions

For compliance/audit: use kompress_model="disabled". Deterministic output = auditable output. The CCR local cache of originals is itself a compliance asset — it records what the LLM actually received vs. the original document.

Air-gap pre-provision:

huggingface-cli download chopratejas/kompress-v2-base
export HF_HUB_OFFLINE=1

Quickstart

pip install "headroom-ai[all]"
headroom proxy --port 8787   # point one app's base_url here
headroom perf                # see savings

Lowest-risk first experiment: Run the proxy in front of one Claude API call path (not a full agent wrap). Diff token counts on a real RAG-heavy request with prompt caching on, and watch whether your cache hit-rate holds — that's the primary failure mode to catch.

Relevance to Stack

  • Localz: Direct fit. RAG over pgvector + heavy Claude API calls. Proxy in front of one call path → measure compression % on real enriched-profile payloads. CacheAligner preserves Anthropic prompt caching.
  • Compliance AI: Use rule-based only — deterministic + CCR local cache = auditable. Design question: could CCR retrieval logs serve as audit-trace evidence?
  • Hermes ambient agent on VPS: RTK plugin for shell commands + Headroom proxy for document reads.
Hermes Wiki