TechResearch/Headroom_Token_Compression/research-headroom

Research: Headroom — The Context Compression Layer for AI Agents

Repo: chopratejas/headroom · Apache 2.0 · ~37k stars · v0.26.0 (Jun 2026) · Python 79% / Rust 17% / TS 2% One-liner: compress everything an agent reads — tool outputs, logs, RAG chunks, files, history — before it hits the LLM. Claims 60–95% fewer tokens, same answers.

1. 🧭 TL;DR

Headroom is a local-first compression layer that sits between your agent and the LLM provider. Instead of asking the model to read 17k tokens of raw grep output, it routes that content through type-specific compressors (JSON, code-AST, prose) and sends the model a dense version — while caching the originals locally so the model can call a retrieval tool if it actually needs the full thing. It ships in four shapes: a library (compress(messages)), a drop-in proxy, an agent wrapper (headroom wrap claude), and an MCP server. The pitch that should interest you: it's reversible, runs on your machine (data residency), and treats compression as a pipeline of routed transforms rather than a single "summarize it" call. The pitch you should be skeptical of: the headline numbers and the accuracy-preservation claims rest on the maintainer's own benchmark harness.

Who should care: anyone running coding agents daily, anyone whose token bill is dominated by tool-output noise, and anyone building RAG where chunk bloat is the cost driver. For you that's Localz's Claude API spend and the Compliance AI document pipeline.

2. 🔍 What It Is — Core Concept

Problem it solves. Agentic workloads waste enormous token budgets on content the model only skims. A code search returns 100 results; an SRE pulls 65k tokens of logs; a RAG retriever stuffs 20 chunks into context to be safe. Most of those tokens are structural ceremony, repetition, or irrelevant fields. You pay input cost on all of it, and you burn context window that real reasoning needs. Provider-native compaction (OpenAI's, Anthropic's) only touches conversation history and only inside one provider's walls — it doesn't compress a tool result before it lands, and it isn't portable across agents.

Core mechanism. A ContentRouter inspects each blob, classifies its type, and dispatches it to the right compressor:

SmartCrusher for JSON (arrays of dicts, nested objects) — strips redundant keys/structure.
CodeCompressor — AST-aware for Python/JS/Go/Rust/Java/C++, so it compresses structurally rather than as text.
Kompress-base — a HuggingFace model they trained on agentic traces, for prose.
Plus an image compressor (ML-routed) and a CacheAligner that stabilizes prompt prefixes so the provider's KV cache still hits after compression.

Mental model. Think of it as a gzip that understands semantics and is selective about what it throws away — except instead of needing the exact bytes back, it keeps the originals in a local cache (CCR) and hands the model a "retrieve if needed" tool. The model reads the cheap version by default and pulls the expensive original only when it must. It's lossy compression with an escape hatch, which is the whole trick.

3. ⚙️ How It Works — Technical Depth

Architecture / data flow.

agent → CacheAligner → ContentRouter → {SmartCrusher | CodeCompressor | Kompress-base} → CCR → LLM
                                                                            ↑ originals cached locally

The README exposes a stable request lifecycle that's the same across the library, SDK, and proxy: Setup → Pre-Start → Post-Start → Input Received → Input Cached → Input Routed → Input Compressed → Input Remembered → Pre-Send → Post-Send → Response Received.

That lifecycle is the interesting part architecturally — it's a plugin seam. Three extension types hang off it: pipeline extensions (on_pipeline_event(...)), compression hooks, and proxy extensions (ASGI middleware/routes). Provider-specific behavior is isolated under headroom/providers/ (claude, codex, copilot, openclaw, gemini) with a registry.py dispatch, keeping core orchestration provider-agnostic. This is a clean modular-monolith shape — worth noting given your own architectural bias.

Key design decisions worth flagging:

CCR (reversible compression) — originals stored locally with a configurable TTL; model retrieves via headroom_retrieve. This is what separates it from one-shot summarizers and from the hosted API competitors.
CacheAligner — they understood that naive compression breaks prompt caching (you change the prefix, you lose the cache hit, you pay more not less). Stabilizing prefixes to preserve KV-cache hits is a non-obvious, correct insight.
Output token reduction — newer feature: trims what the model writes back via verbosity steering (appended terseness note) and effort routing (dials thinking effort down on routine resume-after-tool-result turns). Honest touch: output savings are reported as an estimate with a confidence interval, with an optional 10% unshaped holdout control group for a measured number. That's unusually rigorous self-reporting for a token-savings tool.
Rust core — 17% Rust (ONNX runtime via cdn.pyke.io) means the hot compression path isn't pure-Python slow.

Limitations & tradeoffs.

Lossy by default. The safety net is CCR retrieval, but that adds a round-trip and depends on the model knowing it's missing something. For compliance/audit work, "the model decided it didn't need the original" is a real risk.
Local process required. Explicitly says skip it in sandboxed environments where local processes can't run.
Benchmark provenance. All accuracy numbers (GSM8K ±0.000, TruthfulQA +0.030, SQuAD/BFCL ~97%) are N=100 on the maintainer's own headroom.evals suite. Reproducible, but not third-party.
New runtime assets fetched over TLS (ONNX runtime, the HF model) — a friction point in SSL-inspection corporate networks (they document the workaround, which tells you enterprise users hit it).

Minimal usage:

pip install "headroom-ai[all]"
headroom wrap claude          # wrap a coding agent, zero code changes
# or
headroom proxy --port 8787    # drop-in OpenAI-compatible proxy

from headroom import compress
compressed = compress(messages, model="claude-...")

4. 🆚 Comparison & Landscape

	Scope	Deploy	Local	Reversible
Headroom	All context (tools, RAG, logs, files, history)	proxy / lib / middleware / MCP	✅	✅
RTK	CLI command outputs only	CLI wrapper	✅	❌
lean-ctx	CLI cmds, MCP tools, editor rules	CLI wrapper / MCP	✅	❌
Compresr / Token Co.	text sent to their API	hosted	❌	❌
OpenAI Compaction	conversation history only	provider-native	❌	❌

Positioning. Headroom is the broadest-scope + only-reversible option, and it's a layer not a feature — additive to whatever you run. Notably it doesn't compete with RTK; it bundles RTK for shell-output rewriting and compresses everything downstream. The honest "skip it if" in the README is a credibility signal: skip if you use one provider's native compaction and don't need cross-agent memory.

The real category here is context engineering as infrastructure — this is the same thesis as your "harness engineering > model engineering" principle, applied to the input-token surface. Headroom is a harness component that buys reliability/cost without touching the model.

5. 🔗 Relevance to Your Stack

Localz (Claude API spend). Direct fit. Localz is a modular monolith making heavy Claude API calls with RAG over pgvector. A compression proxy in front of the Claude API could cut input tokens on tool outputs and retrieved chunks with zero changes to your FastAPI code. The CacheAligner matters specifically because you'll want Anthropic prompt caching to keep hitting. Worth a spike: measure compression % on your actual enriched-profile retrieval payloads.

Compliance / GRC AI — use with caution. This is the nuanced call. Compression + a model that self-selects whether to retrieve the original is in tension with audit-grade traceability, which is your stated defensible moat. CCR's local-original cache is actually an asset here (the full evidence is retained and retrievable), but you'd want the compressed-vs-original decision logged as trace evidence, not left implicit. The interesting design question: could the CCR retrieval log itself become part of your "byproduct audit trace"? That's a Principal-lens framing — turn a cost optimization into a governance artifact.

Data residency. Local-first execution and local original-caching align with your Canada-residency hard requirement — nothing leaves the box except the (compressed) prompt that was already going to the provider. The HF model and ONNX runtime are one-time pulls you can pre-provision offline (HF_HUB_OFFLINE=1).

FullStackFusions (unresolved). Strong content angle. "I cut my coding-agent token bill 70% with a local proxy — here's how the routing works" is a concrete, reproducible demo with a satisfying before/after (10,144 → 1,260 tokens in their own GIF). The headroom learn failure-mining feature (writes corrections into CLAUDE.md/AGENTS.md) is a second episode on its own.

Principal Engineer lens. Two signals worth internalizing: (1) the pipeline-lifecycle-as-plugin-seam design is a textbook example of building an extensible harness without a framework — study the lifecycle stage list as a pattern. (2) The honest-estimate-with-CI + holdout control group for output savings is how a serious engineer reports a metric they can't directly measure. Steal that posture for your own Harness Ladder experiment reporting.

6. 🚀 How to Get Started

Quickstart (lowest-risk path):

pip install "headroom-ai[all]"
headroom proxy --port 8787      # point one app's base_url at it
headroom perf                   # see savings

Single most informative experiment. Don't wrap an agent first — run the proxy in front of one Localz Claude call path and diff token counts on a real RAG-heavy request, with prompt caching on, watching whether your cache hit-rate holds (that's the failure mode). One afternoon, clean signal on whether the savings survive your payloads vs. their benchmark payloads.

Resources.

Docs: headroom-docs.vercel.app/docs — Architecture, CCR, "How compression works", Benchmarks, Limitations.
Model card: huggingface.co/chopratejas/kompress-v2-base.
llms.txt in repo root for agent-readable index.
Reproduce benchmarks: python -m headroom.evals suite --tier 1.

7. 📎 References & Links

Repo: https://github.com/chopratejas/headroom
Docs: https://headroom-docs.vercel.app/docs
Kompress-v2-base model: https://huggingface.co/chopratejas/kompress-v2-base
RTK (bundled dependency): https://github.com/rtk-ai/rtk
lean-ctx (alternative CLI context tool): https://github.com/yvgude/lean-ctx

Backlog flags

Spike: Headroom proxy in front of one Localz Claude path — measure compression % + cache hit-rate retention.
Design question: can CCR retrieval logs serve as audit-trace evidence in the Compliance harness? (resolves the lossy-compression-vs-auditability tension)
Compare: Headroom vs. provider-native Anthropic compaction for Localz specifically.