ContextWindowManagement ChunkingStitching
Every model has a hard token limit — when the input (a 10MB+ show ip route diff, times 100 commands, times 1000s of devices) exceeds it, the request simply fails unless you chunk the input and reassemble (stitch) the results.
Why we need this / what value this brings
A request that exceeds the model's token limit doesn't degrade gracefully — it just fails, so at any real scale (10MB+ diffs, 1000s of devices) chunking isn't optional, it's the only way the pipeline runs at all.
When to use this
Any time input size is unbounded or user/system-controlled and could plausibly exceed the model's context window — assume it will happen, don't wait for it to fail in production first.
How to use or implement this
Chunk along logical boundaries (per-command, per-section) rather than fixed byte counts so each chunk is independently meaningful, summarize each chunk, then run a second LLM pass over the chunk summaries to produce the final stitched summary (map-reduce).
Research questions
- Map-reduce summarization is the standard technique here: summarize each chunk independently (map), then summarize the summaries (reduce) — see DataFlowPatterns/MapReduce-BatchAggregation for the general pattern this is an instance of.
- What's actually lost when stitching chunk-level summaries back together — a fact that only makes sense in relation to a different chunk, or a pattern that only appears across chunks (e.g. 'this route flapped on 40 of the 1000 devices') — and how would you design chunking to minimize that loss (e.g. chunk by logical boundary, not fixed byte size)?
- At what scale does per-command, per-device summarization become the wrong shape entirely — e.g. summarizing structured diffs programmatically first, and only sending genuinely ambiguous/anomalous diffs to the LLM?
Empty folder — drop notes, links, and findings here as you research.