Context compaction
Context compaction is the technique an AI coding agent uses to keep a long conversation inside the model's context window by condensing older messages into a compact summary while preserving recent turns verbatim, so work can continue without losing the thread.
Every LLM has a fixed context window. In a long agentic coding session — repeated file reads, tool calls, diffs, and back-and-forth — the transcript grows until it threatens to overflow that window or balloon per-request token cost. Compaction is the agent's answer: instead of truncating blindly (which drops important decisions) or failing outright, it rewrites the older portion of the history into a shorter representation and keeps going. Most agents trigger it automatically once usage crosses a threshold (commonly around 70 percent of the window), and many also expose a manual command.
The naive approach is LLM summarization: send the older half of the conversation to a model and replace it with a prose summary. This works and preserves nuanced reasoning, but it costs an extra LLM round-trip and tokens every time it fires, and a lossy summary can quietly drop a detail the agent later needs. The quality of the surviving context depends entirely on how good that summary is, which is why what to keep verbatim versus compress is a real design decision.
A more structured approach extracts state incrementally as the session runs — which files were touched, what decisions were made, what tool results matter, what failed — rather than reconstructing it from scratch at compaction time. When the threshold is hit, the structured state is already assembled, so serialization is fast and often needs no extra model call at all. Typical tuning knobs include the trigger threshold, how many recent messages to keep untouched, and a rolling window of recent tool results.
Compaction is distinct from prompt caching (which reuses unchanged prefixes to cut cost without removing anything) and from retrieval (which pulls in new context on demand). It is fundamentally a lossy operation on history, so the goal is to lose the least useful tokens first while protecting the recent, high-signal turns the agent is actively reasoning over.