Feature Request: Integrate Headroom for Token-Aware Context Compression

[baduyne]

Why I Think Headroom Fits Agent Zero Very Well

I have been experimenting with integrating Headroom into Agent Zero and wanted to share why I think it is a strong architectural fit for the project.

The Core Problem

As Agent Zero evolves into a more autonomous and tool-heavy runtime, context growth becomes a major engineering bottleneck.

A single session may accumulate:

• memory recalls
• browser annotations
• MCP schemas
• tool outputs
• code execution logs
• long reasoning chains
• conversation history

Over time, this creates several problems:

• ContextWindowExceededError
• unstable long-running sessions
• excessive LiteLLM payload sizes
• embedding failures on oversized chunks
• degraded performance on local/self-hosted models
• rapidly increasing token costs

Several existing issues already point to this pattern:

• Memory ContextWindowExceededError when memory updates on long conversations with limited utility model context size. #1413
• ContextWindowExceededError when single code_execution_tool output exceeds context limit (compression doesn't help) #833
• Embedding calls crash with 400 when stored memory exceeds model context window #1436

---

Why Headroom

Headroom is not just a summarizer.

It acts as a context optimization layer between the runtime and the LLM provider.

Instead of blindly truncating data, it:

• compresses tool outputs
• compresses logs
• compresses RAG chunks
• compresses long histories
• preserves retrieval paths for important details

This is especially important for Agent Zero because the runtime heavily depends on:

• plugins
• browser automation
• memory systems
• MCP/tool ecosystems
• long agent loops

These are exactly the kinds of systems where context bloat becomes difficult to control.

---

Why This Matters for Agent Zero

1. Agent Zero is already LiteLLM-based

This makes integration significantly easier.

Headroom already supports:

• LiteLLM
• proxy mode
• middleware-style integration
• OpenAI-compatible routing

This means we can experiment with:

• zero-code proxy integration
• plugin-level compression
• selective memory compression
• tool-aware compression pipelines

without redesigning the runtime.

---

2. Tool Outputs Are Extremely Expensive

One of the biggest token problems in agent systems is not user prompts — it is tool output accumulation.

Examples:

• large JSON payloads
• browser DOM dumps
• shell logs
• database query results
• repetitive execution traces

Headroom specifically targets these workloads.

Its compressors are content-aware:

• JSON compressors preserve anomalies/errors
• AST compression preserves code structure
• log compression preserves transitions/errors

This is much safer than naive truncation.

---

3. Better Support for Local Models

Agent Zero increasingly supports:

• Ollama
• local providers
• self-hosted inference

Smaller local models suffer much more from context overload.

Reducing context size can:

• improve stability
• reduce latency
• reduce VRAM usage
• improve long-session survivability

This is particularly valuable for self-hosted users.

---

4. Long-Term Runtime Scalability

The more autonomous the runtime becomes, the more important context engineering becomes.

At small scale:

• token usage is manageable.

At large scale:

• context growth becomes exponential.

Without compression or retrieval-aware context management, long-running agents eventually become:

• expensive
• unstable
• slower over time

Headroom introduces an interesting approach:

• compress aggressively
• preserve retrieval access
• keep context small unless detail is truly needed

I think this aligns very well with the future direction of Agent Zero.

---

Possible Integration Directions

Option 1 — Proxy Mode

Simplest experimentation path.

Agent Zero
    ↓
LiteLLM
    ↓
Headroom Proxy
    ↓
Provider

---

Option 2 — Plugin-Level Compression

Compress:

• memory recalls
• browser outputs
• tool results

before they are appended into runtime context.

---

Option 3 — Token-Aware Runtime Policies

Introduce:

• compression thresholds
• context budgets
• selective memory injection
• adaptive tool-output compaction

---

Final Thoughts

I think Agent Zero is approaching the stage where:

• runtime orchestration
• memory architecture
• token management
• context engineering

become just as important as model quality itself.

Headroom looks like a strong candidate for helping solve this problem in a way that is:

• practical
• local-first
• minimally invasive
• compatible with the current LiteLLM architecture

Curious what maintainers think about:

• preferred integration direction
• runtime insertion points
• plugin-level vs middleware-level compression
• observability for token savings