Python: [Bug]: #4991 regresses when any ChatMiddleware is attached — compaction's summary is lost while its exclusion flags persist

[jjwong0915]

Description

A compaction records its result as two mutations of the message list it is given:

• set_excluded(...) mutates the shared Message objects (_compaction.py:1392)
• messages.insert(insertion_index, summary_message) mutates the list (_compaction.py:1395)

apply_compaction then returns project_included_messages(messages) (_compaction.py:1488) — a fresh
list used for that one call — so the durable record of a compaction is entirely the mutation left behind
on the input list. Both halves therefore have to land on a list that outlives the call.

This is exactly what #4991 was about, and what #6299 fixed by compacting in place:

# Compact the caller's list in place when possible. A compaction operation has
# two halves: exclusion flags (mutated on shared Message objects) and inserted
# summary messages. Operating on the original list keeps both halves on the list
# the function-invocation tool loop reuses across iterations; otherwise inserted
# summaries would be lost on a throwaway copy while exclusions persisted, silently
# dropping older groups (issue #4991).
working_messages = messages if isinstance(messages, list) else prepared_messages

— packages/core/agent_framework/_clients.py:383-389

ChatMiddlewareLayer reintroduces exactly that throwaway copy, one layer above. In the layer
composition used by the official clients, it sits between the tool loop and BaseChatClient:

OpenAIChatClient
└── FunctionInvocationLayer     # owns prepared_messages, reused across iterations
    └── ChatMiddlewareLayer     # <-- copies here
        └── ChatTelemetryLayer
            └── BaseChatClient  # runs the compaction

context = ChatContext(..., messages=list(messages), ...)      # _middleware.py:1305  <-- fresh list per model call
...
return super().get_response(messages=context.messages, ...)   # _middleware.py:1347  <-- the copy goes down

Note the short-circuit immediately above line 1305: when no middleware is registered the layer forwards
the original messages untouched. That is why the bug is conditional on a middleware being attached, and
why #6299's regression tests would not have caught it.

What happens, step by step

1. Model call 1 — the loop passes prepared_messages; the middleware layer copies it to L1;
   compaction marks messages excluded and inserts the summary into L1. The model receives a correct
   compacted view.
2. L1 is discarded. The exclusion flags survive, because they live on Message objects shared with
   prepared_messages. The summary does not — it was only ever an element of L1.
3. Model call 2 — the loop passes prepared_messages, which now holds excluded-marked messages and no
   summary. project_included_messages drops the excluded ones. The model receives neither the original
   messages nor the summary that was supposed to replace them.

It cannot self-heal: SummarizationStrategy builds its candidate set from included groups only, filtering
on EXCLUDED_KEY (_compaction.py:1068-1079), so content excluded by an earlier compaction can never be
summarised by a later one. It is gone for the remainder of the run.

Impact

Any agent that (a) has a compaction_strategy and (b) has any chat middleware attached will, on any turn
whose tool loop crosses the compaction trigger, silently drop the summarised history — including the
user's original request — from every subsequent model call in that turn. Nothing is logged.

Expected vs actual

• Expected: identical behaviour with and without a chat middleware attached.
• Actual: with a middleware attached, every model call after a compaction is missing both the
  summarised messages and their summary.

Version history

Bisected with the reproduction below:

agent-framework-core	no chat middleware	with chat middleware
<= 1.7.0	lost	lost
1.8.0 (#6299)	fixed	lost
1.11.0	fixed	lost
1.14.0	fixed	lost
main @ 2213ef8	fixed	lost

Before 1.8.0 summaries were lost universally. Since 1.8.0 the behaviour depends on whether a chat
middleware happens to be attached, which makes it considerably harder to notice.

Possible fixes

The invariant is: the list a compaction mutates must be the list the tool loop reuses across iterations.
Any layer that copies between them preserves the destructive half of a compaction and discards the
restorative half. Options, in no particular order:

• Have ChatMiddlewareLayer write back after the downstream call, e.g. messages[:] = context.messages
  when messages is a list. Simple, but it would also propagate middleware's own message rewrites into
  the tool loop, which is a behaviour change for middleware that rewrites messages per call.
• Apply compaction above the middleware layer so it always sees the loop's list.
• Have ChatContext retain a reference to the caller's list and re-apply only the compaction delta after
  the downstream call returns.

Code Sample

Public imports only, no API key, and the middleware is a literal no-op.

"""Repro: a ChatMiddleware causes in-run compaction to lose its summary while
keeping its exclusion marks, silently deleting history from the model's input.

    pip install agent-framework-core
    python maf_issue_repro.py
"""

import asyncio

from agent_framework import (
    Agent,
    ChatMiddleware,
    ChatMiddlewareLayer,
    ChatResponse,
    ChatResponseUpdate,
    Content,
    FunctionInvocationLayer,
    Message,
    ResponseStream,
    SummarizationStrategy,
)
from agent_framework._clients import BaseChatClient

MODEL_INPUTS: list[list[str]] = []


def tag(m: Message) -> str:
    if text := (m.text or "").strip():
        return text
    return "call" if any(c.type == "function_call" for c in m.contents) else "result"


class NoOpMiddleware(ChatMiddleware):
    """Does nothing at all -- its mere presence is enough."""

    async def process(self, context, call_next):
        await call_next()


class ScriptedModel(FunctionInvocationLayer, ChatMiddlewareLayer, BaseChatClient):
    """Same layer composition as OpenAIChatClient. Calls a tool 4x, then answers."""

    OTEL_PROVIDER_NAME = "scripted"

    def __init__(self, **kw):
        super().__init__(**kw)
        self.turn = 0

    def _inner_get_response(self, *, messages, stream, options, **kwargs):
        self.turn += 1
        turn = self.turn
        MODEL_INPUTS.append([tag(m) for m in messages])

        async def updates():
            if turn <= 4:
                yield ChatResponseUpdate(role="assistant", contents=[
                    Content.from_function_call(call_id=f"c{turn}", name="peek", arguments={"n": turn})])
            else:
                yield ChatResponseUpdate(role="assistant", contents=[Content.from_text("done")])

        return ResponseStream(updates(), finalizer=lambda u: ChatResponse.from_updates(u))


class ScriptedSummarizer(BaseChatClient):
    OTEL_PROVIDER_NAME = "summarizer"

    def _inner_get_response(self, *, messages, stream, options, **kwargs):
        async def run():
            return ChatResponse(messages=[Message(role="assistant", contents=["SUMMARY"])])

        return run()


class CompactOnce:
    """Compacts before the 2nd model call, then leaves the conversation alone."""

    def __init__(self):
        self.inner = SummarizationStrategy(client=ScriptedSummarizer(), target_count=1, threshold=0)
        self.calls = 0

    async def __call__(self, messages: list[Message]) -> bool:
        self.calls += 1
        if self.calls == 2:
            return await self.inner(messages)
        return False


def peek(n: int) -> str:
    """A tool."""
    return f"peeked {n}"


async def scenario(*, with_middleware: bool) -> None:
    MODEL_INPUTS.clear()
    agent = Agent(
        client=ScriptedModel(),
        name="repro",
        tools=[peek],
        middleware=[NoOpMiddleware()] if with_middleware else [],
        compaction_strategy=CompactOnce(),
    )
    stream = agent.run("REMEMBER THIS", stream=True)
    async for _ in stream:
        pass
    await stream.get_final_response()

    print(f"\n  chat middleware attached: {with_middleware}")
    for i, msgs in enumerate(MODEL_INPUTS, start=1):
        print(f"    model call {i}: {msgs}")
    broken = [
        i for i, msgs in enumerate(MODEL_INPUTS, start=1)
        if "REMEMBER THIS" not in msgs and "SUMMARY" not in msgs
    ]
    print(f"    calls missing BOTH the user message and the summary: {broken or 'none'}")


async def main() -> None:
    from importlib.metadata import version

    print(f"agent-framework-core {version('agent-framework-core')}")
    await scenario(with_middleware=False)
    await scenario(with_middleware=True)


asyncio.run(main())

Output

agent-framework-core 1.14.0

  chat middleware attached: False
    model call 1: ['REMEMBER THIS']
    model call 2: ['SUMMARY', 'call', 'result']
    model call 3: ['SUMMARY', 'call', 'result', 'call', 'result']
    model call 4: ['SUMMARY', 'call', 'result', 'call', 'result', 'call', 'result']
    model call 5: ['SUMMARY', 'call', 'result', 'call', 'result', 'call', 'result', 'call', 'result']
    calls missing BOTH the user message and the summary: none

  chat middleware attached: True
    model call 1: ['REMEMBER THIS']
    model call 2: ['SUMMARY', 'call', 'result']
    model call 3: ['call', 'result', 'call', 'result']
    model call 4: ['call', 'result', 'call', 'result', 'call', 'result']
    model call 5: ['call', 'result', 'call', 'result', 'call', 'result', 'call', 'result']
    calls missing BOTH the user message and the summary: [3, 4, 5]

REMEMBER THIS is the user's message. From model call 3 onward it is absent, and no summary stands in
for it.

Error Messages / Stack Traces

None. The failure is entirely silent: no exception, no warning, no log record, no OTel signal. The only
observable trace is a model that has lost the conversation it was working on.

Package Versions

agent-framework-core: 1.14.0 (also reproduced on main @ 2213ef8); bisected across 1.7.0 / 1.8.0 / 1.11.0 / 1.14.0

Python Version

Python 3.14.6

Additional Context

#4991 — the original report. Its symptom description matches this one exactly; only the copy site
differs.

• Python: Fix compaction message-id collisions and tool-loop summary persistence #6299 — the PR that closed it. It fixed the copy in _prepare_messages_for_model_call and did not
  touch the one in ChatMiddlewareLayer. Its regression tests exercise the tool loop without a chat
  middleware in the pipeline, which is why this path stayed uncovered.
• Python: [Feature]: compaction strategies mutate/delete conversation history with zero observability #7224 — open, "compaction strategies mutate/delete conversation history with zero observability."
  Different complaint (silence during correct compaction), but it reports a model visibly
  reconstructing lost intent after a compaction. This bug is a plausible unreported cause of symptoms of
  that shape, so the two are worth cross-referencing.
• Python: [Bug]: ToolResultCompactionStrategy digests EXCLUDED messages, so collapsing a partly-included group GROWS the context #7387 — related prior art on the _excluded/summary split causing incorrect behaviour.

Discovered while investigating why a production agent silently forgot its task mid-run. That agent
attaches a chat middleware that promotes image content out of tool results — the middleware itself is
irrelevant to the failure, as the no-op middleware in the reproduction shows.

Scope of the claims

• The reproduction uses scripted chat clients, not a live model; the message flow is real but the model
  behaviour is synthetic.
• The bisect used the minimal reproduction above. The same failure was separately confirmed on 1.14.0
  with a real threshold-based compaction strategy and a real (non-trivial) chat middleware.
• No measurement of how often this triggers in practice — it requires a tool loop long enough to cross
  the compaction trigger within a single run.

[github-actions]

🤖 Automated triage reproduction notes (agent-authored — trust but verify)

Agent analysis

Repro: ChatMiddlewareLayer.get_response() in packages/core/agent_framework/_middleware.py:1370 creates list(messages) — a throwaway copy — for each downstream model call when any middleware is registered. Compaction (triggered via _clients.py:389) inserts its summary into that copy, which is discarded after the call returns; exclusion flags persist on shared Message objects but the summary is lost. Minimal repro: attach any ChatMiddleware (even a no-op) to an Agent with a compaction_strategy and a tool that triggers multiple model calls in one turn; after compaction fires, subsequent calls are missing both excluded messages and their summary.

• Failing test: python/tests/test_issue_744_repro.py
• Files examined: python/packages/core/agent_framework/_middleware.py, python/packages/core/agent_framework/_clients.py, python/packages/core/pyproject.toml
• Tests run: python/tests/test_issue_744_repro.py::test_compaction_summary_preserved_with_middleware (FAILED), python/tests/test_issue_7744_repro.py::test_compaction_summary_preserved_without_middleware (PASSED)
• Reported version: 1.14.0
• Current version: 1.14.0