Python: [Bug]: Function call arguments that are not a JSON object reach the tool as {"raw": ...} instead of failing validation

Observed Behavior

When the model returns function call arguments that are not a JSON object, Content.parse_arguments reports no error. It wraps what it got in a dictionary with one key, raw: text that is not valid JSON becomes {"raw": "<the text>"}, and valid JSON that is not an object, such as a list or a string, becomes {"raw": <the value>}.

The automatic function-calling loop parses with that method and then runs its top-level checks on the result. For a tool declared with a JSON schema that has no required properties and does not set additionalProperties: false, the checks pass, because raw is an extra key and extra keys are allowed. The function is then invoked with raw=... as if the model had made a well-formed call. The function receives an argument it never declared, the model receives whatever the function made of it, and nothing tells the model that its arguments were malformed.

In our case the tools are proxied to MCP servers, so a malformed call reaches the server carrying a bogus raw argument. A missing required argument or a wrong type is answered with a validation error the model can correct on; malformed arguments are not.

Expected Behavior

Arguments that are not a JSON object are invalid arguments. The loop should answer them the way it answers a missing required property: an error result that names the cause, for example "arguments are not a JSON object", so the model can retry. If the raw wrapping stays for direct callers of parse_arguments, the loop should still not treat its result as valid arguments.

Steps to Reproduce

Build a FunctionTool with the schema {"type": "object", "properties": {}}.
Parse a function call whose arguments string is not {json, [1, 2] or "text".
Invoke the tool with the parsed arguments, as the loop does.
The function runs with {"raw": "not {json"}.

Minimal Reproduction

import asyncio
from typing import Any

from agent_framework import Content, FunctionTool

for text in ["not {json", "[1, 2]", '"text"']:
    call = Content.from_function_call("call-1", "list_accounts", arguments=text)
    print(f"{text!r:12} -> {call.parse_arguments()!r}")


async def list_accounts(**arguments: Any) -> str:
    print("tool called with", arguments)
    return "ok"


tool = FunctionTool(
    name="list_accounts",
    description="List the accounts",
    func=list_accounts,
    input_model={"type": "object", "properties": {}},
)
parsed = Content.from_function_call("call-1", "list_accounts", arguments="not {json").parse_arguments()
asyncio.run(tool.invoke(arguments=parsed))

Output on agent-framework-core 1.22.0:

'not {json'  -> {'raw': 'not {json'}
'[1, 2]'     -> {'raw': [1, 2]}
'"text"'     -> {'raw': 'text'}
tool called with {'raw': 'not {json'}

Where this happens on main at f7c6928:

parse_arguments returns the raw dictionary for a value that is not an object and for a decode error:

agent-framework/python/packages/core/agent_framework/_types.py

Lines 2188 to 2196 in f7c6928

    
           if isinstance(self.arguments, str): 
        
               # If arguments are a string, try to parse it as JSON 
        
               try: 
        
                   loaded = json.loads(self.arguments) 
        
                   if isinstance(loaded, dict): 
        
                       return loaded  # type: ignore[return-value] 
        
                   return {"raw": loaded} 
        
               except (json.JSONDecodeError, TypeError): 
        
                   return {"raw": self.arguments}

The loop parses with it:

agent-framework/python/packages/core/agent_framework/_tools.py

Line 2242 in f7c6928

parsed_args: dict[str, Any] = dict(function_call_content.parse_arguments() or {})

The top-level checks look at required, additionalProperties: false, enum and the property types only, so an undeclared raw key passes unless the schema forbids extra keys:

agent-framework/python/packages/core/agent_framework/_tools.py

Lines 1619 to 1631 in f7c6928

    
           """Run lightweight, top-level argument checks for schema-supplied tools.""" 
        
           parsed_arguments = dict(arguments) 
        
           required_fields = [field for field in schema.get("required", []) if isinstance(field, str)] 
        
           missing_fields = [field for field in required_fields if field not in parsed_arguments] 
        
           if missing_fields: 
        
               raise TypeError(f"Missing required argument(s) for '{tool_name}': {', '.join(sorted(missing_fields))}") 
        
           properties: Mapping[str, Any] = schema.get("properties", {}) 
        
           if schema.get("additionalProperties") is False: 
        
               unexpected_fields = sorted(field for field in parsed_arguments if field not in properties) 
        
               if unexpected_fields: 
        
                   raise TypeError(f"Unexpected argument(s) for '{tool_name}': {', '.join(unexpected_fields)}")

Error Messages and Stack Traces

None. That is the problem: the malformed call is reported nowhere.

Package Versions

agent-framework-core: 1.22.0 (the code is the same on main at f7c6928)

Python Version

Python 3.12

Operating System

macOS

🤖 Automated triage reproduction notes (agent-authored — trust but verify)

Agent analysis

Repro: python/packages/core/agent_framework/_tools.py::_auto_invoke_function at line 2242 accepts Content.parse_arguments() fallback mappings as ordinary arguments. Invalid JSON or valid non-object JSON triggers this when the tool has a permissive object schema, causing its callable to receive a synthetic raw keyword. Minimal repro: create that FunctionTool with a **kwargs callable and await _auto_invoke_function on arguments not {json; the callable receives {"raw": "not {json"}.

Failing test: python/packages/core/tests/core/test_tools.py
Files examined: python/AGENTS.md, python/packages/core/AGENTS.md, docs/specs/004-python-function-calling-loop.md, python/packages/core/agent_framework/_types.py, python/packages/core/agent_framework/_tools.py, python/packages/core/tests/core/test_tools.py, python/packages/core/pyproject.toml, python/pyproject.toml
Tests run: test_auto_invoke_preserves_explicit_null_argument, test_auto_invoke_rejects_arguments_that_are_not_json_objects
Reported version: 1.22.0
Current version: 1.22.0

Fixed in PR #9573.

The cause is where the report pointed: the loop used Content.parse_arguments() as the arguments of the call, and that method wraps a payload which is not a JSON object under a single "raw" key. For a tool whose schema has no required properties and does not set additionalProperties: false, the loop's own checks then passed — raw is just an extra key — so the tool was invoked with raw=... as if the model had made a well-formed call.

The pull request parses the payload strictly in the loop and answers a call whose arguments are not a JSON object the way it already answers a missing required property: with an argument-validation error result that names the cause, so the model can retry.

Two notes on the shape of the fix:

Content.parse_arguments() keeps its documented "raw" wrapping, so direct callers are unaffected; the check lives in the loop because that is the only place where the result becomes a tool invocation.
The error reuses the existing _FunctionArgumentValidationError path, so a malformed call produces the same kind of result as a missing required property, including the redacted message when argument snapshots are in play.

The reproduction from the report is in the pull request as a test, for all three shapes (not {json, [1, 2] and "text"), plus a case that a well-formed call still reaches the tool with its arguments unchanged. On the unmodified file the tool is invoked with {'raw': 'not {json'} and with {'raw': [1, 2]}; with the change the tool is not invoked at all and the error names the cause.

Thanks for the report — the raw fallback and the loop's use of it were the whole bug.

H23 (@Madin-H23)

H23 (@Madin-H23) Thanks for working on this. Has a maintainer confirmed the proposed direction? The contribution guide requires agreement before implementing non-trivial changes, or an explanation in the PR if the change qualifies as trivial. I could not find either here. Since this changes argument validation and whether tools execute, could we get maintainer confirmation?

SergeyMenshykh

Thanks for the pointer — you are right, and we are not treating it as settled.

To answer directly: no maintainer has confirmed the direction yet. The pull request reuses this issue (the first step the contribution guide asks for), and the triage bot has labelled the issue reproduced, but that is bot output, not maintainer agreement. The change also alters how the automatic function-calling loop validates arguments and whether a tool executes, so we do not think it qualifies as a trivial correction, and we are not claiming that it does.

What we are doing now:

We are asking the maintainers for the ready-for-implementation label on this issue, so that the agreed direction is explicit before the pull request goes through review. SergeyMenshykh — could you add the label, or point us at who should decide it?
If no maintainer confirms, we will close the pull request rather than leave it open without the agreement the contribution guide asks for. We would rather not hold a slot in this repository's queue for a change the maintainers have not agreed to.
The patch is still being iterated on: an automated review of our current diff flagged an edge case — a whitespace-only argument payload still slips through the guard and reaches the tool as a raw argument — which we are fixing together with a regression test. So this is a request for the direction to be agreed, not a request to merge now.

One piece of context so you can see the whole picture: the same applies to our other two open pull requests in this repository — #9571 for #9549 and #9574 for #9438. They are in the same state: an existing issue, the triage bot's reproduced label, and no maintainer agreement yet. (For #9549 the reporter also wrote in the issue that they would wait for explicit maintainer agreement before starting implementation, so asking is the only step we should have taken first.)

If you read either change as trivial in the guide's sense — an obvious correction of a wrong result rather than a behaviour decision — say so and we will put that reasoning in the pull request description instead of asking for the label. Either way we will not keep a pull request open here without one of those two on record.

Eduard van Valkenburg (@eavanvalkenburg)

Eduard van Valkenburg (@eavanvalkenburg) — one request about the label.

Per the contribution guide, a non-trivial change needs a maintainer to mark the direction as ready for implementation before the pull request is opened. We did not wait for that here, and we are asking to correct it: if the direction in this issue is welcome for implementation, could you add the ready-for-implementation label? We asked SergeyMenshykh on 2026-10-22 as well, and the label history in this repository shows it comes from you.

The pull request changes only how a function call whose arguments are not a JSON object is answered. The tool is no longer called with an undeclared raw argument; the model gets an error that names the reason and can retry from it. The public Content.parse_arguments() keeps its current wrapping, so direct callers are unaffected.

#9573

Either answer is fine. If there is no confirmation by 2026-10-27 we will close the pull request ourselves, as we said on this issue. We will not ask again.

/plan Coordinate with existing PR #9573 and the current core-team review; do not start a parallel implementation or treat this invocation as ready-for-implementation. Current main still invokes permissive tools using Content.parse_arguments() synthetic raw mappings. Preserve that public parser fallback and synthesize the agreed execution boundary: middleware must see and may repair parsed arguments, while an unrepaired malformed/non-object payload must not produce tool side effects. Incorporate the current wrapper-presence/NaN-safe review rather than repeating it. Explicitly settle zero-parameter null/whitespace compatibility, tools declaring raw, approval/resume ordering, snapshots/redaction, and the separate MCP sampling conversion/error contract. Map affected scenarios to spec 004; require streaming/finalized/non-streaming and history/caller-visible coverage, full core plus affected provider/transport tests, typing and syntax checks. Core-team scope/ownership agreement is still required; no new assignment or ready label while planning.

Summary

Content.parse_arguments() wraps non-object or undecodable arguments as {"raw": ...}. On current main, _auto_invoke_function (_tools.py, around L2353) feeds that mapping to the tool unchanged as dict(function_call_content.parse_arguments() or {}). A tool with a permissive schema (no required properties, no additionalProperties: false) therefore runs with a synthetic raw argument. The model never learns its call was malformed. I read this in the code and did not run it. The issue is closed and PR #9573 is open. Per your direction I did not look at the PR's diff or reviews, and this plan stands on the repository alone.

Recommendation

Code changes are warranted, but not yet ready to implement. This is a real defect in the function-calling loop (spec 004). Docs or a sample would not help, because the model cannot work around a failure it is never told about. Core-team agreement on scope and ownership is still outstanding. This is a planning input only, not a ready-for-implementation signal. Whoever owns it should fold these decisions into #9573 rather than start a parallel change.

Why the boundary is subtle. The no-middleware path validates and then invokes straight away. The middleware path calls _prepare_arguments and deliberately lets a failure through so middleware can repair the arguments (spec 004, L433-437). A fix that rejects bad payloads before middleware would break that repair contract. A fix that only rejects at the end would be too late for no-middleware tools. The agreed shape is:

Parse in the loop so that the loop knows when the payload is synthetic.
Middleware sees the parsed arguments and may repair them.
The innermost handler refuses to invoke the tool if the arguments are still malformed, whether or not middleware ran.
The refusal returns the existing Argument parsing failed result, so the model can retry.

Proposed decisions to settle

Parser fallback. Keep parse_arguments() and its docstring unchanged for direct callers. Do not make it raise. The loop needs its own way to tell "the model sent a real object" from "synthetic wrapper". The review rule is wrapper presence, not the key name. A real object that happens to contain raw must still work.
NaN-safe comparison. Any "was this repaired?" check must reuse the existing float-bit-exact comparison (_argument_comparison_token, test_nan_prepared_and_approval_snapshots_are_stable). Plain equality would treat unchanged NaN as changed.
Zero-parameter tools and null/whitespace payloads.
- Today None becomes {}, and so does an empty string.
- Proposal: keep None and "" as "no arguments" for tools with no parameters, so that currently working calls do not start failing.
- Treat whitespace-only text (the case the PR author flagged) as malformed only if it is not valid JSON.
- Decide explicitly whether a literal JSON null counts as "no arguments". I lean toward yes for compatibility, but this needs a core-team decision, and it needs a test either way.
Tools that declare raw. A tool whose own schema has a raw parameter must keep working when the model sends a real object. Only a synthetic wrapper is rejected. Add a test for a declared raw with a real object, and one where the model sends a non-object payload.
Approval and resume ordering. Approval binds to the normalized arguments seen at middleware entry (spec L732). Decide whether a malformed call can request approval at all.
- My proposal: a malformed call produces the validation-error result before any approval request, so nothing is approved against a synthetic value.
- A repaired call follows the existing replacement-approval rule.
- A resumed approved call must re-check the recorded arguments, not trust the grant.
Snapshots and redaction. Keep the existing redacted-message path (_FunctionArgumentValidationError.redacted_message). The error must name the cause ("arguments are not a JSON object") without echoing the payload or hidden values. Approval and security snapshots must not store the synthetic raw mapping as if it were model intent.
MCP sampling is a separate contract. _mcp.py (around L1643) calls parse_arguments() or {} to build a ToolUseContent input. A raw wrapper there goes out as if it were real tool input to an MCP server. That is a conversion and error contract of its own: reject, skip the content, or raise a sampling error. Decide it separately and do not let the loop change silently alter it. The same applies to the other callers (security.py around L1266 and L1409, and _harness/_tool_approval.py around L41), which should be audited and either kept or adjusted deliberately.

Proposed plan

Get core-team sign-off on scope, ownership and decisions 3-7 above. Add the agreed result to Python: fix(tools): answer a function call whose arguments are not a JSON object #9573.
In _tools.py, make the loop detect a synthetic wrapper once. Carry that flag through the direct, middleware, approval-replay and short-circuit paths. Reuse the existing validation-error result.
Refuse invocation in the innermost handler if the arguments are still malformed after middleware. Also refuse on approval resume.
Leave Content.parse_arguments() untouched, then audit its other callers as in decision 7. Make a separate decision for MCP sampling.
Update spec 004 in the smallest affected sections. Add a scenario row mapping the new tests: unrepaired malformed payload, middleware repair, short-circuit, security middleware ordering, approval, resume, redaction, zero-parameter compatibility and a declared raw.
Add tests in streaming, finalized-stream and non-streaming modes. Each should check both the history (what is persisted) and the caller-visible result. Cover each non-object shape (not {json, [1, 2], "text", whitespace, null).
Add tests for the affected provider and transport packages, plus MCP sampling.

Validation: the full core test suite plus affected provider and transport tests (python/packages/core, ag-ui, any providers touched), typing (pyright/mypy via uv run poe), and Ruff/syntax checks. Spec 004's "Minimum validation commands" lists the required set.

Done means: a malformed or non-object payload that nobody repairs never runs a tool body and gets an error the model can answer, in every mode. Repaired and well-formed calls behave as before. parse_arguments() still returns its wrapper. No changes have been made.

Generated by Issue Planning for #9562 · copilot · auto · ~35 AIC · ⌖ ~1 AIC · ⊞ 4.8K · ◷

Navigation Menu

Repository navigation

Python: [Bug]: Function call arguments that are not a JSON object reach the tool as {"raw": ...} instead of failing validation #9562

Observed Behavior

Expected Behavior

Steps to Reproduce

Minimal Reproduction

Error Messages and Stack Traces

Package Versions

Python Version

Operating System

Summary

Recommendation

Proposed plan

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	if isinstance(self.arguments, str):
	# If arguments are a string, try to parse it as JSON
	try:
	loaded = json.loads(self.arguments)
	if isinstance(loaded, dict):
	return loaded # type: ignore[return-value]
	return {"raw": loaded}
	except (json.JSONDecodeError, TypeError):
	return {"raw": self.arguments}

	"""Run lightweight, top-level argument checks for schema-supplied tools."""
	parsed_arguments = dict(arguments)

	required_fields = [field for field in schema.get("required", []) if isinstance(field, str)]
	missing_fields = [field for field in required_fields if field not in parsed_arguments]
	if missing_fields:
	raise TypeError(f"Missing required argument(s) for '{tool_name}': {', '.join(sorted(missing_fields))}")

	properties: Mapping[str, Any] = schema.get("properties", {})
	if schema.get("additionalProperties") is False:
	unexpected_fields = sorted(field for field in parsed_arguments if field not in properties)
	if unexpected_fields:
	raise TypeError(f"Unexpected argument(s) for '{tool_name}': {', '.join(unexpected_fields)}")

Python: [Bug]: Function call arguments that are not a JSON object reach the tool as {"raw": ...} instead of failing validation #9562

Description

Observed Behavior

Expected Behavior

Steps to Reproduce

Minimal Reproduction

Error Messages and Stack Traces

Package Versions

Python Version

Operating System

Activity

github-actions commented on Oct 20, 2026

Madin-H23 commented on Oct 21, 2026

danielquintas8 commented on Oct 21, 2026

Madin-H23 commented on Oct 22, 2026

Madin-H23 commented on Oct 22, 2026

eavanvalkenburg commented on Oct 26, 2026

eavanvalkenburg commented on Nov 11, 2026

eavanvalkenburg commented on Nov 12, 2026

github-actions commented on Nov 12, 2026

Summary

Recommendation

Proposed plan

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions