Skip to content

Python: report an over-long Windows path as ENAMETOOLONG, not a missing file - #10202

Open
Simin Yuan (simin-yuan) wants to merge 3 commits into
microsoft:mainfrom
simin-yuan:fix/filesystem-store-enametoolong-r2
Open

Simin Yuan (simin-yuan) wants to merge 3 commits into
microsoft:mainfrom
simin-yuan:fix/filesystem-store-enametoolong-r2

Conversation

@simin-yuan

Copy link
Copy Markdown

Motivation & Context

Fixes #10141. On Windows without long-path support, FileSystemAgentFileStore.write() reports a key whose absolute path is past the OS limit as FileNotFoundError, which reads as "the file is missing" and sends callers looking for a bug in their own path handling.

What the OS actually raises depends on the call site. Measured on Windows 11 23H2 (OS build 22631.6199, LongPathsEnabled=0, Python 3.11.16, main at 13956ad), with the store rooted at a 9-character path:

absolute path write() today underlying error
257 succeeds —
261 OSError(38) ENAMETOOLONG FileNotFoundError, errno 2, no winerror
303 OSError(38) ENAMETOOLONG FileNotFoundError, errno 2, WinError 3
310 OSError(38) ENAMETOOLONG FileNotFoundError, errno 2, no winerror
361 OSError(38) ENAMETOOLONG FileNotFoundError, errno 2, WinError 3

The write() column is what this patch produces; the right-hand column is the exception chained as __cause__, i.e. what the OS raised before the rewrite. ERROR_FILENAME_EXCED_RANGE (206) does not appear for the calls write() makes: Path.mkdir(parents=True) surfaces WinError 3 and os.open surfaces none. That is why the detection here keys on the resolved path length next to an error that already happened, rather than on a particular winerror.

Description & Review Guide

  • What are the major changes? Both except OSError blocks in _write_file_sync now call _raise_if_path_too_long(exc, path, *, limit), which re-raises the failure as OSError(errno.ENAMETOOLONG) chained to the original error when the path is at or past the Windows limit and a prefix inside the limit exists. Directories use 248 characters and files 260, from the documented MAX_PATH rules. _WINDOWS_PATH_LIMITS_APPLY pins sys.platform == "win32" so the tests can exercise both branches on any CI OS.
  • What is the impact of these changes? A path that is merely too long now says so instead of reading as missing. The length is consulted only inside an except block, never before the call, so a host with long-path support behaves exactly as it does today and every in-limit path is untouched.
  • Scope: this covers write() only. read(), delete(), list_children() and file_exists() still report an over-long path as absent — whether they should raise instead, and whether \\?\ should be used rather than raising, are your call (asked in the issue).
  • What do you want reviewers to focus on? Whether the 248/260 split is the right way to express the limit, and whether requiring a prefix inside the limit to exist is the condition you want (it keeps an over-long store root reporting as missing rather than as "too long").

Testing

  • python/packages/core/tests/core/test_harness_file_access.py → 148 passed, 8 skipped on Windows 11 with LongPathsEnabled=0.
  • New: test_filesystem_store_normalizes_windows_path_too_long_error[mkdir|open] — the failure is injected at both call sites and the platform flag is pinned, so it runs on every CI OS.
  • New: test_filesystem_store_never_rejects_a_path_past_the_windows_limit — a key past the limit must still be written and read when the host supports long paths. This is the guarantee that a length check applied before the OS call would break; it is skipped on the host used above (that host rejects such paths) and runs on a host with long paths enabled and on Linux.
  • test_filesystem_store_propagates_other_file_not_found_errors still passes: an unrelated FileNotFoundError keeps its type and identity.
  • ruff format --check, ruff check, pyright (with pyrightconfig.tests.json) → clean.

Known gap: LongPathsEnabled=1 local run pending. Flipping that switch needs an elevated registry write, which the machine used here does not have. The expectation is that all five shapes above simply succeed on such a host; I will add the measured result to this description once the run is done.

Related Issue

Fixes #10141

AI Assistance

  • This is an AI-assisted contribution. I reviewed, understood, and verified all submitted content and accept responsibility for it.

AI assistance details: implementation, tests, and this description were drafted with an AI assistant. I ran every verification listed above myself on the Windows machine described.

…ng file

On Windows without long-path support, FileSystemAgentFileStore.write() reported a path
that merely exceeded the OS limit as FileNotFoundError, which reads as a missing file
and sends callers looking for a bug in their own path handling.

The limit is now used only to explain an error the OS already raised: when write() fails
and a prefix of the path that is still inside the limit exists, the failure is re-raised
as OSError(errno.ENAMETOOLONG), chained to the original error. Paths are never rejected
up front, so a host with long-path support keeps its current behaviour and every
in-limit path is untouched. Directories use the 248-character directory limit, files the
260-character one.

The within-bound half of test_filesystem_store_matches_in_memory_complexity_bound now
expects ENAMETOOLONG on a Windows host with long paths disabled. New tests cover both
call sites with a pinned platform flag, check that unrelated FileNotFoundErrors still
propagate unchanged, and check that a path past the limit is still written and read when
the host supports long paths.

Fixes microsoft#10141
Copilot AI balanced review requested due to automatic review settings November 9, 2026 13:37
@agent-framework-automation agent-framework-automation Bot added the python Usage: [Issues, PRs], Target: Python label Nov 9, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The long-path test can fail depending on the temporary-directory location, while error normalization masks collision and permission failures and incorrectly counts Windows path lengths.

3 open issues
What changed in this PR

This PR addresses #10141 by translating over-long Windows path failures during FileSystemAgentFileStore.write() into ENAMETOOLONG, helping callers distinguish path-length limits from missing files.

Changes:

  • Adds error normalization using separate Windows file and directory path limits.
  • Documents platform-specific differences between filesystem and in-memory stores.
  • Adds regression tests for write failures, successful long-path writes, and unrelated missing-file errors.
File Description
python/​packages/​core/​tests/​core/​test_harness_file_access.py Adds Windows path-limit tests and adjusts backend compatibility expectations.
python/​packages/​core/​agent_framework/​_harness/​_file_access.py Adds write-error normalization and documents Windows path limits.

Review effort: Balanced

Count Windows path limits in UTF-16 code units instead of Python code
points, restrict the ENAMETOOLONG rewrite to FileNotFoundError so
collisions and permission failures keep their own types, and size the
regression test's file name from the temporary directory so the path
always crosses MAX_PATH.
@simin-yuan
Simin Yuan (simin-yuan) force-pushed the fix/filesystem-store-enametoolong-r2 branch from bc4f2e3 to b2fd616 Compare November 10, 2026 15:40
@simin-yuan

Copy link
Copy Markdown
Author

Eduard van Valkenburg (@eavanvalkenburg) thanks for the detailed review — all three points are addressed in b2fd616.

  1. Test path length — the file name is now sized from current (max(1, 260 - current - 1)), so the absolute path always reaches 260 while the parent stays below 248, wherever tmp_path lands. I reproduced the [open] failure locally with a 38-character temp path before the change; it passes now at several short temp-path lengths.
  2. UTF-16 code units — added _windows_path_length(), which counts utf-16-le code units (with surrogatepass), and both the guard and the in-limit prefix loop use it. A regression test covers an emoji path that is 259 code points but 260 UTF-16 units.
  3. Error scoping — _raise_if_path_too_long() now returns unless the error is a FileNotFoundError, so FileExistsError and PermissionError keep their types. A parametrized regression test covers both on a 260-character path.

Local verification on Windows 11, Python 3.12, LongPathsEnabled=0: full test_harness_file_access.py → 151 passed, 8 skipped; ruff check and ruff format --check clean. I also ran the new regression tests against the pre-fix module: they fail there and pass now.

One thing I cannot do from the fork: the workflow runs for b2fd616 are sitting in action_required, so CI has not started. Could you approve the run when you get a chance? I'll follow up once it is green.

@simin-yuan

Copy link
Copy Markdown
Author

Eduard van Valkenburg (@eavanvalkenburg) all three points are addressed in b2fd616, and I've just updated the branch. I resolved the six review threads; the specifics, so you can check them without reopening the diff:

  1. Test path length — the file name is now sized from the actual root: max(1, 260 - len(str(store.root_path)) - 1), so the final path reaches 260 regardless of where tmp_path lives.
  2. Normalization scope — _raise_if_path_too_long() only rewrites a FileNotFoundError whose prefix inside the limit actually exists. A collision or permission failure keeps FileExistsError / PermissionError, and hosts with long-path support are untouched.
  3. UTF-16 length — _windows_path_length() counts len(path.encode(utf-16-le, surrogatepass)) // 2, so non-BMP keys are measured the way Windows measures them.

I verified these against the head file rather than only the diff; I did not re-run the Windows test suite locally on this machine for this commit. If any of the three still differs from what you intended, say which and I'll reopen it.

This branch was successfully deployed

1 active deployment
github-app-auth — f488ec58 Deployed Nov 10, 2026 by simin-yuan via add_label #27539
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: [Bug]: On Windows without long paths, FileSystemAgentFileStore reports FileNotFoundError for long-but-legal keys

3 participants