Repository navigation
Description
Bug Description
The PowerShell helpers print their -Json output with ConvertTo-Json, which leaves non-ASCII characters as they are. When an agent captures that stdout, Windows PowerShell 5.1 and PowerShell 7 encode it with the console output code page (437 or 1252 on most Western setups, 936 on Chinese ones), not UTF-8. Characters missing from the code page become ?, and the rest arrive as bytes that are not valid UTF-8.
This is the PowerShell side of #5048, which fixed the same problem in the Python helpers by writing their --json output with ensure_ascii=True.
What an agent gets today (stdout captured with a new hidden console, which keeps the default code page, here 437):
- Since [bug-fix] Fix non-latin-feature-names: preserve Unicode feature names #4780, feature names keep non-Latin letters.
create-new-feature.ps1 -Json "添加用户导出"createsspecs\001-添加用户导出, but prints"BRANCH_NAME":"001-??????"and aSPEC_FILEunder001-??????.setup-plan.ps1 -Jsonandcheck-prerequisites.ps1 -Jsonthen print the same?paths, so the agent looks for files that do not exist. resolve-template.ps1 constitution-template -Json, which/speckit.constitutionruns, cannot be parsed at all with code page 437: the stock constitution template contains→, which code page 437 writes as byte 0x1A, a raw control character inside a JSON string. This happens even when every name and path is ASCII.setup-tasks.ps1 -Jsonhas the same problem with the stock tasks template (emoji), and every helper has it with a non-ASCII project path.
Steps to Reproduce
Windows 11, from Python, capturing stdout the way an agent does:
uvx --from git+https://github.com/github/spec-kit.git@v1.4.0 specify init demo --integration codex --script ps --ignore-agent-tools --non-interactiveimport subprocess
r = subprocess.run(
["powershell.exe", "-NoProfile", "-File", r".specify\scripts\powershell\create-new-feature.ps1", "-Json", "添加用户导出"],
cwd="demo", capture_output=True, creationflags=subprocess.CREATE_NO_WINDOW,
)
print(r.stdout)Expected Behavior
JSON whose values are the real names, paths and template text, as the Python helpers print since #5048.
Actual Behavior
{"BRANCH_NAME":"001-??????","SPEC_FILE":"C:\\...\\demo\\specs\\001-??????\\spec.md","FEATURE_NUM":"001"}
For resolve-template.ps1 constitution-template -Json, json.loads fails with Invalid control character at: line 1 column 690.
Specify CLI Version
v1.4.0. main (2ee2b1e) has the same scripts.
AI Agent
Codex integration for the reproduction; the scripts are agent-independent.
Operating System
Windows 11, Windows PowerShell 5.1.26100 and PowerShell 7.7.0, console code page 437
Python Version
3.13.5
Error Logs
See Actual Behavior.
Additional Context
I have a fix with regression tests and will open a PR for it right away.
AI Disclosure
Found, reproduced and written up by OpenCode with Claude Opus 5.5 in autonomous agent mode, on behalf of @adityajai119. The example above was run locally.
Activity
github-actions commentedon Nov 16, 2026
Bug assessment — powershell-json-codepage: Valid · severity high
Bug Assessment: PowerShell JSON output is encoded with console code page
Report (summarized)
Issue author @adityajai119 reports that several PowerShell helpers emit
-Jsonoutput viaConvertTo-Jsondirectly to stdout, which is then encoded by the host code page (commonly cp437/cp1252/cp936 on Windows) when captured by agents.Observed failures in the report:
?(example:001-添加用户导出becomes001-??????).json.loadsfailure with an invalid control character).Issue context also references an open fix PR: #5417.
No additional issue comments were present at assessment time.
Symptom
When PowerShell helper scripts run in
-Jsonmode under Windows with stdout captured by an agent/process, JSON payloads that contain non-ASCII text (feature names, paths, template content) are emitted in the active console code page instead of transport-safe UTF-8/ASCII-escaped JSON. Consumers parsing UTF-8 JSON receive corrupted data or parse errors.Expected behavior is stable, parseable JSON preserving semantic values (paths, names, template text), matching the Python helpers’ guarded JSON behavior.
Reproduction
-Jsonmode with stdout captured by a subprocess (hidden/no-window capture as reported).添加用户导出, constitution/tasks template content with Unicode symbols).[NEEDS CLARIFICATION: whether every affected host path reproduces identically across all Windows locales/code pages, though code-path evidence indicates broad exposure.]
Suspected Code Paths
scripts/powershell/create-new-feature.ps1:348— emits JSON viaConvertTo-Json -Compressdirectly to stdout.scripts/powershell/check-prerequisites.ps1:174(and:84) — JSON emission paths for regular/paths output use directConvertTo-Json -Compress.scripts/powershell/setup-plan.ps1:84— JSON result emitted directly withConvertTo-Json -Compress.scripts/powershell/resolve-template.ps1:35— emits template content JSON directly viaConvertTo-Json -Compress.scripts/powershell/setup-tasks.ps1:79— emits JSON payload containing template content/path values directly.scripts/powershell/common.ps1:532(Resolve-TemplateContent) — returns UTF-8 template text that later flows into JSON payload fields; safe read path, but output path above is unsafe.templates/constitution-template.md:14— contains→in stock template content that can trigger encoding-path failures when serialized into JSON text output.templates/commands/tasks.md:172— contains Unicode symbols (e.g.,✅) in stock template content, same risk when emitted through affected JSON path.Root Cause Hypothesis
The PowerShell scripts serialize objects with
ConvertTo-Jsonand write the resulting Unicode string to stdout without forcing a transport-safe representation for redirected/captured output. On Windows PowerShell 5.1 and PowerShell 7, captured stdout commonly uses a legacy code page unless explicitly controlled; characters outside that code page are replaced or transformed, yielding lossy bytes or invalid JSON text for UTF-8 parsers. Confidence: high.Proposed Remediation
Preferred: Add a shared JSON output helper in
scripts/powershell/common.ps1that serializes withConvertTo-Json -Compress, then escapes non-ASCII code points to\uXXXX(including correct surrogate-pair escapes for non-BMP characters), and writes the final ASCII-only JSON line to stdout. Replace direct JSON emission sites in affected scripts (create-new-feature.ps1,check-prerequisites.ps1,setup-plan.ps1,resolve-template.ps1,setup-tasks.ps1) to use this helper.This mirrors the robust transport approach already used in Python (
ensure_ascii=True) for command JSON output, avoiding dependence on host code page behavior while preserving exact decoded values for consumers.Alternatives (optional):
[Console]::OutputEncoding = ...). Trade-off: less deterministic across hosts/invocation paths and riskier side effects for mixed text/JSON output flows.-Jsonbranches. Trade-off: more invasive per-script plumbing and duplication versus one shared helper.Files likely to change:
scripts/powershell/common.ps1scripts/powershell/create-new-feature.ps1scripts/powershell/check-prerequisites.ps1scripts/powershell/setup-plan.ps1scripts/powershell/resolve-template.ps1scripts/powershell/setup-tasks.ps1tests/test_create_new_feature_python_parity.pytests/test_check_prerequisites_python_parity.pytests/test_setup_plan_python_parity.pytests/test_setup_tasks_python_parity.pytests/test_resolve_template_python_parity.pyTests to add or update:
-Jsonstdout in a legacy code page context and assert JSON parsing + value fidelity for:Risks & Considerations
-Json.Open Questions
extensions/.../resolve-tasks.ps1,create-new-feature-branch.ps1) for consistency and future-proofing?]