Skip to content

fix(powershell): write -Json output of the PowerShell helpers as ASCII - #5417

Open
adityajai119 wants to merge 2 commits into
github:mainfrom
adityajai119:fix/5416-powershell-ascii-json
Open

adityajai119 wants to merge 2 commits into
github:mainfrom
adityajai119:fix/5416-powershell-ascii-json

Conversation

@adityajai119

Copy link
Copy Markdown
Contributor

Description

Fixes #5416.

This is the PowerShell side of #5048. The PowerShell helpers print their -Json output with ConvertTo-Json, which leaves non-ASCII characters as they are, and a captured stdout is encoded with the console output code page (437 or 1252 on most Western setups), not UTF-8. So when an agent runs them:

  • create-new-feature.ps1 -Json "添加用户导出" creates specs\001-添加用户导出 but prints "BRANCH_NAME":"001-??????", and setup-plan.ps1 and check-prerequisites.ps1 print the same ? paths. Feature names have kept non-Latin letters since [bug-fix] Fix non-latin-feature-names: preserve Unicode feature names #4780.
  • resolve-template.ps1 constitution-template -Json (used by /speckit.constitution) cannot be parsed with code page 437: the stock template's → becomes byte 0x1A, a raw control character inside a JSON string.
  • setup-tasks.ps1 -Json has the same problem with the stock tasks template's emoji.

This PR adds a small ConvertTo-AsciiJson helper to common.ps1. It runs ConvertTo-Json -Compress and writes every non-ASCII character as a \uXXXX escape, so characters outside the BMP (such as 🎯) become surrogate-pair escapes. The six -Json outputs in create-new-feature.ps1, setup-plan.ps1, check-prerequisites.ps1 (two), setup-tasks.ps1 and resolve-template.ps1 use it. JSON parsers read back exactly the same values, so the output no longer depends on the console code page, and it matches the Python helpers (ensure_ascii=True since #5048, or UTF-8 stdout in create_new_feature.py). Output without -Json is unchanged, as in #5048.

Not changed: the git extension's create-new-feature-branch.ps1 also prints JSON with ConvertTo-Json. Two open PRs (#5404 and #5390) touch that file, so I left it for a follow-up.

Testing

  • Tested locally with .\.venv\Scripts\specify.exe --help
  • Ran relevant existing tests with .\.venv\Scripts\python.exe -m pytest tests -q, using the affected test paths: the five *_python_parity.py files above plus test_check_prerequisites_paths_only.py, test_setup_plan_no_overwrite.py and test_timestamp_branches.py give 220 passed, 316 skipped with PowerShell 7.7.0 and 199 passed, 337 skipped with Windows PowerShell 5.1.26100 (the skips are bash cases and the other host's cases)
  • Tested with a sample project (if applicable)

Five new tests, one next to each Python legacy-stdout test from #5048: test_powershell_json_preserves_non_ascii_feature_on_legacy_stdout (setup-plan, check-prerequisites), test_powershell_json_preserves_non_ascii_template_on_legacy_stdout (setup-tasks), test_powershell_json_preserves_constitution_on_legacy_stdout (resolve-template) and test_powershell_json_outputs_unicode_on_legacy_stdout (create-new-feature). They run the script through a new ps_legacy_stdout_cmd helper in tests/parity_helpers.py, which sets [Console]::OutputEncoding to ISO-8859-1 for the run and restores it afterwards. It stands in for a legacy console code page, the way #5048's tests use PYTHONIOENCODING=cp1252. With main's scripts all five fail on both PowerShell 7.7.0 and Windows PowerShell 5.1.26100; with this change all five pass on both.

Sample project: specify init demo --integration codex --script ps --ignore-agent-tools --non-interactive from v1.4.0 and from main (2ee2b1e). I ran the helpers from Python with stdout captured and a new hidden console (code page 437):

Helper v1.4.0 / main This PR
create-new-feature.ps1 -Json "添加用户导出" BRANCH_NAME is 001-?????? 001-添加用户导出
setup-plan.ps1 -Json IMPL_PLAN under 001-?????? the real plan.md path
check-prerequisites.ps1 -Json -PathsOnly FEATURE_SPEC under 001-?????? the real spec.md path
resolve-template.ps1 constitution-template -Json not valid JSON (Invalid control character) parses; content equals the template

The same results with Windows PowerShell 5.1 and PowerShell 7.7.0.

Also ran uvx ruff@0.15.0 check src tests (all checks passed) and git diff --check (clean), and parsed the six scripts with the Windows PowerShell 5.1 parser (no errors).

AI Disclosure

  • I did not use AI assistance for this contribution
  • I did use AI assistance (fill in the disclosure below)

AI disclosure: OpenCode with Claude Opus 5.5 (max reasoning effort, autonomous agent mode), acting on behalf of @adityajai119, found and reproduced the bug, wrote the fix and the tests, ran the checks above, and drafted this description. The commit carries an Assisted-by: OpenCode (model: Claude Opus 5.5, autonomous) trailer.

ConvertTo-Json leaves non-ASCII characters as they are, and a captured
stdout uses the console output code page, not UTF-8. Non-Latin feature
names and paths came out as ? and the stock constitution template's
arrow became a raw control character, so the JSON was wrong or
unparseable. Escape non-ASCII characters as \uXXXX in the -Json output
of create-new-feature, setup-plan, check-prerequisites, setup-tasks and
resolve-template, as github#5048 did for the Python helpers.

Fixes github#5416

Assisted-by: OpenCode (model: Claude Opus 5.5, autonomous)
@adityajai119
adityajai119 requested a review from mnriem as a code owner November 12, 2026 18:50
Copilot AI adaptive review requested due to automatic review settings November 12, 2026 18:50

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the changes preserving Unicode content in PowerShell JSON output.

No open issues 🎉

What changed in this PR

Fixes #5416 by making core PowerShell helpers emit ASCII-safe JSON, preserving Unicode feature names and template content across legacy console code pages. This complements the Python fix in #5048 without changing non-JSON output.

Changes:

  • Adds ConvertTo-AsciiJson, escaping non-ASCII characters while preserving decoded JSON values.
  • Updates all six JSON output sites across five core helpers.
  • Adds five regression tests and a shared legacy-encoding test helper.

The PR reports before-and-after regression results on Windows PowerShell 5.1 and PowerShell 7; these were not independently rerun.

File Description
tests/​test_setup_tasks_python_parity.py Tests Unicode template content under legacy stdout encoding.
tests/​test_setup_plan_python_parity.py Tests Unicode feature paths under legacy stdout encoding.
tests/​test_resolve_template_python_parity.py Tests Unicode constitution content under legacy stdout encoding.
tests/​test_create_new_feature_python_parity.py Tests Unicode feature names and returned spec paths.
tests/​test_check_prerequisites_python_parity.py Tests Unicode feature paths in prerequisites JSON.
tests/​parity_helpers.py Adds a PowerShell command wrapper using legacy stdout encoding.
scripts/​powershell/​setup-tasks.ps1 Uses ASCII-safe serialization for tasks JSON.
scripts/​powershell/​setup-plan.ps1 Uses ASCII-safe serialization for plan JSON.
scripts/​powershell/​resolve-template.ps1 Uses ASCII-safe serialization for template JSON.
scripts/​powershell/​create-new-feature.ps1 Uses ASCII-safe serialization for feature JSON.
scripts/​powershell/​common.ps1 Defines the shared ASCII-safe JSON serializer.
scripts/​powershell/​check-prerequisites.ps1 Updates both JSON output modes to use the serializer.

Review effort: Balanced (auto)

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified in the PowerShell ASCII-safe JSON serialization changes.

No open issues 🎉

Review effort: Balanced (dynamic)

@mnriem

mnriem commented Nov 16, 2026

Copy link
Copy Markdown
Collaborator

Please resolve conflicts

Assisted-by: OpenCode (model: Gemini 4 Carbon, autonomous)
Copilot AI adaptive review requested due to automatic review settings November 17, 2026 07:23
@adityajai119

Copy link
Copy Markdown
Contributor Author

Merged main and resolved the tests/parity_helpers import conflict in 7e36b2b (pytest and ruff pass).

AI disclosure: drafted and verified autonomously by OpenCode (model: Gemini 4 Carbon).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues block the changes to preserve Unicode in PowerShell JSON output under legacy console encodings.

No open issues 🎉

Review effort: Balanced (adaptive)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

triage-must-have Verdict: high-value, important work for Spec Kit — do first

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: PowerShell helpers write -Json output in the console code page, so non-ASCII names and template text arrive as ?

3 participants