Summary
Three issues were discovered in our imported official test suites that cause false failures or misleading results. These are problems with test infrastructure/test assets themselves, not with MoonBit implementation correctness.
Reproduction Environment
All issues reproduce with the safe test runner parameters:
pnpm exec vitest run --no-cache --pool=forks --maxWorkers=1 --fileParallelism=false --silent <suite>
This matches the execution model of scripts/test-safe.sh (fork + single worker).
Issues
1. parse-errors.comparison — exit code fallback logic is incorrect
- File:
tests/comparison/parse-errors.comparison.test.ts:27
- Problem: The original logic uses
err.status || 1. Under the fork pool, execSync can throw an error object (e.g. code: "EPERM") that carries status: 0 with normal stdout. The || 1 coercion treats status: 0 as falsy, replacing it with 1, which corrupts the real bash baseline exit code.
- Impact: 11 cases that should pass (
for, if, while, until, true, ||, blank commands, etc.) are falsely reported as failures.
- Fix: Change to
typeof err.status === "number" ? err.status : 1.
- Verification: After fix,
parse-errors.comparison.test.ts passes 24/24. Overall tests/comparison restores to 523/523 under safe parameters.
2. grep spec — stale skip entries cause UNEXPECTED PASS
- File:
tests/spec/grep/skips.ts
- Problem: Multiple skipped cases actually pass now, causing the runner to report
UNEXPECTED PASS errors. Typical categories:
- "RE2 doesn't support backreferences" entries (multiple)
- "RE2 treats grouped *** differently ..."
- "BRE literal
\{,2\} / \{,\} ... not implemented"
- Suggested fix: Remove stale skip entries. Separate remaining skips into "genuine implementation gaps" vs "historical compatibility policy differences" to avoid skip pollution of regression results.
3. jq spec — stale skip entries cause UNEXPECTED PASS
- File:
tests/spec/jq/skips.ts
- Problem: Multiple skipped cases now pass, causing
UNEXPECTED PASS. Typical entries:
Invalid \v escape sequence test
String interpolation with complex expression
Empty array/object pattern ...
Undefined variable $bar behavior differs
delpaths with large negative index
join error message
unique sort order differs
- Suggested fix: Review and remove stale skips. Add "expected removal condition" annotations to remaining skip entries for easier future cleanup.
Scope Clarification
These three items are test asset issues — problems with test infrastructure, baselines, or skip lists. They are distinct from implementation compatibility issues (grep/jq functional gaps), which should be tracked separately.
Checklist
Summary
Three issues were discovered in our imported official test suites that cause false failures or misleading results. These are problems with test infrastructure/test assets themselves, not with MoonBit implementation correctness.
Reproduction Environment
All issues reproduce with the safe test runner parameters:
This matches the execution model of
scripts/test-safe.sh(fork + single worker).Issues
1.
parse-errors.comparison— exit code fallback logic is incorrecttests/comparison/parse-errors.comparison.test.ts:27err.status || 1. Under the fork pool,execSynccan throw an error object (e.g.code: "EPERM") that carriesstatus: 0with normalstdout. The|| 1coercion treatsstatus: 0as falsy, replacing it with1, which corrupts the real bash baseline exit code.for,if,while,until,true,||, blank commands, etc.) are falsely reported as failures.typeof err.status === "number" ? err.status : 1.parse-errors.comparison.test.tspasses 24/24. Overalltests/comparisonrestores to 523/523 under safe parameters.2.
grepspec — stale skip entries cause UNEXPECTED PASStests/spec/grep/skips.tsUNEXPECTED PASSerrors. Typical categories:\{,2\}/\{,\}... not implemented"3.
jqspec — stale skip entries cause UNEXPECTED PASStests/spec/jq/skips.tsUNEXPECTED PASS. Typical entries:Invalid \v escape sequence testString interpolation with complex expressionEmpty array/object pattern ...Undefined variable $bar behavior differsdelpaths with large negative indexjoin error messageunique sort order differsScope Clarification
These three items are test asset issues — problems with test infrastructure, baselines, or skip lists. They are distinct from implementation compatibility issues (grep/jq functional gaps), which should be tracked separately.
Checklist
parse-errors.comparison.test.tsexit code fallbacktests/spec/grep/skips.tsstale entriestests/spec/jq/skips.tsstale entries