fix: recognize literal paths without splitting words - #437
Merged
Conversation
An agent repeatedly failed to read Git metadata because a dot-prefixed path became two adjacent arguments. The quoting diagnostic exposed the lexer's fragment boundary rather than a useful language rule. Recognize slash-containing relative words as paths, including numeric first components and bare current/parent directory references. Preserve their spelling and align @ and + handling across path categories. Extend colon fusion so Git revision paths remain one argument. The neighboring command-name audit found a silent split: ./bin$x could run ./bin with $x as its first argument. Accept literal relative executable paths and require a gap before the first argument. Leave adjacent equals signs with the assignment grammar so its diagnostics remain authoritative. Regression tests were run red first and now cover exact argv, redirects, assignments, executable names, interpolation boundaries, collections, globs, and the original command against fixture files. Update the help and language reference to explain literal paths independently of token count. Reviewed with kaibo: GLM-5.2 and DeepSeek V4 Flash, including a follow-up review after tests exposed diagnostic selection errors.
This was referenced Sep 8, 2026
tobert
added a commit
that referenced
this pull request
Sep 9, 2026
An assignment followed immediately by a home-relative path was tokenized as a regex comparison operator. Separate the assignment delimiter from the tilde path outside tests, while preserving the regex operator inside tests and respecting nested command-substitution scopes.
Home-relative assignments now parse:
```sh
p=~/x
local p=~/x
p={x:empty}; p[x]=~/x
foo --path=~/x
```
Regex comparisons retain their operator, including after logical operators:
```sh
[[ a == a && x=~/x ]]
[[ $(p=~/x; echo $p) =~ /x ]]
```
The lexer uses its existing test and substitution frames to distinguish these contexts. A closing bracket qualifies only after a statement-head assignment target, so a glob character class cannot become an assignment delimiter. Logical operators inside a test now retain that test frame. Split and joined tokens use the original source spans; text adjacent to interpolation still requires quoting. Runtime tilde expansion keeps its existing behavior.
Validation: 12 regression cases failed before the fix; all 27 targeted cases pass, including token spans, plain/local/bracket assignments, named arguments, nested substitutions, regex operators with and without spaces, interpolation refusal, and a glob bracket that must not be treated as an assignment target. Workspace tests, clippy across all targets with warnings denied, rustdoc with warnings denied, kernel tests without default features, and the WASI build passed. Help fragments and generated syntax are synchronized.
Reviewed through kaibo's default cast, GLM-5.2 with DeepSeek V4 Flash investigation. The review also covers the preceding literal-word layer's final numeric punctuation cases. This layer builds on #440 in the native stack rooted at #437; no version bump.
tobert
added a commit
that referenced
this pull request
Sep 9, 2026
For-loop item parsing accepted adjacent expressions as separate iterations. A command substitution followed immediately by a path suffix ran the substitution and then ran the loop body twice. Apply the existing quote-to-join rule to loop items and reject the complete adjacent run before execution, with a diagnostic pointing at the word to quote. Write one interpolated item: ```sh for x in "$(echo foo)/b"; do echo "$x"; done ``` Whitespace-separated items, standalone command-substitution newline splitting, and typed collection iteration keep their existing behavior. The argv diagnostic-rescan tests now use a valid test-condition boundary rather than the former loop exception. The overlay glob test also used that exception and missed an extra iteration; it now uses the explicit glob builtin and checks the exact returned paths. Validation: regression tests failed before the fix and pass afterward; workspace tests, clippy across all targets with warnings denied, and rustdoc with warnings denied. Added nested diagnostic spans, longer adjacent runs, separate runs, and a test that neither substitution nor body side effects execute on rejection. Help fragments and generated syntax are synchronized. Reviewed through kaibo by GLM-5.2 with DeepSeek V4 Flash investigation; no blocking findings. Review suggestions for nested spans and longer runs are covered. Built on #437 as the first follow-up layer; no version bump.
tobert
added a commit
that referenced
this pull request
Sep 9, 2026
An assignment followed immediately by a home-relative path was tokenized as a regex comparison operator. Separate the assignment delimiter from the tilde path outside tests, while preserving the regex operator inside tests and respecting nested command-substitution scopes.
Home-relative assignments now parse:
```sh
p=~/x
local p=~/x
p={x:empty}; p[x]=~/x
foo --path=~/x
```
Regex comparisons retain their operator, including after logical operators:
```sh
[[ a == a && x=~/x ]]
[[ $(p=~/x; echo $p) =~ /x ]]
```
The lexer uses its existing test and substitution frames to distinguish these contexts. A closing bracket qualifies only after a statement-head assignment target, so a glob character class cannot become an assignment delimiter. Logical operators inside a test now retain that test frame. Split and joined tokens use the original source spans; text adjacent to interpolation still requires quoting. Runtime tilde expansion keeps its existing behavior.
Validation: 12 regression cases failed before the fix; all 27 targeted cases pass, including token spans, plain/local/bracket assignments, named arguments, nested substitutions, regex operators with and without spaces, interpolation refusal, and a glob bracket that must not be treated as an assignment target. Workspace tests, clippy across all targets with warnings denied, rustdoc with warnings denied, kernel tests without default features, and the WASI build passed. Help fragments and generated syntax are synchronized.
Reviewed through kaibo's default cast, GLM-5.2 with DeepSeek V4 Flash investigation. The review also covers the preceding literal-word layer's final numeric punctuation cases. This layer builds on #440 in the native stack rooted at #437; no version bump.
tobert
added a commit
that referenced
this pull request
Sep 9, 2026
The lexer classified prefixes of ordinary literal words as numbers, keywords, or flags, then rejected the pieces as adjacent arguments. Recognize numeric filenames and version strings as whole words, accept embedded plus signs in word classes, and let numeric and keyword prefixes participate in existing colon and glob fusion. These words now retain their complete spelling: ```sh echo 123.txt 1.2.3 .123.txt true:foo do:foo a+b 123+b 123@host echo ~/a:b ls 1.0* true* ``` A numeric filename can exceed the integer range because it is text. Complete scalar numbers still follow the existing numeric rules: incomplete floats and integer overflow remain errors, leading zeros remain text, and ordinary integers and floats keep their types. Glob fusion slices the original source, so numeric formatting cannot change which filenames match. Standalone keywords, booleans, plus-prefixed flags, collection delimiters, and quote-to-join checks retain their roles. Validation: numeric and keyword regression suites failed before the fix; the final tests check exact token spans, argument values, assignment and redirect values, actual glob matches with misleading numeric-prefix controls, tilde expansion, and control-flow/record behavior. Workspace tests, clippy across all targets with warnings denied, and rustdoc with warnings denied passed. Help fragments and generated syntax are synchronized. Reviewed through kaibo's default cast, GLM-5.2 with DeepSeek V4 Flash investigation. Review exposed the numeric prefix followed directly by punctuation; it is fixed with failing-then-passing regression tests rather than preserved as another word-character exception. This layer builds on #438 in the native stack rooted at #437. Tilde assignment and its regex-operator collision are a separate layer.
tobert
added a commit
that referenced
this pull request
Sep 9, 2026
An assignment followed immediately by a home-relative path was tokenized as a regex comparison operator. Separate the assignment delimiter from the tilde path outside tests, while preserving the regex operator inside tests and respecting nested command-substitution scopes.
Home-relative assignments now parse:
```sh
p=~/x
local p=~/x
p={x:empty}; p[x]=~/x
foo --path=~/x
```
Regex comparisons retain their operator, including after logical operators:
```sh
[[ a == a && x=~/x ]]
[[ $(p=~/x; echo $p) =~ /x ]]
```
The lexer uses its existing test and substitution frames to distinguish these contexts. A closing bracket qualifies only after a statement-head assignment target, so a glob character class cannot become an assignment delimiter. Logical operators inside a test now retain that test frame. Split and joined tokens use the original source spans; text adjacent to interpolation still requires quoting. Runtime tilde expansion keeps its existing behavior.
Validation: 12 regression cases failed before the fix; all 27 targeted cases pass, including token spans, plain/local/bracket assignments, named arguments, nested substitutions, regex operators with and without spaces, interpolation refusal, and a glob bracket that must not be treated as an assignment target. Workspace tests, clippy across all targets with warnings denied, rustdoc with warnings denied, kernel tests without default features, and the WASI build passed. Help fragments and generated syntax are synchronized.
Reviewed through kaibo's default cast, GLM-5.2 with DeepSeek V4 Flash investigation. The review also covers the preceding literal-word layer's final numeric punctuation cases. This layer builds on #440 in the native stack rooted at #437; no version bump.
tobert
added a commit
that referenced
this pull request
Sep 9, 2026
An assignment followed immediately by a home-relative path was tokenized as a regex comparison operator. Separate the assignment delimiter from the tilde path outside tests, while preserving the regex operator inside tests and respecting nested command-substitution scopes.
Home-relative assignments now parse:
```sh
p=~/x
local p=~/x
p={x:empty}; p[x]=~/x
foo --path=~/x
```
Regex comparisons retain their operator, including after logical operators:
```sh
[[ a == a && x=~/x ]]
[[ $(p=~/x; echo $p) =~ /x ]]
```
The lexer uses its existing test and substitution frames to distinguish these contexts. A closing bracket qualifies only after a statement-head assignment target, so a glob character class cannot become an assignment delimiter. Logical operators inside a test now retain that test frame. Split and joined tokens use the original source spans; text adjacent to interpolation still requires quoting. Runtime tilde expansion keeps its existing behavior.
Validation: 12 regression cases failed before the fix; all 27 targeted cases pass, including token spans, plain/local/bracket assignments, named arguments, nested substitutions, regex operators with and without spaces, interpolation refusal, and a glob bracket that must not be treated as an assignment target. Workspace tests, clippy across all targets with warnings denied, rustdoc with warnings denied, kernel tests without default features, and the WASI build passed. Help fragments and generated syntax are synchronized.
Reviewed through kaibo's default cast, GLM-5.2 with DeepSeek V4 Flash investigation. The review also covers the preceding literal-word layer's final numeric punctuation cases. This layer builds on #440 in the native stack rooted at #437; no version bump.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Literal paths keep one word
An agent repeatedly failed to read Git metadata because the lexer split a dot-prefixed path at its slash. The parser then refused the adjacent fragments with an interpolation-quoting hint. Recognize slash-containing relative words as literal paths, including numeric first components and bare current/parent directory references. Preserve their source spelling, accept the same @ and + characters in relative, absolute, tilde, and scoped-package paths, and let colon fusion keep Git revision paths whole.
The neighboring command-name check found a silent argument split: text attached to an executable name could become its first argument. Require a gap before the first argument and accept literal relative executable paths in command position. Command operators can still touch the name. Text joined with an expansion still requires quoting in argument position.
.git/hooks/pre-commit ../bin/check argument cat "$dir/HEAD"This is a correctness change intended for 0.17.2; versioning and release remain separate. It adds no word splitting or general token concatenation. Loop-list fragment adjacency, numeric filename/version classification, float-prefixed globs, keyword-prefixed colon words, and the tilde-assignment/operator collision are separate existing gaps recorded for follow-up.
Validation
Regression tests were run failing before the implementation. The suite checks token spans, exact argument counts and values, assignment and redirect targets, executable names, quote/expansion boundaries, collections, numbers, and globs. It also executes the original three-command Git-metadata read against fixture files.
cargo test --all --locked: 6,612 passed, 0 failed, 13 ignored.cargo clippy --all --all-targets --locked -- -D warnings: passed, zero warnings.for_newline_split_tests.rsremains recorded for cleanup.Review: kaibo, GLM-5.2 with DeepSeek V4 Flash. The initial static review found no blocking path-rule regressions; the diagnostic tests then caught message selection problems, which were corrected and reviewed in a follow-up. Both reviews were static; the test runs provide execution evidence.