Skip to content

feat: add shell parser - #2

Open
nzakas wants to merge 3 commits into
scaffoldingfrom
parser
Open

nzakas wants to merge 3 commits into
scaffoldingfrom
parser

Conversation

@nzakas

@nzakas nzakas commented Sep 18, 2026 •

Copy link
Copy Markdown
Member

Adds a shell parser that produces an ESTree-style syntax tree. It wraps mvdan-sh, a JavaScript build of mvdan/sh (the parser behind shfmt), and translates its output.

What's included

  • src/parser/parse.ts: parseShell(text, { variant, path }) returns { ast, comments }.
    • On invalid input it throws ShellSyntaxError, with 1-based line and column.
    • variant is "bash" (default), "posix", or "mksh". Unknown variants throw a TypeError.
  • src/parser/mvdan-sh.d.ts: minimal ambient types for the parts of mvdan-sh used here.
  • src/types.ts: the AST node types only. Rule and language types come in the next PR.
  • docs/syntax-tree.md: documents every node type.
  • src/index.ts: re-exports parseShell, ShellSyntaxError, and the AST types.
  • README: a "Syntax tree" section. It mentions sourceCode.getRange() and getLoc(), which the next PR adds.

Tree format

  • Positions: nodes carry only type, start, and end (0-based character offsets, end exclusive). There are no range or loc properties; the next PR's SourceCode derives them. mvdan-sh reports UTF-8 byte offsets, which are converted to JavaScript string offsets. Tests cover multi-byte and astral characters.
  • Operators: mvdan-sh exposes operators as numeric enums, so operator text (&&, >>, :-, ;;&, …) is read from the source between the relevant child nodes.
  • Shape:
    • Pipelines are flattened into one Pipeline node.
    • &&/|| chains are left-associative LogicalExpressions.
    • elif becomes a nested IfStatement, and else becomes an ElseClause.
    • Redirects, ! negation, and & background are properties of each statement.
  • Comments: collected in Program.comments. They are not traversed.

Known limitations

  • Deprecated dependency: mvdan-sh was deprecated upstream in April 2025 in favor of sh-syntax (Deprecate mvdan-sh npm package in favor of sh-syntax mvdan/sh#1145), and 0.10.1 is its last release. It works for everything tested here. sh-syntax is not a drop-in replacement for this plugin: its parse output stops at the statement level (each command is only positions, with no type, arguments, expansions, or nested statements), and its API is async while ESLint parsing must be synchronous.
  • Heredocs: a heredoc body appears after the statement that declares it, so a Redirect's heredoc word can lie outside its parent statement's range.
  • [ ... ]: a plain Command named [. Only [[ ... ]] is parsed as a TestCommand.

Testing

63 parser tests cover every node type, all ${...} operator forms, redirects and heredocs, comments, Unicode offsets, error locations, and the posix/mksh variants. This PR also adds an entry-point test for the parser export. 82 tests total, including the existing metadata test; build, lint, and format checks pass.

🤖 Generated with Claude Code


Stack created with GitHub Stacks CLI • Give Feedback 💬

Summary by CodeRabbit

  • New Features
    • Added a Bash parser that returns an ESTree-style syntax tree, with support for Bash, POSIX, and mksh syntax.
    • Added public TypeScript types and exports for parsing and syntax errors. Parser errors include line and column details, and can include the source path.
  • Documentation
    • Added guidance on syntax tree node shapes, source positions, traversal, and heredoc ranges, plus examples for using the parser and reading node locations.

@coderabbitai

coderabbitai Bot commented Sep 18, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 55 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Repository: eslint/coderabbit/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 06fe508a-0260-48be-ad82-827f10756f32
📥 Commits

Reviewing files that changed from the base of the PR and between db341d1 and 1d35e65.

📒 Files selected for processing (8)
  • README.md
  • docs/syntax-tree.md
  • src/index.spec.ts
  • src/index.ts
  • src/parser/mvdan-sh.d.ts
  • src/parser/parse.spec.ts
  • src/parser/parse.ts
  • src/types.ts
ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Free

Run ID: a14a20ef-c73a-4728-b1fc-c7fb533eb874

📥 Commits

Reviewing files that changed from the base of the PR and between dd79cac and db341d1.

📒 Files selected for processing (8)
  • README.md
  • docs/syntax-tree.md
  • src/index.spec.ts
  • src/index.ts
  • src/parser/mvdan-sh.d.ts
  • src/parser/parse.spec.ts
  • src/parser/parse.ts
  • src/types.ts

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

The change adds a parser for Bash, POSIX, and mksh syntax. It translates parsed input into a typed ESTree-style AST, exports the parser and types, adds tests, and documents node shapes and source-position behavior.

Changes

Bash parser

Layer / File(s) Summary
AST contracts and public API
src/types.ts, src/parser/mvdan-sh.d.ts, src/parser/parse.ts, src/index.ts
Adds AST node types, shell variant and parser interfaces, mvdan-sh declarations, and public exports for parseBash, BashSyntaxError, and the types.
Parsing, translation, and validation
src/parser/parse.ts, src/parser/parse.spec.ts, src/index.spec.ts
Adds source-position conversion and translation for Bash-family syntax, including statements, words, redirects, and expressions. Tests cover AST shapes, heredoc ranges, Unicode offsets, syntax errors, and parser variants.
Syntax-tree documentation
docs/syntax-tree.md, README.md
Documents AST node shapes, position conventions, traversal, heredoc range behavior, and a public parser usage example.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Caller
  participant parseBash
  participant mvdanSh as mvdan-sh parser
  participant Translator as AST translator
  Caller->>parseBash: Bash source and parse options
  parseBash->>mvdanSh: Parse source with selected shell variant
  mvdanSh-->>parseBash: Syntax tree and comments
  parseBash->>Translator: Translate syntax tree and source positions
  Translator-->>parseBash: AST and comments
  parseBash-->>Caller: AST and comments
Loading

Architecture Summary

Architecture risk: 🔵 Low · up to db341

The change affects 3 systems.

Changed systems: src, docs, README.md

Architecture concerns
No architecture-level concerns identified.

Review details

Systems and components

  • observed — src (service) was modified; 6 changed files map to changed impact.
  • observed — docs (service) was modified; 1 changed file maps to changed impact.
  • observed — README.md (service) was modified; 1 changed file maps to changed impact.

Before / after behavior

  • observed — Modified behavior in README.md: Added documentation for the syntax-tree format and node location accessors, plus an example importing and calling the public parseBash export.
  • observed — Modified behavior in docs/syntax-tree.md: Adds documentation for the parser and node positions, including 0-based inclusive/exclusive character offsets, SourceCode position helpers, and the fact that nodes omit range and loc. Describes how heredoc bodies can lie outside redirect or enclosing-node ranges, and the resulting limitations for offset-based lookup, statement text, sibling ordering, and fixes.
  • observed — Modified behavior in docs/syntax-tree.md: Documents statement-node shapes and fields for programs, commands, pipelines, logical expressions, compound statements, conditionals, loops, cases, functions, tests, arithmetic, declarations, timing, and coprocesses.
  • observed — Modified behavior in docs/syntax-tree.md: Documents Word composition and the shapes of literal, quoted-string, parameter-expansion, substitution, arithmetic-expansion, and extended-glob nodes, including their associated fields.
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@nzakas
nzakas added this pull request to stack #16 September 18, 2026 15:00
@nzakas
nzakas marked this pull request as ready for review September 18, 2026 15:01
@nzakas
nzakas force-pushed the parser branch 4 times, most recently from 1155c3a to a100dd6 Compare September 22, 2026 21:21
@nzakas
nzakas requested a balanced review from Copilot September 28, 2026 18:24

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Operator extraction mishandles comments, error paths are omitted, and parts of the public documentation are inaccurate.

Review effort: Balanced
Findings: 1 High severity · 1 Medium severity · 3 Low severity

Open (5)
What changed in this PR

Adds a public Bash parser that translates mvdan-sh output into an ESTree-style AST.

Changes:

  • Adds AST types, parser translation, errors, variants, and Unicode offsets.
  • Adds comprehensive parser and entry-point tests.
  • Documents the syntax tree and direct parser API.
File Description
src/​parser/​parse.ts Implements parsing and AST translation.
src/​parser/​parse.spec.ts Tests parser behavior and node types.
src/​parser/​mvdan-sh.d.ts Declares the required dependency API.
src/​types.ts Defines public AST types.
src/​index.ts Exports the parser and types.
src/​index.spec.ts Tests public parser exports.
README.md Documents direct parser usage.
docs/​syntax-tree.md Documents AST structure and semantics.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/parser/parse.ts
Comment on lines +164 to +170
#operatorBetween(left: MvdanNode, right: MvdanNode): string {
const slice = this.#text
.slice(this.#end(left), this.#start(right))
.trim();

return slice.split(/\s+/u)[0] as string;
}

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partly right. The arithmetic example isn't valid Bash (# doesn't start a comment inside (( )), and the parser rejects it), but the same helper is used for [[ ]], where [[ a # note && b ]] is valid and did yield "#" as the operator. Fixed in db341d1 by skipping leading comments as well as whitespace, with a regression test covering a comment on either side of the operator.

Comment thread src/parser/parse.ts Outdated
Comment on lines +1008 to +1012
throw new BashSyntaxError(
parseError.Text || parseError.Error(),
line,
column,
);

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed that path had no observable effect. I didn't switch to Error() though: that string is prefixed with path:line:col:, which duplicates the location ESLint already prints from line/column. Instead db341d1 exposes the option as BashSyntaxError#path, updates the option's doc comment to say so, and adds a test.

Comment thread docs/syntax-tree.md Outdated
Comment thread docs/syntax-tree.md
Comment on lines +508 to +523
## Traversal

The traversal order for each node type is defined by the exported
`visitorKeys` object. Rules receive nodes by type name and may use `:exit`
suffixes:

```js
export default {
create(context) {
return {
Command(node) {},
"Program:exit"(node) {},
};
},
};
```

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leaving this as is. This PR is the bottom of a stack that lands in order, and the visitorKeys export arrives in the very next PR (#3). The doc describes the tree format as a whole, so removing the section here only to re-add it one PR later would just be churn.

Comment thread src/types.ts Outdated
@nzakas nzakas changed the title feat: add Bash parser feat: add Shell parser Oct 2, 2026
@nzakas nzakas changed the title feat: add Shell parser feat: add shell parser Oct 2, 2026
@nzakas
nzakas force-pushed the parser branch 2 times, most recently from bbaef73 to 67b3785 Compare October 6, 2026 19:26
nzakas and others added 3 commits October 8, 2026 16:49
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Skip comments on either side of a test operator, expose the path option
as BashSyntaxError#path, and document the mksh ;| case terminator.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Renames parseBash, BashSyntaxError, BashNode, and the related types to
their Shell equivalents (BashShellVariant becomes ShellVariant), and
updates the docs to refer to @eslint/shell.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@fasttime fasttime left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants