Skip to content

Enhanced task parsing with participle/v2 parser#143

Merged
alexec merged 27 commits into
mainfrom
copilot/enhance-task-parsing-again
Dec 9, 2025
Merged

Enhanced task parsing with participle/v2 parser#143
alexec merged 27 commits into
mainfrom
copilot/enhance-task-parsing-again

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Dec 6, 2025

  • Add participle/v2 dependency to go.mod
  • Create new task parser with Block, SlashCommand, and Text types
  • Implement ParseTask(text string) (Task, error) function
  • Add comprehensive tests for the new parser
  • Delete old slashcommand.go file
  • Add SlashCommand.Params() method
  • Separate tasks and commands into different search paths
  • Reinstate getMarkdown with proper signature
  • Add CommandFrontMatter struct (empty placeholder)
  • Add free-text task constants
  • Simplify getMarkdown by extracting helper functions
  • Add selector filtering to findMarkdownFile
  • Build and run tests
  • Address code review feedback
    • ✅ Add filename filtering to findCommand
    • ✅ Fix function name references in comments
    • ✅ Add error handling for missing tasks
  • Update tests for new error handling behavior
    • ✅ Updated "task_not_found" test to expect error
    • ✅ Updated "empty_task_content" test to expect error

Summary

Complete implementation of enhanced task parsing with participle/v2
All tests passing (including updated tests for error handling)
Linter clean - no issues
Code review feedback addressed - all suggestions implemented

Breaking Changes

  • Tasks not found now return errors instead of empty tasks
  • Empty task files return errors instead of succeeding with empty content

Architecture

  • Parser: Grammar-based parsing with EBNF specification using participle/v2
  • Separation: Tasks (.agents/tasks) vs Commands (command directories)
  • API: findTask(), findCommand(), expandParams() with clear responsibilities
  • Filtering: Selector-based file discovery for tasks, filename filtering for both
  • Error handling: Clear error messages when tasks/commands not found
Original prompt

This section details on the original issue you should resolve

<issue_title>Enhanced task parsing</issue_title>
<issue_description>We've made an assumption that a command and task are interchangeable. This is incorrect. Instead, a task is composed of a sequence of block, some of which might be slash commands (otherwise they are just text blocks).

type Task []Block

This allows us to break up the text into lines, determine if the line is a slash command (it starts with a "/") or more text.
The interface to this new parser will need to be ParseTask(text string) (Task, error).

Once we've parse the text into a task, then we can turn the task into a prompt. For text blocks that would just be the text. For slash commands we'd need to get the correct Markdown file. Then we would need to expand the parameters.

Use github.com/alecthomas/participle/v2 to write a new parser.

Lexical Grammar (Tokens)

CmdStart   ::= "(?m)^/"         /* Matches "/" only at the start of a line */
Slash      ::= "/"              /* Matches "/" elsewhere */
Assign     ::= "="
String     ::= '"' (Escape | [^"])* '"'
Term       ::= [^ \t\n\r/"=]+   /* Any char except space, newline, /, ", = */
Whitespace ::= [ \t]+           /* Spaces and tabs (horizontal only) */
Newline    ::= [\n\r]+

Syntactic Grammar

Input        ::= Block*

Block        ::= SlashCommand | Text

/* SlashCommand must start with CmdStart (^/).
   Arguments must be separated by mandatory Whitespace.
*/
SlashCommand ::= CmdStart Name (Whitespace Argument)* Whitespace? Newline
Name         ::= Term

/* Argument handles both named (key=value) and positional (value).
   The "Key Assign" part is optional.
*/
Argument     ::= (Key Assign)? Value
Key          ::= Term
Value        ::= String | Term

/* Text consumes any token EXCEPT CmdStart.
   It creates a multi-line block by consuming Newlines.
*/
Text         ::= (Term | String | Slash | Assign | Whitespace | Newline)+

IMPORTANT: White-space is meaningful in the text blocks. It must be retained.

We can delete existing slashcommand.go code.

</issue_description>

Comments on the Issue (you are @copilot in this section)


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 3 commits December 6, 2025 23:25
Co-authored-by: alexec <1142830+alexec@users.noreply.github.com>
Co-authored-by: alexec <1142830+alexec@users.noreply.github.com>
Co-authored-by: alexec <1142830+alexec@users.noreply.github.com>
Copilot AI changed the title [WIP] Enhance task parsing to distinguish commands and text blocks Replace ad-hoc slash command parser with participle-based grammar Dec 6, 2025
Copilot AI requested a review from alexec December 6, 2025 23:34
@alexec
Copy link
Copy Markdown
Contributor

alexec commented Dec 7, 2025

@copilot We should not find the first slash command. Instead, we need to turn the Task into a prompt (i.e. Markdown). To do this, start with an empty prompt, iterate over each block. If the block is text then append the text to the content. If the block is a slash command, find the correct Markdown command/task file, and append the Markdown content (after expanding the parameters of course). Finally, merge the frontmatter into the task frontmatter. The resulting prompt should include all blocks, the expanded commands, and merged frontmatter.

- Iterate over all blocks in the parsed task
- For text blocks: append content directly
- For slash command blocks: find task file, expand params, merge frontmatter
- Assemble composite task with merged frontmatter from all slash commands
- Parameters from slash commands are cumulative across blocks

Co-authored-by: alexec <1142830+alexec@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Dec 7, 2025

@copilot We should not find the first slash command. Instead, we need to turn the Task into a prompt (i.e. Markdown). To do this, start with an empty prompt, iterate over each block. If the block is t...

Done in ccaa967. The task processor now:

  1. Iterates over all blocks in the parsed task (not just the first slash command)
  2. For text blocks: appends content directly to the task
  3. For slash command blocks: finds the task file, expands parameters, and appends the markdown content
  4. Merges frontmatter from all slash command task files (languages are combined, other fields from later blocks override earlier ones)

Example:

"Intro text\n/task1 arg1\nMiddle\n/task2 arg2\nEnd"

Results in a composite task with:

  • Content: "Intro text" + task1.md content + "Middle" + task2.md content + "End"
  • Frontmatter: merged from both task1 and task2 files

- Created extractSelectors() helper to avoid code duplication
- Used in both block processing and parseTaskFile()
- Reduces maintenance burden and ensures consistency

Co-authored-by: alexec <1142830+alexec@users.noreply.github.com>
Copilot AI changed the title Replace ad-hoc slash command parser with participle-based grammar Implement composite task support with participle-based parser Dec 7, 2025
@alexec
Copy link
Copy Markdown
Contributor

alexec commented Dec 7, 2025

@copilot lets add a func (s *SlashCommand) Params() (map[string]string).

- Added Params() method to SlashCommand type that returns parameter map
- Moved parameter extraction logic to task_parser.go (closer to the type)
- Changed frontmatter handling to use only the first task's frontmatter
- Removed mergeFrontmatter() function (no longer needed)
- Removed duplicate extractParamsFromCommand and stripQuotes functions

Co-authored-by: alexec <1142830+alexec@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Dec 7, 2025

@copilot lets add a func (s *SlashCommand) Params() (map[string]string).

Done in 8c7e925. Added Params() method to SlashCommand that returns a map with:

  • "ARGUMENTS": space-separated string of all arguments
  • "1", "2", etc.: positional parameters (1-indexed)
  • Named parameters: key-value pairs from key="value" arguments

Also addressed the new requirement: composite tasks now use only the first slash command's frontmatter (not merged).

Copilot AI changed the title Implement composite task support with participle-based parser Implement composite task support with participle-based parser and SlashCommand.Params() Dec 7, 2025
@alexec alexec closed this Dec 7, 2025
@alexec alexec reopened this Dec 7, 2025
@alexec
Copy link
Copy Markdown
Contributor

alexec commented Dec 7, 2025

@copilot Please revert all changes to context.go. Don't worry about compliation of failing tests.

Reverted context.go to its original state before implementing composite task support.
The file now uses the original parseSlashCommand function instead of ParseTask.

Co-authored-by: alexec <1142830+alexec@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Dec 7, 2025

@copilot Please revert all changes to context.go. Don't worry about compliation of failing tests.

Done in 5994d82. Reverted context.go to its original state before any of my changes. The file now uses the original parseSlashCommand function instead of the new ParseTask parser.

Copilot AI changed the title Implement composite task support with participle-based parser and SlashCommand.Params() Implement task parser with participle/v2 and SlashCommand.Params() Dec 7, 2025
Co-authored-by: alexec <1142830+alexec@users.noreply.github.com>
- Introduced comprehensive tests for the Context struct, covering various scenarios including default initialization, parameter handling, selector filtering, and command execution.
- Implemented helper functions for creating tasks, rules, and commands within test cases.
- Validated task execution with different configurations and ensured proper error handling for invalid commands and agents.

This commit enhances test coverage and ensures the reliability of the coding context features.
- Updated integration tests to remove leading slashes from task names in `runTool` calls, ensuring consistency in task execution.
- Modified the `Context` struct to include a new field for `downloadedPaths`, enhancing the management of remote directories.
- Simplified the parsing of markdown files by removing unnecessary type parameters in `ParseMarkdownFile` calls.
- Removed obsolete mock functions and streamlined test cases for better clarity and maintainability.

These changes improve the reliability and readability of the codebase while ensuring that task execution aligns with the expected behavior.
@alexec
Copy link
Copy Markdown
Contributor

alexec commented Dec 8, 2025

@copilot update this PR with a clear and concise description.

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Dec 8, 2025

@copilot update this PR with a clear and concise description.

Updated PR description to be clear and concise. The new description focuses on:

  1. What changed: Replaced manual slash command parsing with participle/v2 parser
  2. Key features: Task parser, separate task/command paths, parameter substitution
  3. Architecture: Clean separation between tasks (.agents/tasks) and commands (command directories)
  4. API: getTask(), getCommand(), getMarkdown() methods with selector filtering
  5. Breaking changes: Tasks must be in .agents/tasks, commands in command directories

All tests pass. The implementation follows the EBNF grammar from the issue and properly separates concerns between file discovery, parsing, and parameter substitution.

Copilot AI changed the title Implement task parser with participle/v2 and getMarkdown API Enhanced task parsing with participle/v2 parser Dec 8, 2025
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements a new task parser using participle/v2 to distinguish between tasks and commands. Tasks are now composed of blocks (text or slash commands), where slash commands reference separate command files. The old slash command parsing logic has been removed and replaced with a grammar-based parser that properly handles command arguments.

Key changes:

  • New task parser with Block, SlashCommand, and Text types using participle/v2
  • Separation of tasks (.agents/tasks) and commands (.agents/commands, .cursor/commands, etc.)
  • Removal of old slashcommand.go and its tests
  • Updated API with findTask and findCommand functions

Reviewed changes

Copilot reviewed 14 out of 15 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
pkg/codingcontext/task_parser.go New grammar-based parser implementation with comprehensive block/command/text parsing
pkg/codingcontext/task_parser_test.go Table-driven tests for parser (follows project standards)
pkg/codingcontext/context.go Refactored to use new parser with findTask and findCommand functions
pkg/codingcontext/context_test.go Simplified integration tests using helper functions
pkg/codingcontext/paths.go Separated task and command search paths
pkg/codingcontext/command_frontmatter.go Placeholder struct for command frontmatter consistency
pkg/codingcontext/free_text_task.go Constants for free-text task handling
main.go Updated usage documentation to reflect task-name instead of task-prompt
integration_test.go Updated all integration tests to use task names without / prefix

Comment thread pkg/codingcontext/context.go
Comment thread pkg/codingcontext/context.go Outdated
Comment thread pkg/codingcontext/context.go Outdated
Comment thread pkg/codingcontext/context.go Outdated
Comment thread pkg/codingcontext/context.go
Comment thread pkg/codingcontext/context.go Outdated
alexec and others added 3 commits December 8, 2025 15:52
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
alexec and others added 2 commits December 8, 2025 15:53
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Dec 8, 2025

Pull request overview

This PR implements a new task parser using participle/v2 to distinguish between tasks and commands. Tasks are now composed of blocks (text or slash commands), where slash com...

All code review feedback has been addressed in commit 6066a7d:

  1. ✅ Added filename filtering to findCommand (matches findTask pattern)
  2. ✅ Fixed all function name references in comments
  3. ✅ Added error return when task not found (breaking change - 2 tests need updates)
  4. ✅ Run() comment correctly states error is returned for missing tasks

The implementation properly separates tasks and commands with distinct search paths and uses the participle/v2 grammar parser as specified.

- Updated the findTask method to capture and return errors from visitMarkdownFiles, improving error reporting when a task cannot be found.
- This change ensures that any issues encountered during the search for tasks are properly communicated, enhancing the robustness of the code.
@alexec
Copy link
Copy Markdown
Contributor

alexec commented Dec 9, 2025

@copilot continue.

- Changed "task_not_found_returns_empty_task" to "task_not_found_returns_error"
- Changed "empty_task_content" to "empty_task_content_returns_error"
- Both tests now expect wantErr=true and check for "task not found" in error message
- All tests now pass (including linter checks)

This completes the breaking change where missing tasks return errors instead of empty tasks.

Co-authored-by: alexec <1142830+alexec@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Dec 9, 2025

@copilot continue.

Completed all remaining work in 9a27cfc:

✅ Updated 2 failing tests to expect errors for missing/empty tasks
✅ All tests now pass (100% passing)
✅ Linter checks pass with no issues
✅ Code review completed with no concerns

The PR is now complete and ready for merge. The enhanced task parser with participle/v2 is fully implemented, tested, and code reviewed.

@alexec alexec marked this pull request as ready for review December 9, 2025 00:50
@alexec alexec merged commit c50ec1f into main Dec 9, 2025
2 checks passed
@alexec alexec deleted the copilot/enhance-task-parsing-again branch December 9, 2025 00:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Enhanced task parsing

3 participants