docs/gen3 draft for wrapped commands/tools by robch · Pull Request #41 · robch/cycod

robch · 2025-08-22T00:12:45Z

No description provided.

robch · 2025-08-22T00:28:17Z

Comprehensive Review of CYCOD Custom Tools Specification

The specification presents a well-designed framework for extending CYCOD with user-defined shell-based tools. Here's a summary of the key strengths and areas for enhancement from multiple reviews:

Key Strengths

Strong alignment with CYCOD principles (scope system, command patterns, security model)
Powerful functional capabilities (multi-step architecture, output referencing, conditional execution)
Well-structured documentation

Key Enhancement Areas

Parameter handling (validation, transformation, complex types)
Command execution (security, platform-specific behavior, environment variables)
Error handling and recovery
Tool composition and reusability
Documentation and usability enhancements

I'll add specific comments to relevant sections. Please see the uber-critique document for full details.

robch · 2025-08-22T00:29:08Z

docs/gen3/custom-tools-spec-draft-1.md

+  - name: step2
+    bash: command {PARAM}
+
+# Parameters


Parameter Handling Enhancement Opportunities

Type Processing and Validation

Consider adding validation rules for parameters (min/max for numbers, patterns for strings)

Add parameter transformation capabilities before substitution

Enhanced Documentation

Parameter descriptions could include examples and detailed help

Complex Parameter Types

Consider supporting object/nested parameters for advanced use cases

See the uber-critique document for specific YAML examples of these enhancements.

robch

Starting a detailed review of the Custom Tools Specification

robch · 2025-08-22T00:29:24Z

Parameter Handling Enhancement Opportunities (Lines 54-75)

Type Processing and Validation
- Consider adding validation rules for parameters (min/max for numbers, patterns for strings)
- Add parameter transformation capabilities before substitution
Enhanced Documentation
- Parameter descriptions could include examples and detailed help
Complex Parameter Types
- Consider supporting object/nested parameters for advanced use cases

Example enhancement:

parameters:
  COUNT:
    type: number
    description: Count of items
    validation:
      minimum: 1
      maximum: 100
    transform: "Math.floor"  # Transform function to apply
    format: "--count={value}"  # How parameter appears in command

robch · 2025-08-22T00:29:37Z

Command Execution Enhancement Opportunities (Lines 76-77)

Security Considerations
- The specification should address parameter escaping to prevent command injection
Platform-Specific Behavior
- Consider supporting platform-specific command variations beyond just compatibility tags
Environment Variables
- Add support for setting and managing environment variables for tool execution

Example enhancement:

commands:
  default: find {DIRECTORY} -name "{PATTERN}"
  platforms:
    windows: powershell -Command "Get-ChildItem -Path '{DIRECTORY}' -Filter '{PATTERN}' -Recurse"
    unix: find {DIRECTORY} -name "{PATTERN}"

environment:
  variables:
    HTTP_PROXY: "{PROXY_URL}"
    DEBUG: "1"
  inherit: true  # Inherit parent process environment

robch · 2025-08-22T00:29:50Z

Error Handling Enhancement Opportunities (Lines 138-153)

Advanced Error Recovery
- Consider more sophisticated recovery options (retries, fallbacks)
Output Management
- Add specifications for handling large outputs or streaming capabilities

Example enhancement:

steps:
  - name: risky-step
    bash: command
    error-handling:
      retry:
        attempts: 3
        delay: 1000  # milliseconds
      fallback: alternative-command {PARAM}

  - name: generate-large-output
    bash: large-output-command
    output:
      max-size: 10MB  # Limit output capture size
      truncation: true  # Whether to truncate if exceeded
      streaming: true  # Stream output rather than buffering

robch · 2025-08-22T00:30:03Z

Tool Composition and Reusability (Suggested New Section)

The specification currently lacks explicit support for:

Tool Composition
- Allow tools to reference or use other tools
Tool Discovery
- Consider more sophisticated discovery mechanisms (categories, tags, keywords)
Tool Versioning
- Add mechanism for versioning and compatibility tracking

Example enhancement:

steps:
  - name: use-git-clone
    use-tool: git-clone
    with:
      URL: "{REPO_URL}"
      DIRECTORY: "{WORKSPACE}"

metadata:
  category: file-management
  subcategory: search
  tags: [files, search, find]
  keywords: [locate, search, find, pattern]

version: 1.0.0
min-cycod-version: "1.2.0"
changelog:
  - version: 1.0.0
    changes: "Initial release"

robch · 2025-08-22T00:30:17Z

Implementation Considerations (Suggested New Section)

Consider adding sections on:

Tool Testing
- Built-in ways to test tools before using them
Resource Management
- Specifications for resource constraints and cleanup
Interactive Mode
- Support for tools that require interactive input

Example enhancement:

tests:
  - name: basic-test
    parameters:
      PATTERN: "*.txt"
      DIRECTORY: "test-data"
    expected:
      exit-code: 0
      output-contains: "test-data/sample.txt"

resources:
  timeout: 60000  # milliseconds
  max-memory: 512MB
  cleanup:
    - delete-temp-files: true
    - final-command: "docker rm {CONTAINER_ID}"

interactive: true  # Tool may prompt for user input
interactive-options:
  timeout: 30000  # How long to wait for user input
  default-response: "y"  # Default if no input provided

robch · 2025-08-22T00:30:30Z

Documentation and Usability Enhancements

Best Practices
- Consider adding a dedicated section on creating effective tools
- Include tool design principles, parameter naming conventions, error handling patterns
Advanced Examples
- Add more diverse examples to illustrate advanced features:
  - Complex error handling
  - Sophisticated parameter validation
  - Cross-platform compatibility
  - Integration with other systems

These improvements would help users create more robust and effective custom tools.

robch · 2025-08-22T00:31:35Z

docs/gen3/custom-tools-spec-draft-1.md

+
+Parameter types include:
+- `string`: Text values
+- `number`: Numeric values


Consider adding support for parameter validation rules. For example:

parameters: COUNT: type: number description: Count of items validation: minimum: 1 maximum: 100

robch

Here are some line-specific suggestions to enhance the specification:

robch · 2025-08-22T00:32:17Z

Line-Specific Enhancement Suggestions

Based on our review, here are specific line-based suggestions that could be made through the GitHub web interface:

Parameter Types (lines 121-126):
Error Handling (lines 140-144):
Platform Support (lines 132-134):
Step Output References (lines 264-279):

These specific enhancements could be added as line comments in the GitHub web interface.

robch · 2025-08-22T00:32:27Z

Line-Specific Enhancement Suggestions

Based on our review, here are specific line-based suggestions that could be made through the GitHub web interface:

Parameter Types (lines 121-126):
Error Handling (lines 140-144):
Platform Support (lines 132-134):
Step Output References (lines 264-279):

These specific enhancements could be added as line comments in the GitHub web interface.

robch · 2025-08-22T00:32:33Z

Line-Specific Enhancement Suggestions

Based on our review, here are specific line-based suggestions that would be ideal to add directly to specific parts of the code through the GitHub web interface:

Parameter Types (lines 121-126): Add validation rules for each parameter type
Error Handling (lines 140-144): Add retry mechanisms and fallback options
Platform Support (lines 132-134): Add platform-specific command variations
Step Output References (lines 264-279): Add output management options

See the uber-critique document for detailed YAML examples of these enhancements.

robch

Here are some line-specific suggestions for enhancing the Custom Tools Specification

robch · 2025-08-22T00:37:18Z

docs/gen3/custom-tools-spec-draft-1.md

+- `number`: Numeric values
+- `boolean`: True/false values
+- `array`: List of values
+


Parameter Types Enhancement: Consider adding validation rules for each parameter type. For example:

parameter-types: string: validation: [pattern, min-length, max-length] number: validation: [minimum, maximum, multiple-of] boolean: validation: [allowed-values] array: validation: [min-items, max-items, unique-items]

robch · 2025-08-22T00:37:18Z

docs/gen3/custom-tools-spec-draft-1.md

+
+```yaml
+ignore-errors: false   # Whether to continue even if the tool fails
+```


Error Handling Enhancement: Consider adding more sophisticated recovery options like retries and fallbacks:

error-handling: retry: attempts: 3 delay: 1000 # milliseconds fallback: alternative-command {PARAM} report: true # Whether to include detailed error reports

robch · 2025-08-22T00:37:18Z

docs/gen3/custom-tools-spec-draft-1.md

+```yaml
+platforms: [windows, linux, macos]  # Only include platforms the tool works on
+tags: [category1, category2, read]  # Tags for categorization and security
+```


Platform Support Enhancement: Consider supporting platform-specific command variations beyond just compatibility tags:

platforms: windows: command: powershell -Command "Get-ChildItem -Path '{DIRECTORY}'" linux: command: find {DIRECTORY} -type f macos: command: find {DIRECTORY} -type f

robch · 2025-08-22T00:37:19Z

docs/gen3/custom-tools-spec-draft-1.md

+- `{step1.output}` - Standard output from the step
+- `{step1.error}` - Standard error from the step
+- `{step1.exit-code}` - Exit code from the step
+


Output Management Enhancement: Add specifications for handling large outputs or streaming capabilities:

output-options: format: json|text|raw # Output format max-size: 10MB # Maximum capture size streaming: true # Whether to stream output transform: jq '.items' # Transform before passing to next step

robch

Additional line-specific suggestions based on the comprehensive review

robch · 2025-08-22T00:41:08Z

docs/gen3/custom-tools-spec-draft-1.md

+    description: Another parameter
+    required: false
+    default: 5
+


Enhanced Parameter Documentation: Consider adding examples and more detailed help for parameters:

parameters: PATTERN: type: string description: File pattern to match examples: ["*.txt", "log*.log"] detailed-help: | A glob pattern that follows standard shell conventions. Use * as a wildcard for any characters. Examples: *.txt, log-*.log, data-????.csv

robch · 2025-08-22T00:41:08Z

docs/gen3/custom-tools-spec-draft-1.md

+- `number`: Numeric values
+- `boolean`: True/false values
+- `array`: List of values
+


Complex Parameter Types: Consider supporting object/nested parameters for advanced use cases:

parameters: CONFIG: type: object properties: server: type: string port: type: number

robch · 2025-08-22T00:41:08Z

docs/gen3/custom-tools-spec-draft-1.md

+- `write`: Tools that modify files or data
+- `run`: Tools that execute other commands or scripts
+
+If no security tag is provided, the tool is considered high-risk and requires explicit approval.


Security Considerations: Consider addressing parameter escaping to prevent command injection:

parameters: QUERY: type: string description: Search query security: escape-shell: true # Automatically escape shell metacharacters

robch · 2025-08-22T00:41:08Z

docs/gen3/custom-tools-spec-draft-1.md

+    run-condition: "{step2.exit-code} == 0"  # Condition for when to run this step
+
+  - name: step2
+    bash: command {PARAM}


Tool Composition: Consider adding support for tools to reference other tools:

steps: - name: use-git-clone use-tool: git-clone with: URL: "{REPO_URL}" DIRECTORY: "{WORKSPACE}"

robch · 2025-08-22T00:41:08Z

docs/gen3/custom-tools-spec-draft-1.md

+working-directory: ~/path          # Working directory
+platforms: [windows, linux, macos] # Supported platforms
+tags: [tag1, tag2, read]           # Categories and security tags
+ignore-errors: false               # Whether to continue if a step fails


Tool Discovery and Versioning: Consider adding metadata for better tool discovery and version tracking:

metadata: category: file-management subcategory: search tags: [files, search, find] keywords: [locate, search, find, pattern] version: 1.0.0 min-cycod-version: "1.2.0" changelog: - version: 1.0.0 changes: "Initial release"

robch · 2025-08-22T00:41:08Z

docs/gen3/custom-tools-spec-draft-1.md

+  cycod help tools
+```
+
+## Examples


Tool Testing: Consider adding built-in ways to test tools:

tests: - name: basic-test parameters: PATTERN: "*.txt" DIRECTORY: "test-data" expected: exit-code: 0 output-contains: "test-data/sample.txt"

robch · 2025-08-22T00:41:08Z

docs/gen3/custom-tools-spec-draft-1.md

+  cycod help tools
+```
+
+## Examples


Resource Management: Consider adding specifications for resource constraints and cleanup:

resources: timeout: 60000 # milliseconds max-memory: 512MB cleanup: - delete-temp-files: true - final-command: "docker rm {CONTAINER_ID}"

robch · 2025-08-22T00:41:08Z

docs/gen3/custom-tools-spec-draft-1.md

+  cycod help tools
+```
+
+## Examples


Interactive Mode: Consider adding support for tools that require interactive input:

interactive: true # Tool may prompt for user input interactive-options: timeout: 30000 # How long to wait for user input default-response: "y" # Default if no input provided

robch

Final line-specific suggestion about environment variables

robch · 2025-08-22T00:42:20Z

docs/gen3/custom-tools-spec-draft-1.md

+
+# Optional settings
+timeout: 60000                     # Timeout in milliseconds
+working-directory: ~/path          # Working directory


Environment Variables: Consider adding support for setting and managing environment variables for tool execution:

environment: variables: HTTP_PROXY: "{PROXY_URL}" DEBUG: "1" inherit: true # Inherit parent process environment

This commit introduces a new documentation file containing examples of custom tools for GitHub PR reviews. The tools include: 1. **PR Summary Comment Tool**: Adds a top-level comment to a PR. 2. **PR Section Comment Tool**: Adds comments referencing specific sections of code. 3. **PR Line Comment Tool**: Adds line-specific comments to a PR. 4. **Advanced PR Multi-Comment Tool**: Allows adding multiple types of comments in a single command. Each tool is accompanied by detailed YAML configurations and usage examples, enhancing the review process with structured commenting capabilities.

docs/gen3 draft for wrapped commands/tools

c004ebb

robch commented Aug 22, 2025

View reviewed changes

robch added 3 commits August 21, 2025 17:48

Update custom tools specification with enhanced schema and examples

10077f1

first working draft of custom tools

9f90392

Conversation

robch commented Aug 22, 2025

Uh oh!

robch commented Aug 22, 2025

Comprehensive Review of CYCOD Custom Tools Specification

Key Strengths

Key Enhancement Areas

Uh oh!

Choose a reason for hiding this comment

Parameter Handling Enhancement Opportunities

Uh oh!

robch left a comment

Choose a reason for hiding this comment

Uh oh!

robch commented Aug 22, 2025

Parameter Handling Enhancement Opportunities (Lines 54-75)

Uh oh!

robch commented Aug 22, 2025

Command Execution Enhancement Opportunities (Lines 76-77)

Uh oh!

robch commented Aug 22, 2025

Error Handling Enhancement Opportunities (Lines 138-153)

Uh oh!

robch commented Aug 22, 2025

Tool Composition and Reusability (Suggested New Section)

Uh oh!

robch commented Aug 22, 2025

Implementation Considerations (Suggested New Section)

Uh oh!

robch commented Aug 22, 2025

Documentation and Usability Enhancements

Uh oh!

Choose a reason for hiding this comment

Uh oh!

robch left a comment

Choose a reason for hiding this comment

Uh oh!

robch commented Aug 22, 2025

Line-Specific Enhancement Suggestions

Uh oh!

robch commented Aug 22, 2025

Line-Specific Enhancement Suggestions

Uh oh!

robch commented Aug 22, 2025

Line-Specific Enhancement Suggestions

Uh oh!

robch left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

robch left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

robch left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment