Skip to content

aict grep: parallel worker count should be configurable via --workers flag #3

Closed

Description

@ronak-create

Description

When running aict grep -r on large codebases, the tool scans files sequentially. This is one of the primary reasons aict grep is ~100× slower than GNU grep or ripgrep. Adding a --workers flag to enable parallel file scanning would significantly improve performance on multi-core systems.

Current Behavior

$ aict grep "func" . -r
# Scans files one at a time in a single goroutine

Proposed Behavior

$ aict grep "func" . -r --workers 4
# Spawns 4 worker goroutines to scan files in parallel

$ aict grep "func" . -r --workers auto
# Automatically sets workers to runtime.NumCPU()

Implementation Notes

  • Use a worker pool pattern with a buffered channel for file paths
  • Collect results via a separate goroutine that writes XML elements as they're discovered
  • Default to --workers 1 (sequential) to maintain current behavior and avoid memory spikes on huge directories
  • --workers auto should resolve to runtime.NumCPU()
  • Ensure thread-safe result ordering if --sort is used (or document that parallel mode does not guarantee order)

Acceptance Criteria

  • --workers <n> flag accepts positive integers
  • --workers auto resolves to number of logical CPUs
  • Default remains sequential (1 worker) for backward compatibility
  • Benchmark shows >2× speedup on a 10k-file codebase with --workers 4
  • XML output remains valid regardless of worker count
  • Tests cover: single worker, multiple workers, auto, and invalid values (0, -1)

Activity

  1. tristaamnee commented on Apr 6, 2026

    @tristaamnee
    Contributor

    Hi @ronak-create , I see this project is in its early stages and I'd love to contribute by implementing the parallel worker pool for grep. I've been working with Go for a while. Can you assign this to me?

  2. tristaamnee commented on Apr 7, 2026

    @tristaamnee
    Contributor

    Hi @ronak-create , I just woke up and noticed I was unassigned. I'm still very interested and currently working on the implementation. Could you please re-assign it to me? I'll share my progress shortly. Thanks!

  3. tristaamnee commented on Apr 7, 2026

    @tristaamnee
    Contributor

    Hi @ronak-create , I've successfully implemented and tested the --workers flag locally. It correctly parses both integer values and is ready for the worker pool logic. Could you please assign this back to me so I can submit the formal Pull Request? Thanks!

    Image
  4. ronak-create commented on Apr 7, 2026

    @ronak-create
    MemberAuthor

    wait, ill assign you after a short cooldown

  5. u84u commented on Apr 7, 2026

    @u84u
    Member

    completed in #10

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions