Skip to content

Background auto-sync uses a synchronous file walk that can block the event loop #1

Description

@ashishmishra26

The MCP server auto-syncs the index in the background as queries arrive (maybeAutoSync in src/mcp/server.ts). The sync runs syncRepo -> walkRepo, and walkRepo (src/core/walker.ts) uses synchronous fs (readdirSync, statSync, readFileSync). So while a background sync's stat-walk runs, it blocks the Node event loop for the duration of the walk.

Impact

The triggering query is usually unaffected: the sync is deferred with setImmediate, debounced (default CX_SYNC_INTERVAL_SECS=30), and single-flight, so a fast query typically resolves before the walk starts. The residual case is a concurrent or immediately-following query that lands while the walk is running, which can be delayed by up to the walk time. That is a few tens of ms on typical repos but can reach ~1-2s on very large trees.

This does not affect token counts or tool-call counts (the walk is server-side and invisible to the model). It is purely a tail-latency concern on large repos.

Proposed fix

Move the walk and the change-detection reads off the event loop:

  • async fs (fs/promises / opendir) so the walk yields between entries, or
  • run the sync in a worker thread.

Either one removes the event-loop blocking entirely, so a background sync can never delay a concurrent query.

Notes

  • Debounce + deferral + single-flight already bound how often and how much this happens. This is about removing the residual blocking, so it is a v0.2 candidate rather than urgent.
  • For latency benchmarking in the meantime: on a static repo the walk finds no changes (near no-op), and CX_AUTO_SYNC=0 removes even the stat-walk from the query path.

Refs: src/core/walker.ts (walkRepo, readdirSync/statSync), src/mcp/server.ts (maybeAutoSync).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions