Skip to content

feat: runner-aware tools#346

Draft
branchseer wants to merge 1 commit intofeat/output-restorationfrom
runner-aware-tools
Draft

feat: runner-aware tools#346
branchseer wants to merge 1 commit intofeat/output-restorationfrom
runner-aware-tools

Conversation

@branchseer
Copy link
Copy Markdown
Member

@branchseer branchseer commented Apr 18, 2026

Set up a IPC channel between vite-task and the processes it spawns, so the spawned tools can declare at runtime what they actually read, wrote, or cared about, and then vite-task uses that to decide what to fingerprint in the cache.

Design notes: docs/runner-task-ipc/.

Problems this PR solves

Every example below is exercised by patches/vite.patch, which wires vite build into the IPC through @voidzero-dev/vite-task-client.

1. Dynamic tracked envs

Before: the user had to declare every relevant env in vite-task.json, statically:

{
  "tasks": {
    "build": { "env": ["NODE_ENV", "VITE_*"], "cache": true }
  }
}

This duplicates knowledge the tool already has. Forgetting NODE_ENV silently skips cache invalidation on mode change. envPrefix-matching envs (VITE_* by default) get inlined into the bundle through import.meta.env.* — so changing envPrefix: 'MYAPP_' in vite.config.js without updating vite-task.json drifts: the runner still tracks VITE_* while the build output is driven by MYAPP_*.

After: the tool declares its envs at runtime, driven by its own config.

// vite's resolveConfig
fetchEnv("NODE_ENV", { tracked: true });

// vite's loadEnv, one call per configured prefix — these envs are
// exposed to client code as import.meta.env.*, so their values are
// baked into the bundle
for (const prefix of envPrefix) {
  fetchEnvs(`${prefix}*`, { tracked: true });
}

The build task in vite-task.json needs no env: at all. Changing envPrefix in vite.config.js dynamically changes the set of envs the runner tracks, with zero config edits on the runner side.

2. Exclude tool's cache dir from input/output

Vite stores pre-bundled deps under node_modules/.vite/ and bundled configs under node_modules/.vite-temp/. Every build reads the cache metadata (to check staleness) and writes fresh entries when it isn't stale. Without intervention the runner sees:

  • the reads → implicit inputs, so the cache key depends on dep-cache contents
  • the writes → implicit outputs
  • the same directory both read and written → the runner refuses to cache the run at all (read-write overlap)

There is a workaround already in vite-plus: voidzero-dev/vite-plus#1096 plus its follow-up #1198 hardcode !node_modules/.vite-temp/**, !node_modules/.vite/**/results.json, and !dist/** as negative input globs on every vp subcommand (build, test, pack). That's not good enough:

  • Leaks vite internals into vp. Every time vite changes its cache layout (new path under .vite/, moved temp dir, new subcommand with its own transient files), vp has to ship a matching glob update. It's a lockstep coupling that design-wise shouldn't exist.
  • Input-only, not symmetric. The globs suppress reads for the input fingerprint (which is enough to break the read-write overlap check), but the writes are still captured as outputs — meaning transient cache contents get archived into the runner's cache and restored on every hit, bloating the cache store.
  • Per-subcommand, per-tool duplication. #1198 already had to retrofit the same glob into three subcommands. Any new subcommand, and any third-party tool with similar behavior (Nuxt's .nuxt/, SvelteKit's .svelte-kit/, Next's .next/), needs its own hand-maintained list — vp can't ship it generically.

After:

// in loadCachedDepOptimizationMetadata
const depsCacheDir = getDepsCacheDir(environment);
ignoreInput(depsCacheDir);
ignoreOutput(depsCacheDir);

The declaration lives with the tool that owns the directory. The dep cache is vite's private concern.

3. Exclude output from input when a tool clears the folder before writing it

vite build calls emptyDir(outDir) before writing dist/. emptyDir has to read the directory entries to know what to delete — those reads look identical to genuine input reads. Since dist/ is also where vite writes its final output, the runner sees a read-write overlap on the same paths and refuses to cache.

After:

// in prepareOutDir, right before emptyDir()
ignoreInput(outDir);

Only the writes count. The pattern generalizes: any tool that wipes-then-writes the same directory needs to tell the runner "my enumeration reads aren't inputs."

What's in this PR

  • Step 1 — Protocol (vite_task_ipc_shared): message types + serialization shared by both ends.
  • Step 2 — Transport (vite_task_server + vite_task_client): async server, sync blocking client, tested Rust-to-Rust.
  • Step 3 — Extract artifact crate out of fspy for dylib embedding. (Landed on main via refactor: extract materialized_artifact crate out of fspy #344 as materialized_artifact.)
  • Step 4 — JS bridge: vite_task_client_napi + @voidzero-dev/vite-task-client JS wrapper (fetchEnv single-name + fetchEnvs glob, with dedupe against already-set process.env).
  • Step 5 — Runner integration: server started per task execution, client dylib embedded/extracted, IPC envs injected via serve()'s returned iterator.
  • Step 6 — Cache integration: runner consumes reported ignored inputs/outputs, tracked env requests (single + glob), and disable-cache signals when fingerprinting.

Test plan

  • Rust integration tests for server/client transport (vite_task_server/tests/integration.rs)
  • E2E snapshot fixtures per client method: ignore_input, ignore_output, fetch_env, fetch_envs_glob, disable_cache
  • E2E test caching a real vite build via patches/vite.patch (vite_build_cache fixture): NODE_ENV-change invalidation, envPrefix-driven tracked-env set change, dist/ write restoration on cache hit

@branchseer branchseer changed the base branch from main to graphite-base/346 April 20, 2026 02:19
@branchseer branchseer changed the base branch from graphite-base/346 to feat/output-restoration April 20, 2026 02:19
Copy link
Copy Markdown
Member Author

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@branchseer branchseer changed the title feat(ipc): runner-aware tools — protocol + transport (partial) feat: runner-aware tools Apr 20, 2026
@branchseer branchseer force-pushed the feat/output-restoration branch from 0008bd7 to 994624a Compare April 20, 2026 04:20
* @param {string} name
* @param {{ tracked?: boolean }} [options]
*/
export function fetchEnv(name, { tracked = true } = {}) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is synchronous, and doesn't fetch anything, I recommend calling it getEnv instead.

@socket-security
Copy link
Copy Markdown

socket-security Bot commented Apr 23, 2026

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addedcargo/​napi@​3.8.68210093100100
Addednpm/​vite@​8.0.8991008298100
Addedcargo/​napi-build@​2.3.19810093100100
Addedcargo/​napi-derive@​3.5.59910093100100
Addedcargo/​ctor@​0.11.19810093100100
Addedcargo/​interprocess@​2.4.299100100100100

View full report

@branchseer branchseer force-pushed the runner-aware-tools branch 2 times, most recently from e18c21a to 09a310f Compare April 23, 2026 07:42
@branchseer branchseer force-pushed the feat/output-restoration branch 2 times, most recently from 589a626 to 9df3054 Compare April 23, 2026 08:00
@branchseer branchseer force-pushed the feat/output-restoration branch from 9df3054 to 3a8b605 Compare May 7, 2026 08:08
@branchseer branchseer force-pushed the runner-aware-tools branch from 4edcd12 to cfa4282 Compare May 7, 2026 08:08
@branchseer branchseer force-pushed the feat/output-restoration branch from 3a8b605 to 6198f6b Compare May 7, 2026 08:23
@branchseer branchseer force-pushed the runner-aware-tools branch 2 times, most recently from ae11247 to 24f5889 Compare May 7, 2026 08:35
Squashed rebase of 51 commits from runner-aware-tools branch onto
feat/output-restoration. Original commit history preserved on the
ras-backup ref for reference.

Key features bundled:
- vite_task_ipc_shared: shared protocol (Request/GetEnvResponse, NativeStr)
- vite_task_server: per-task IPC server (Handler trait + Recorder)
- vite_task_client: sync Rust client
- vite_task_client_napi + @voidzero-dev/vite-task-client: node addon + JS wrapper
- vite_task: wire IPC server into spawn; inject VP_IPC + VP_RUN_NODE_CLIENT_PATH;
  bundle with fspy via Tracking struct; materialize .node addon on first use
- consume runner-aware tool reports for cache decisions:
  * disableCache() short-circuits via ToolRequested
  * ignoreInput / ignoreOutput filter fspy reads/writes
  * tracked: true env / env-glob records folded into PostRunFingerprint
- IPC server failure surfaces via IpcServerError; cache update is skipped
- schema bumped to user_version = 13 (CacheEntryKey carries output_config,
  CacheEntryValue carries output_archive + tracked envs)

Conflicts resolved against post-rebase main (#352 cfg(fspy) gating, #321
output archiving, input-negative reads-only filter): TrackingOutcome
post-run summary preserved alongside Tracking pre-run handle; auto-input
reads gated on input_config.includes_auto and filtered by input negatives
+ ignoreInput; auto-output writes filtered by negatives + ignoreOutput.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@branchseer branchseer force-pushed the runner-aware-tools branch from 24f5889 to ffca388 Compare May 7, 2026 08:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants