diff --git a/docs/bug-reports/testrun-enobufs-stream-to-file.md b/docs/bug-reports/testrun-enobufs-stream-to-file.md new file mode 100644 index 0000000..4a04dc9 --- /dev/null +++ b/docs/bug-reports/testrun-enobufs-stream-to-file.md @@ -0,0 +1,158 @@ +# Bug report — `provar_automation_testrun` fails with `ENOBUFS` on verbose runs + +> **Recommendation:** replace the in-memory `spawnSync` capture in `runSfCommand` with **stream-to-file** stdio so there is no buffer ceiling to overflow. + +| | | +| -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | +| **Component** | `@provartesting/provardx-cli` — MCP server (`provar mcp start`) | +| **Tool** | `provar_automation_testrun` (also affects `…_compile`, `…_metadata_download`, `…_setup`, all `qualityhub_*` — shared code path) | +| **Version observed** | 1.6.0 (`develop`) | +| **Severity** | High — recurring; the primary MCP run tool fails on real Salesforce UI runs and forces a manual CLI fallback | +| **Platform observed** | Windows 11, sf standalone CLI (`C:\Program Files\sf\client\bin\sf.cmd`, so `shell:true`), Provar 307 | +| **Status of current mitigation** | `maxBuffer` was already raised 1 MB → 50 MB (`sfSpawn.ts:95`); **still reproduces** | + +--- + +## 1. Symptom + +The tool returns an opaque error and the run is lost (even though the underlying Provar run actually started and wrote results to disk): + +```json +{ + "error_code": "ENOBUFS", + "message": "spawnSync C:\\WINDOWS\\system32\\cmd.exe ENOBUFS", + "retryable": false, + "requestId": "cb8b2b24-…" +} +``` + +Running the same thing in the terminal succeeds: + +``` +sf provar automation test run --json # → {"status":0,"result":{"success":true}} +``` + +## 2. Reproduction + +1. `provar_automation_config_load` a `provardx-properties.json` with `testOutputLevel: "DETAILED"`, `connectionRefreshType: "Reload"`, `metadataLevel: "Reuse"`, targeting a real Salesforce Lightning UI test (e.g. a Lead-convert flow against a sandbox). +2. `provar_automation_testrun`. +3. On a verbose run — **especially the first run after a metadata reload** — the tool aborts with `ENOBUFS`. The same command via `sf provar automation test run --json` in a terminal completes normally. + +## 3. Root cause + +`runSfCommand` captures the child's **entire** stdout/stderr in an in-memory buffer: + +```ts +// src/mcp/tools/sfSpawn.ts:282 +const result = sfSpawnHelper.spawnSync(spawnExecutable, spawnArgs, { + encoding: 'utf-8', + shell: useShell, + maxBuffer: MAX_BUFFER, // 50 * 1024 * 1024 (sfSpawn.ts:95) +}); +``` + +`spawnSync` buffers all child output in RAM and, the moment the combined output exceeds `maxBuffer`, **aborts the whole call** with `result.error.code === 'ENOBUFS'` (message `spawnSync ENOBUFS`). `runSfCommand` then re-throws that error (`sfSpawn.ts:288-294`) and `handleSpawnError` surfaces it verbatim (`automationTools.ts:59-75`). + +A `DETAILED` Provar run emits a very large stdout — every step plus the Java **schema-validator / logger noise** that `filterTestRunOutput` exists to strip. Crucially, that filter runs **after** the full buffer is captured (`automationTools.ts` testrun handler), so it cannot prevent the overflow that kills the call. + +### Evidence the 50 MB build still overflows (not just a stale process) + +- `MAX_BUFFER` is already 50 MB in `1.6.0` and committed on `develop` (`sfSpawn.ts:95`), and the tool description claims ENOBUFS "is now rare" (`automationTools.ts:309`). +- The MCP update cache (`$PROVAR_HOME/.cache/.mcp-update-cache.json`) recorded `currentVersion: "1.6.0"` with a `checkedAt` timestamp **~13 minutes before** the failing run — i.e. a 50 MB build was live and _still_ hit ENOBUFS on a single verbose Lead-convert run. + +### Secondary (Windows) factor + +With `shell: true` through `cmd.exe`, a large piped output can also surface a genuine OS-level `ENOBUFS` ("No buffer space available") independent of `maxBuffer`. Both failure modes share the same remedy below. + +## 4. Why raising the cap is not the fix + +A cap is a ceiling. Set it to _N_ and a verbose-enough run (bigger org, more steps, validator dumps, first-run metadata logging) dies at _N+1_. The 1 MB → 50 MB bump reduced frequency but did not remove the failure class. **The output must not be buffered in memory at all.** + +## 5. Proposed fix — stream child stdio to temp files + +Capture the child's stdout/stderr to **files** instead of an in-memory pipe, then read them back after the process exits. No in-memory cap ⇒ `ENOBUFS` becomes structurally impossible from `maxBuffer`, and because the child writes straight to a file descriptor there is no pipe back-pressure (also fixes the Windows OS-level `ENOBUFS`). Provar already persists JUnit + logs to the results dir, so disk is the natural sink. + +```ts +// Sketch for runSfCommand — replace the single spawnSync block. +import { mkdtempSync, openSync, closeSync, readFileSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; + +const dir = mkdtempSync(join(tmpdir(), 'provar-sf-')); +const outPath = join(dir, 'stdout.log'); +const errPath = join(dir, 'stderr.log'); +const outFd = openSync(outPath, 'w'); +const errFd = openSync(errPath, 'w'); +try { + const result = sfSpawnHelper.spawnSync(spawnExecutable, spawnArgs, { + shell: useShell, + stdio: ['ignore', outFd, errFd], // no maxBuffer, no encoding — not piped to memory + }); + closeSync(outFd); + closeSync(errFd); + if (result.error) { + const err = result.error as NodeJS.ErrnoException; + if (err.code === 'ENOENT') throw new SfNotFoundError(resolvedSfPath); + throw result.error; + } + return { + stdout: readFileSync(outPath, 'utf-8'), + stderr: readFileSync(errPath, 'utf-8'), + exitCode: result.status ?? 1, + }; +} finally { + try { + closeSync(outFd); + } catch { + /* already closed */ + } + try { + closeSync(errFd); + } catch { + /* already closed */ + } + rmSync(dir, { recursive: true, force: true }); +} +``` + +### Notes / edge cases + +- **Signature unchanged** — callers still receive `{ stdout, stderr, exitCode }` strings; no change needed in `config_load`, `testrun`, `compile`, `metadata_download`, `setup`, `qualityhub_*`, or `probeProvarTopic`. +- **Tiny probe spawns** (`resolveSfExecutable` `--version` at `sfSpawn.ts:180/193`, and `updateChecker` at `updateChecker.ts:42`) can stay in-memory (output is bytes) or be given the same treatment for consistency — low priority. +- **Reading back a huge file** still allocates a large string. Since this is mainly for `filterTestRunOutput`, consider an optional enhancement: for `testrun`, return only a head+tail slice (e.g. first/last 200 KB) plus a pointer to the full on-disk log, leaving the complete output untouched on disk. This bounds the MCP response size as well. +- **Cleanup** in `finally`; `mkdtempSync` gives a unique dir so there's no collision (note: `Date.now()`/`Math.random()` are fine in the CLI; only the MCP-workflow sandbox forbids them). +- **Windows `shell:true`** works with fd stdio — `cmd.exe` inherits the handles. + +## 6. Secondary improvement — make `ENOBUFS` actionable (defense in depth) + +Even after the streaming fix, `handleSpawnError` (`automationTools.ts:59`) should translate a residual `code: 'ENOBUFS'` into something useful instead of `spawnSync … ENOBUFS`, e.g.: + +> "Provar produced more output than the capture buffer. The full run results are on disk at ``. Re-run with `sf provar automation test run --json`, or lower `testOutputLevel`." + +And update the now-inaccurate tool-description lines (`automationTools.ts:309-310`) once buffering is removed. + +## 7. Affected code + +- `src/mcp/tools/sfSpawn.ts` — `runSfCommand` (primary change), `MAX_BUFFER` constant retired/relocated. +- No caller signature changes. + +## 8. Test impact + +- `test/unit/mcp/sfSpawn.test.ts` and `test/unit/mcp/automationTools.test.ts` stub `sfSpawnHelper.spawnSync` and currently assert on `{ maxBuffer, encoding }` and read `result.stdout`. These must move to the file-backed model — stub/seed the temp files (or inject the temp dir) and assert on the `stdio: ['ignore', fd, fd]` shape instead of `maxBuffer`. +- **Add a regression test**: simulate child output larger than the old cap and assert (a) no throw and (b) full capture round-trips through the temp files. + +## 9. Rollout caveats + +- The fix only takes effect once it is **built, published, and the MCP reconnects**. A hand-edited copy under `…/AppData/Local/sf/node_modules/@provartesting/provardx-cli` is **overwritten by `--auto-update true` on the next publish**, so the change must ship in the package — local patching is not a durable workaround. + +## 10. Acceptance criteria + +1. A `DETAILED` testrun whose stdout exceeds 50 MB completes through `provar_automation_testrun` with a success result and **no `ENOBUFS`** — on Windows (`shell:true`) and POSIX. +2. The tool still returns filtered stdout + JUnit `steps[]`. +3. Unit tests updated to the streaming model and green; new over-cap regression test added. + +--- + +### Appendix — current call chain + +`provar_automation_testrun` handler (`automationTools.ts`, ~`registerAutomationTestRun`) → `runSfCommand(['provar','automation','test','run', …flags])` (`sfSpawn.ts:253`) → `sfSpawnHelper.spawnSync(… { maxBuffer: 50 MB })` (`sfSpawn.ts:282`) → on overflow throws `ENOBUFS` → `handleSpawnError` (`automationTools.ts:59`). diff --git a/docs/mcp.md b/docs/mcp.md index 826898a..03fbf98 100644 --- a/docs/mcp.md +++ b/docs/mcp.md @@ -1665,6 +1665,8 @@ Triggers a Provar Automation test run using the currently loaded properties file The `stdout` field is filtered before returning: Java schema-validator lines (`com.networknt.schema.*`) and stale logger-lock `SEVERE` warnings are stripped. If any lines were suppressed, `output_lines_suppressed` contains the count. +**Output handling:** the child process's `stdout`/`stderr` are streamed to temporary files rather than buffered in memory, so a verbose run (e.g. `testOutputLevel: "DETAILED"`) no longer fails with `ENOBUFS` from an output-buffer overflow, even for tens-of-MB logs that previously exceeded the in-memory cap. (The captured output is still read back into the response after the run, so an extreme multi-hundred-MB run can hit Node's max string length — far above the old 50 MB boundary.) If a residual OS-level `ENOBUFS` ever surfaces, the tool returns an actionable `ENOBUFS` error (with a `details.suggestion`) pointing to the on-disk results and the `--json` re-run path rather than the opaque `spawnSync … ENOBUFS`. + After each run, the tool scans the results directory for JUnit XML files and adds a `steps` array when results are found: ```json @@ -1693,7 +1695,7 @@ warning is additive and never flips exitCode or sets isError; the failure surfac ▎ what ran". In those cases the response carries details.warning (explaining why structured step data is missing) and RUN-001 is suppressed to ▎ avoid misdirecting the agent toward a typo when the real issue is a missing/unreadable results dir. -Error codes: AUTOMATION_TESTRUN_FAILED, SF_NOT_FOUND, PROVAR_PLUGIN_NOT_FOUND +Error codes: AUTOMATION_TESTRUN_FAILED, SF_NOT_FOUND, PROVAR_PLUGIN_NOT_FOUND, ENOBUFS (residual; output is streamed to disk so this is now rare) Warning codes: RUN-001 (zero tests executed despite success) ``` diff --git a/src/mcp/tools/automationTools.ts b/src/mcp/tools/automationTools.ts index e4295ce..22681a9 100644 --- a/src/mcp/tools/automationTools.ts +++ b/src/mcp/tools/automationTools.ts @@ -18,6 +18,7 @@ import { assertPathAllowed, PathPolicyError } from '../security/pathPolicy.js'; import { WARNING_CODES, formatWarning } from '../utils/warningCodes.js'; import { parseJUnitResults } from './antTools.js'; import { runSfCommand, isProvarPluginMissing, PROVAR_PLUGIN_INSTALL_HINT } from './sfSpawn.js'; +import { handleSpawnError } from './spawnErrors.js'; import { desc } from './descHelper.js'; // Re-export sf resolution helpers so existing test imports from automationTools continue to work @@ -56,24 +57,6 @@ function provarPluginErrorResponse( }; } -function handleSpawnError( - err: unknown, - requestId: string, - toolName: string -): { isError: true; content: Array<{ type: 'text'; text: string }> } { - const error = err as Error & { code?: string }; - log('error', `${toolName} failed`, { requestId, error: error.message }); - return { - isError: true as const, - content: [ - { - type: 'text' as const, - text: JSON.stringify(makeError(error.code ?? 'SF_ERROR', error.message, requestId, false)), - }, - ], - }; -} - // ── Tool: provar_automation_config_load ────────────────────────────────────── export function registerAutomationConfigLoad(server: McpServer, config: ServerConfig): void { @@ -306,8 +289,7 @@ export function registerAutomationTestRun(server: McpServer, config: ServerConfi 'Requires Provar to be installed locally and provarHome set correctly in the properties file.', 'Use provar_automation_setup first if Provar is not yet installed.', 'For grid/CI execution via Provar Quality Hub instead of running locally, use provar_qualityhub_testrun.', - 'Output buffer: a 50 MB maxBuffer is set so ENOBUFS on verbose Provar runs is now rare.', - 'If ENOBUFS still occurs (extremely verbose logging), run `sf provar automation test run --json` directly in the terminal and pipe or tail the output instead of retrying this tool.', + 'Output handling: the child stdout/stderr are streamed to disk (not buffered in memory), so a verbose run (e.g. testOutputLevel DETAILED) no longer fails with ENOBUFS from an output-buffer overflow.', 'Zero-tests guard: if the sf exit code is 0, the results directory was located, and at least one JUnit XML file parsed successfully but contains zero executed tests, a RUN-001 warning is added to `warnings[]` — usually a typo such as `testCase` vs `testCases` in provardx-properties.json. When no JUnit data is available (dir missing or all XML unparseable), `details.warning` is set instead and RUN-001 stays silent.', 'Typical local AI loop: config.load → compile → testrun → inspect results.', 'Each failed step in `steps[]` may include optional error_category (INFRASTRUCTURE|ASSERTION|LOCATOR|TIMEOUT|OTHER)', diff --git a/src/mcp/tools/qualityHubTools.ts b/src/mcp/tools/qualityHubTools.ts index ca5921e..8bfff6f 100644 --- a/src/mcp/tools/qualityHubTools.ts +++ b/src/mcp/tools/qualityHubTools.ts @@ -13,26 +13,9 @@ import { log } from '../logging/logger.js'; import { applyDetailLevel, type DetailLevel } from '../utils/detailLevel.js'; import { maskFields, parseFieldsParam } from '../utils/fieldMask.js'; import { runSfCommand } from './sfSpawn.js'; +import { handleSpawnError } from './spawnErrors.js'; import { desc } from './descHelper.js'; -function handleSpawnError( - err: unknown, - requestId: string, - toolName: string -): { isError: true; content: Array<{ type: 'text'; text: string }> } { - const error = err as Error & { code?: string }; - log('error', `${toolName} failed`, { requestId, error: error.message }); - return { - isError: true as const, - content: [ - { - type: 'text' as const, - text: JSON.stringify(makeError(error.code ?? 'SF_ERROR', error.message, requestId, false)), - }, - ], - }; -} - const QH_SUMMARY_FIELDS = ['requestId', 'exitCode']; // ── Tool: provar_qualityhub_connect ─────────────────────────────────────────── diff --git a/src/mcp/tools/sfSpawn.ts b/src/mcp/tools/sfSpawn.ts index 494dc6a..e93ac13 100644 --- a/src/mcp/tools/sfSpawn.ts +++ b/src/mcp/tools/sfSpawn.ts @@ -92,8 +92,6 @@ export interface SpawnResult { exitCode: number; } -const MAX_BUFFER = 50 * 1024 * 1024; // 50 MB — prevents ENOBUFS on verbose Provar runs - // ── SF CLI discovery ────────────────────────────────────────────────────────── /** @@ -279,25 +277,118 @@ export function runSfCommand(args: string[], sfPath?: string): SpawnResult { const spawnExecutable = useShell ? quoteWindowsToken(executable) : executable; const spawnArgs = useShell ? args.map(quoteWindowsToken) : args; - const result = sfSpawnHelper.spawnSync(spawnExecutable, spawnArgs, { - encoding: 'utf-8', - shell: useShell, - maxBuffer: MAX_BUFFER, - }); + return captureSpawnToFiles(spawnExecutable, spawnArgs, useShell, resolvedSfPath); +} - if (result.error) { - const err = result.error as NodeJS.ErrnoException; - if (err.code === 'ENOENT') { - throw new SfNotFoundError(resolvedSfPath); +/** + * Run spawnSync capturing the child's stdout/stderr to temp files instead of an + * in-memory pipe, then read them back after the child exits. + * + * spawnSync's `maxBuffer` buffers all child output in RAM and aborts the whole + * call with `ENOBUFS` the moment that ceiling is crossed — a verbose Provar run + * (e.g. testOutputLevel DETAILED) routinely overflowed even a 50 MB cap and lost + * the run. Streaming straight to file descriptors removes the in-memory ceiling + * during capture, so `maxBuffer`-induced ENOBUFS is structurally impossible. + * Because the child writes directly to a file descriptor there is also no pipe to + * fill, which avoids the OS-level `ENOBUFS` seen on Windows under `shell: true`. + * + * (The captured output is still read back into a string after the child exits, so + * an extreme multi-hundred-MB run can hit Node's max string length — far above the + * 50 MB pipe boundary this removes. Bounding that read is tracked separately.) + * + * The temp directory is removed on a best-effort basis. Throws SfNotFoundError on + * ENOENT (sf missing) and rethrows any other spawn error. + */ +function captureSpawnToFiles( + spawnExecutable: string, + spawnArgs: string[], + useShell: boolean, + resolvedSfPath: string | undefined +): SpawnResult { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'provar-sf-')); + try { + return spawnCapturingToDir(dir, spawnExecutable, spawnArgs, useShell, resolvedSfPath); + } finally { + // Best-effort cleanup: a temp-dir removal failure (e.g. Windows EBUSY/EPERM + // from an antivirus/indexer handle, which `force: true` does NOT suppress) + // must never overwrite the real return value or mask the real spawn error. + try { + fs.rmSync(dir, { recursive: true, force: true }); + } catch { + /* leave the dir; the OS reclaims its temp directory eventually */ } - throw result.error; + } +} + +/** + * Open the capture files in `dir`, run the child with its stdout/stderr inherited + * onto those descriptors, and read the captured output back. Split out from + * captureSpawnToFiles so the temp-dir removal in its `finally` also covers the + * case where opening the second descriptor fails. + */ +function spawnCapturingToDir( + dir: string, + spawnExecutable: string, + spawnArgs: string[], + useShell: boolean, + resolvedSfPath: string | undefined +): SpawnResult { + const outPath = path.join(dir, 'stdout.log'); + const errPath = path.join(dir, 'stderr.log'); + const outFd = fs.openSync(outPath, 'w'); + let errFd: number; + try { + errFd = fs.openSync(errPath, 'w'); + } catch (openErr) { + // The first descriptor is already open; close it before unwinding so a + // failure to open the second (e.g. EMFILE) doesn't leak it. + fs.closeSync(outFd); + throw openErr; } - return { - stdout: result.stdout ?? '', - stderr: result.stderr ?? '', - exitCode: result.status ?? 1, + let fdsClosed = false; + const closeFds = (): void => { + if (fdsClosed) return; + fdsClosed = true; + try { + fs.closeSync(outFd); + } catch { + /* already closed */ + } + try { + fs.closeSync(errFd); + } catch { + /* already closed */ + } }; + + try { + // No `encoding`/`maxBuffer`: stdout/stderr are streamed to the inherited file + // descriptors rather than captured in memory, so there is no buffer to overflow. + const result = sfSpawnHelper.spawnSync(spawnExecutable, spawnArgs, { + shell: useShell, + stdio: ['ignore', outFd, errFd], + }); + // spawnSync has already reaped the child, so its writes are flushed to disk; + // close our inherited copies of the descriptors before reading the files back. + closeFds(); + + if (result.error) { + const err = result.error as NodeJS.ErrnoException; + if (err.code === 'ENOENT') { + throw new SfNotFoundError(resolvedSfPath); + } + throw result.error; + } + + return { + stdout: fs.readFileSync(outPath, 'utf-8'), + stderr: fs.readFileSync(errPath, 'utf-8'), + exitCode: result.status ?? 1, + }; + } finally { + closeFds(); + } } // ── SOQL safety ─────────────────────────────────────────────────────────────── diff --git a/src/mcp/tools/spawnErrors.ts b/src/mcp/tools/spawnErrors.ts new file mode 100644 index 0000000..b91286d --- /dev/null +++ b/src/mcp/tools/spawnErrors.ts @@ -0,0 +1,61 @@ +/* + * Copyright (c) 2024 Provar Limited. + * All rights reserved. + * Licensed under the BSD 3-Clause license. + * For full license text, see LICENSE.md file in the repo root or https://opensource.org/licenses/BSD-3-Clause + */ + +import { makeError } from '../schemas/common.js'; +import { log } from '../logging/logger.js'; + +// Defense in depth: runSfCommand streams child output to disk, so a maxBuffer +// overflow can no longer raise ENOBUFS. Should a residual ENOBUFS still surface +// (e.g. an exotic OS-level pipe condition), translate the opaque +// `spawnSync … ENOBUFS` into something the agent can act on rather than retry. +// Kept tool-agnostic because this handler is shared by every sf-backed tool +// (automation + Quality Hub); the test-run example is illustrative, not assumed. +const ENOBUFS_MESSAGE = + 'The sf command produced more output than could be captured. Its full output was written to disk ' + + '(for test runs, under the resultsPath configured in your provardx-properties.json). ' + + 'Re-run the command directly in a terminal with --json, or reduce its output verbosity ' + + '(for test runs, lower testOutputLevel, e.g. DETAILED → BASIC).'; +const ENOBUFS_SUGGESTION = + 'Re-run the sf command directly with --json, or reduce its output verbosity ' + + '(for test runs, lower testOutputLevel in provardx-properties.json).'; + +/** + * Shared error handler for the sf-CLI-backed tools (automation + Quality Hub). + * Surfaces a thrown spawn error as an MCP error response, translating a residual + * ENOBUFS into actionable remediation and otherwise preserving the error's own + * code (falling back to SF_ERROR). + */ +export function handleSpawnError( + err: unknown, + requestId: string, + toolName: string +): { isError: true; content: Array<{ type: 'text'; text: string }> } { + const error = err as Error & { code?: string }; + log('error', `${toolName} failed`, { requestId, error: error.message }); + if (error.code === 'ENOBUFS') { + return { + isError: true as const, + content: [ + { + type: 'text' as const, + text: JSON.stringify( + makeError('ENOBUFS', ENOBUFS_MESSAGE, requestId, false, { suggestion: ENOBUFS_SUGGESTION }) + ), + }, + ], + }; + } + return { + isError: true as const, + content: [ + { + type: 'text' as const, + text: JSON.stringify(makeError(error.code ?? 'SF_ERROR', error.message, requestId, false)), + }, + ], + }; +} diff --git a/test/unit/mcp/automationTools.test.ts b/test/unit/mcp/automationTools.test.ts index 4765e69..bdd99ea 100644 --- a/test/unit/mcp/automationTools.test.ts +++ b/test/unit/mcp/automationTools.test.ts @@ -50,19 +50,45 @@ type FakeSpawnResult = { stdout: string; stderr: string; status: number | null; - error: Error | undefined; + error: (Error & { code?: string }) | undefined; pid: number | undefined; output: string[]; signal: null; }; +type SpawnFake = (...callArgs: unknown[]) => FakeSpawnResult; + +// runSfCommand streams the child's stdout/stderr to temp files rather than +// buffering them in memory, so the stub for sfSpawnHelper.spawnSync mimics a real +// child: it writes the captured output to the file descriptors passed via +// `stdio: ['ignore', outFd, errFd]`, then returns the spawn result. The in-memory +// probe path (encoding/maxBuffer, no fd stdio) just receives the object. +function spawnFake(result: FakeSpawnResult): SpawnFake { + return (...callArgs) => { + const stdio = (callArgs[2] as { stdio?: unknown[] } | undefined)?.stdio; + const outFd = stdio?.[1]; + const errFd = stdio?.[2]; + if (typeof outFd === 'number' && typeof errFd === 'number') { + fs.writeSync(outFd, result.stdout); + fs.writeSync(errFd, result.stderr); + } + return result; + }; +} -function makeSpawnResult(stdout: string, stderr: string, status: number): FakeSpawnResult { - return { stdout, stderr, status, error: undefined, pid: 1, output: [], signal: null }; +function makeSpawnResult(stdout: string, stderr: string, status: number): SpawnFake { + return spawnFake({ stdout, stderr, status, error: undefined, pid: 1, output: [], signal: null }); } -function makeEnoentResult(): FakeSpawnResult { +function makeEnoentResult(): SpawnFake { const err = Object.assign(new Error('spawn sf ENOENT'), { code: 'ENOENT' }); - return { stdout: '', stderr: '', status: null, error: err, pid: undefined, output: [], signal: null }; + return spawnFake({ stdout: '', stderr: '', status: null, error: err, pid: undefined, output: [], signal: null }); +} + +// A raw spawn-result object carrying a spawn-level error (e.g. ENOBUFS / EPIPE). +// runSfCommand throws on `result.error` before reading the temp files, so this is +// returned directly rather than wrapped as a file-writing fake. +function makeSpawnResultRaw(error: Error & { code?: string }): FakeSpawnResult { + return { stdout: '', stderr: '', status: null, error, pid: undefined, output: [], signal: null }; } function parseBody(result: unknown): Record { @@ -100,7 +126,7 @@ describe('automationTools', () => { describe('provar_automation_testrun', () => { it('calls sf with correct args and returns stdout', () => { - spawnStub.returns(makeSpawnResult('tests passed', '', 0)); + spawnStub.callsFake(makeSpawnResult('tests passed', '', 0)); const result = server.call('provar_automation_testrun', { flags: [] }); const body = parseBody(result); assert.equal(body.exitCode, 0); @@ -111,7 +137,7 @@ describe('automationTools', () => { }); it('forwards extra flags to sf', () => { - spawnStub.returns(makeSpawnResult('ok', '', 0)); + spawnStub.callsFake(makeSpawnResult('ok', '', 0)); server.call('provar_automation_testrun', { flags: ['--project-path', '/my/project'] }); const args = spawnStub.firstCall.args[1] as string[]; assert.ok(args.includes('--project-path')); @@ -119,7 +145,7 @@ describe('automationTools', () => { }); it('returns isError and AUTOMATION_TESTRUN_FAILED on non-zero exit', () => { - spawnStub.returns(makeSpawnResult('', 'compilation error', 1)); + spawnStub.callsFake(makeSpawnResult('', 'compilation error', 1)); const result = server.call('provar_automation_testrun', { flags: [] }); assert.ok(isError(result)); const body = parseBody(result); @@ -128,14 +154,14 @@ describe('automationTools', () => { }); it('uses stdout as message when stderr is empty', () => { - spawnStub.returns(makeSpawnResult('test failed: assertion error', '', 1)); + spawnStub.callsFake(makeSpawnResult('test failed: assertion error', '', 1)); const result = server.call('provar_automation_testrun', { flags: [] }); const body = parseBody(result); assert.equal(body.message, 'test failed: assertion error'); }); it('returns SF_NOT_FOUND on ENOENT with actionable message', () => { - spawnStub.returns(makeEnoentResult()); + spawnStub.callsFake(makeEnoentResult()); const result = server.call('provar_automation_testrun', { flags: [] }); assert.ok(isError(result)); const body = parseBody(result); @@ -143,13 +169,38 @@ describe('automationTools', () => { assert.ok((body.message as string).includes('npm install -g @salesforce/cli')); }); + it('translates a residual ENOBUFS into an actionable message with a suggestion', () => { + // runSfCommand now streams to disk so maxBuffer can no longer raise ENOBUFS, + // but if an OS-level ENOBUFS ever surfaces, handleSpawnError must make it + // actionable instead of echoing the opaque `spawnSync … ENOBUFS`. + const enobufs = Object.assign(new Error('spawnSync C:\\WINDOWS\\system32\\cmd.exe ENOBUFS'), { + code: 'ENOBUFS', + }); + spawnStub.callsFake(() => makeSpawnResultRaw(enobufs)); + const result = server.call('provar_automation_testrun', { flags: [] }); + assert.ok(isError(result)); + const body = parseBody(result); + assert.equal(body.error_code, 'ENOBUFS'); + assert.match(String(body.message), /written to disk|testOutputLevel/); + assert.ok((body.details as { suggestion?: string }).suggestion, 'should include an actionable suggestion'); + }); + + it('does not rewrite a non-ENOBUFS spawn error', () => { + const generic = Object.assign(new Error('something else broke'), { code: 'EPIPE' }); + spawnStub.callsFake(() => makeSpawnResultRaw(generic)); + const result = server.call('provar_automation_testrun', { flags: [] }); + const body = parseBody(result); + assert.equal(body.error_code, 'EPIPE'); + assert.equal(body.message, 'something else broke'); + }); + it('strips schema-validator noise from stdout and sets output_lines_suppressed', () => { const noisy = [ 'com.networknt.schema.validator.SchemaLoader - loading schema', 'INFO Starting test run', 'Tests: 5 passed, 0 failed', ].join('\n'); - spawnStub.returns(makeSpawnResult(noisy, '', 0)); + spawnStub.callsFake(makeSpawnResult(noisy, '', 0)); const result = server.call('provar_automation_testrun', { flags: [] }); const body = parseBody(result); assert.ok(!(body.stdout as string).includes('networknt'), 'Filtered stdout should not contain schema noise'); @@ -158,7 +209,7 @@ describe('automationTools', () => { }); it('does not set output_lines_suppressed when stdout has no noise', () => { - spawnStub.returns(makeSpawnResult('Tests: 3 passed, 0 failed', '', 0)); + spawnStub.callsFake(makeSpawnResult('Tests: 3 passed, 0 failed', '', 0)); const result = server.call('provar_automation_testrun', { flags: [] }); const body = parseBody(result); assert.equal(body.output_lines_suppressed, undefined, 'output_lines_suppressed should be absent'); @@ -180,7 +231,7 @@ describe('automationTools', () => { setSfResultsPathForTesting(tmpDir); try { - spawnStub.returns(makeSpawnResult('tests done', '', 0)); + spawnStub.callsFake(makeSpawnResult('tests done', '', 0)); const result = server.call('provar_automation_testrun', { flags: [] }); const body = parseBody(result); assert.ok(Array.isArray(body.steps), 'steps should be an array'); @@ -207,7 +258,7 @@ describe('automationTools', () => { setSfResultsPathForTesting(tmpDir); try { - spawnStub.returns(makeSpawnResult('tests done', '', 0)); + spawnStub.callsFake(makeSpawnResult('tests done', '', 0)); const result = server.call('provar_automation_testrun', { flags: [] }); const body = parseBody(result); assert.equal(body.steps, undefined, 'steps should be absent when no XML found'); @@ -233,7 +284,7 @@ describe('automationTools', () => { setSfResultsPathForTesting(tmpDir); try { - spawnStub.returns(makeSpawnResult('tests done', '', 0)); + spawnStub.callsFake(makeSpawnResult('tests done', '', 0)); const result = server.call('provar_automation_testrun', { flags: [] }); const body = parseBody(result); assert.equal(body.steps, undefined, 'steps should be absent when XML is malformed'); @@ -266,7 +317,7 @@ describe('automationTools', () => { setSfResultsPathForTesting(tmpDir); try { - spawnStub.returns(makeSpawnResult('tests done', '', 0)); + spawnStub.callsFake(makeSpawnResult('tests done', '', 0)); const result = server.call('provar_automation_testrun', { flags: [] }); assert.ok(!isError(result), 'tool should still report success — RUN-001 is additive'); const body = parseBody(result); @@ -293,7 +344,7 @@ describe('automationTools', () => { setSfResultsPathForTesting(tmpDir); try { - spawnStub.returns(makeSpawnResult('tests done', '', 0)); + spawnStub.callsFake(makeSpawnResult('tests done', '', 0)); const result = server.call('provar_automation_testrun', { flags: [] }); const body = parseBody(result); assert.equal(body.exitCode, 0); @@ -315,7 +366,7 @@ describe('automationTools', () => { setSfResultsPathForTesting(tmpDir); try { - spawnStub.returns(makeSpawnResult('', 'compilation error', 1)); + spawnStub.callsFake(makeSpawnResult('', 'compilation error', 1)); const result = server.call('provar_automation_testrun', { flags: [] }); assert.ok(isError(result), 'should report error on non-zero exit'); const body = parseBody(result); @@ -338,7 +389,7 @@ describe('automationTools', () => { setSfResultsPathForTesting(null); try { - spawnStub.returns(makeSpawnResult('tests done', '', 0)); + spawnStub.callsFake(makeSpawnResult('tests done', '', 0)); const result = server.call('provar_automation_testrun', { flags: [] }); const body = parseBody(result); assert.equal(body.exitCode, 0); @@ -354,7 +405,7 @@ describe('automationTools', () => { describe('provar_automation_compile', () => { it('calls sf with project compile args', () => { - spawnStub.returns(makeSpawnResult('compiled ok', '', 0)); + spawnStub.callsFake(makeSpawnResult('compiled ok', '', 0)); const result = server.call('provar_automation_compile', { flags: [] }); const body = parseBody(result); assert.equal(body.exitCode, 0); @@ -363,21 +414,21 @@ describe('automationTools', () => { }); it('forwards project-path flag', () => { - spawnStub.returns(makeSpawnResult('ok', '', 0)); + spawnStub.callsFake(makeSpawnResult('ok', '', 0)); server.call('provar_automation_compile', { flags: ['--project-path', '/my/project'] }); const args = spawnStub.firstCall.args[1] as string[]; assert.ok(args.includes('/my/project')); }); it('returns AUTOMATION_COMPILE_FAILED on non-zero exit', () => { - spawnStub.returns(makeSpawnResult('', 'syntax error in TestCase.testcase', 1)); + spawnStub.callsFake(makeSpawnResult('', 'syntax error in TestCase.testcase', 1)); const result = server.call('provar_automation_compile', { flags: [] }); assert.ok(isError(result)); assert.equal(parseBody(result).error_code, 'AUTOMATION_COMPILE_FAILED'); }); it('returns SF_NOT_FOUND on ENOENT', () => { - spawnStub.returns(makeEnoentResult()); + spawnStub.callsFake(makeEnoentResult()); const result = server.call('provar_automation_compile', { flags: [] }); assert.equal(parseBody(result).error_code, 'SF_NOT_FOUND'); }); @@ -403,6 +454,13 @@ describe('automationTools', () => { existsStub = sinon.stub(fs, 'existsSync').returns(false); readdirStub = sinon.stub(fs, 'readdirSync').throws(Object.assign(new Error('ENOENT'), { code: 'ENOENT' })); readFileStub = sinon.stub(fs, 'readFileSync').throws(Object.assign(new Error('ENOENT'), { code: 'ENOENT' })); + // runSfCommand captures child stdout/stderr to its own temp files (provar-sf-*) + // and reads them back with fs.readFileSync. Let those reads through so the + // ENOENT default above only models the absent Provar install files, not the + // capture buffer that the streaming fix relies on. + readFileStub + .withArgs(sinon.match((p: unknown) => typeof p === 'string' && p.includes('provar-sf-'))) + .callThrough(); sinon.stub(process, 'cwd').returns(fakeCwd); }); @@ -487,7 +545,7 @@ describe('automationTools', () => { it('force: true downloads even when a local install is already present', () => { makeValidInstall(localProvarHome); - spawnStub.returns(makeSpawnResult('setup complete', '', 0)); + spawnStub.callsFake(makeSpawnResult('setup complete', '', 0)); const body = parseBody(server.call('provar_automation_setup', { force: true })); @@ -508,9 +566,9 @@ describe('automationTools', () => { } return false; }); - spawnStub.callsFake(() => { + spawnStub.callsFake((...callArgs: unknown[]) => { installExists = true; // "download" creates the directory - return makeSpawnResult('Provar downloaded successfully', '', 0); + return makeSpawnResult('Provar downloaded successfully', '', 0)(...callArgs); }); const result = server.call('provar_automation_setup', {}); @@ -524,7 +582,7 @@ describe('automationTools', () => { }); it('forwards --version flag to sf when version is specified', () => { - spawnStub.returns(makeSpawnResult('ok', '', 0)); + spawnStub.callsFake(makeSpawnResult('ok', '', 0)); server.call('provar_automation_setup', { version: '2.10.0' }); @@ -534,7 +592,7 @@ describe('automationTools', () => { }); it('does not forward --version flag when version is omitted', () => { - spawnStub.returns(makeSpawnResult('ok', '', 0)); + spawnStub.callsFake(makeSpawnResult('ok', '', 0)); server.call('provar_automation_setup', {}); @@ -543,7 +601,7 @@ describe('automationTools', () => { }); it('returns AUTOMATION_SETUP_FAILED when sf exits non-zero', () => { - spawnStub.returns(makeSpawnResult('', 'Provided version is not a valid version.', 1)); + spawnStub.callsFake(makeSpawnResult('', 'Provided version is not a valid version.', 1)); const result = server.call('provar_automation_setup', { version: '0.0.0' }); @@ -553,7 +611,7 @@ describe('automationTools', () => { }); it('uses stdout as error message when stderr is empty', () => { - spawnStub.returns(makeSpawnResult('Network timeout', '', 1)); + spawnStub.callsFake(makeSpawnResult('Network timeout', '', 1)); const body = parseBody(server.call('provar_automation_setup', {})); @@ -561,7 +619,7 @@ describe('automationTools', () => { }); it('returns SF_NOT_FOUND when sf CLI is not installed', () => { - spawnStub.returns(makeEnoentResult()); + spawnStub.callsFake(makeEnoentResult()); const result = server.call('provar_automation_setup', {}); @@ -622,7 +680,7 @@ describe('automationTools', () => { describe('provar_automation_metadata_download', () => { it('calls sf with metadata download args', () => { - spawnStub.returns(makeSpawnResult('downloaded', '', 0)); + spawnStub.callsFake(makeSpawnResult('downloaded', '', 0)); const result = server.call('provar_automation_metadata_download', { flags: [] }); const body = parseBody(result); assert.equal(body.exitCode, 0); @@ -631,7 +689,7 @@ describe('automationTools', () => { }); it('forwards --target-org flag', () => { - spawnStub.returns(makeSpawnResult('ok', '', 0)); + spawnStub.callsFake(makeSpawnResult('ok', '', 0)); server.call('provar_automation_metadata_download', { flags: ['--target-org', 'myorg'] }); const args = spawnStub.firstCall.args[1] as string[]; assert.ok(args.includes('--target-org')); @@ -639,20 +697,20 @@ describe('automationTools', () => { }); it('returns AUTOMATION_METADATA_FAILED on non-zero exit', () => { - spawnStub.returns(makeSpawnResult('', 'auth failed', 1)); + spawnStub.callsFake(makeSpawnResult('', 'auth failed', 1)); const result = server.call('provar_automation_metadata_download', { flags: [] }); assert.ok(isError(result)); assert.equal(parseBody(result).error_code, 'AUTOMATION_METADATA_FAILED'); }); it('returns SF_NOT_FOUND on ENOENT', () => { - spawnStub.returns(makeEnoentResult()); + spawnStub.callsFake(makeEnoentResult()); const result = server.call('provar_automation_metadata_download', { flags: [] }); assert.equal(parseBody(result).error_code, 'SF_NOT_FOUND'); }); it('includes suggestion in details when [DOWNLOAD_ERROR] is in the message', () => { - spawnStub.returns(makeSpawnResult('', 'Error (1): [DOWNLOAD_ERROR] ERROR\n', 1)); + spawnStub.callsFake(makeSpawnResult('', 'Error (1): [DOWNLOAD_ERROR] ERROR\n', 1)); const result = server.call('provar_automation_metadata_download', { flags: ['-c', 'MyOrg'] }); assert.ok(isError(result)); const body = parseBody(result); @@ -664,7 +722,7 @@ describe('automationTools', () => { }); it('does NOT include suggestion for other failure messages', () => { - spawnStub.returns(makeSpawnResult('', 'Error (2): Nonexistent flag: --properties-file\n', 1)); + spawnStub.callsFake(makeSpawnResult('', 'Error (2): Nonexistent flag: --properties-file\n', 1)); const result = server.call('provar_automation_metadata_download', { flags: [] }); assert.ok(isError(result)); const body = parseBody(result); @@ -679,7 +737,7 @@ describe('automationTools', () => { describe('provar_automation_config_load', () => { it('calls sf with config load args and the given properties_path', () => { - spawnStub.returns(makeSpawnResult('', '', 0)); + spawnStub.callsFake(makeSpawnResult('', '', 0)); server.call('provar_automation_config_load', { properties_path: '/my/project/provardx-properties.json' }); const [cmd, args] = spawnStub.firstCall.args as [string, string[]]; assert.equal(cmd, 'sf'); @@ -694,7 +752,7 @@ describe('automationTools', () => { }); it('returns properties_path in the response', () => { - spawnStub.returns(makeSpawnResult('Config loaded', '', 0)); + spawnStub.callsFake(makeSpawnResult('Config loaded', '', 0)); const result = server.call('provar_automation_config_load', { properties_path: '/my/project/provardx-properties.json', }); @@ -704,14 +762,14 @@ describe('automationTools', () => { }); it('returns AUTOMATION_CONFIG_LOAD_FAILED on non-zero exit', () => { - spawnStub.returns(makeSpawnResult('', 'INVALID_PATH', 1)); + spawnStub.callsFake(makeSpawnResult('', 'INVALID_PATH', 1)); const result = server.call('provar_automation_config_load', { properties_path: '/missing.json' }); assert.ok(isError(result)); assert.equal(parseBody(result).error_code, 'AUTOMATION_CONFIG_LOAD_FAILED'); }); it('returns SF_NOT_FOUND on ENOENT', () => { - spawnStub.returns(makeEnoentResult()); + spawnStub.callsFake(makeEnoentResult()); const result = server.call('provar_automation_config_load', { properties_path: '/my/project/provardx-properties.json', }); @@ -719,7 +777,7 @@ describe('automationTools', () => { }); it('uses the explicit sf_path when provided', () => { - spawnStub.returns(makeSpawnResult('', '', 0)); + spawnStub.callsFake(makeSpawnResult('', '', 0)); server.call('provar_automation_config_load', { properties_path: '/proj/props.json', sf_path: '/custom/bin/sf' }); const [cmd] = spawnStub.firstCall.args as [string, string[]]; assert.equal(cmd, '/custom/bin/sf'); @@ -759,7 +817,7 @@ describe('automationTools', () => { }); it('allows properties_path within allowed paths', () => { - spawnStub.returns(makeSpawnResult('', '', 0)); + spawnStub.callsFake(makeSpawnResult('', '', 0)); const allowed = path.join(allowedDir, 'provardx-properties.json'); const result = restrictedServer.call('provar_automation_config_load', { properties_path: allowed, @@ -813,7 +871,7 @@ describe('automationTools', () => { }); it('passes shell: true when sf_path is a .cmd file', () => { - spawnStub.returns(makeSpawnResult('ok', '', 0)); + spawnStub.callsFake(makeSpawnResult('ok', '', 0)); server.call('provar_automation_compile', { flags: [], sf_path: 'C:\\npm\\sf.cmd' }); const opts = spawnStub.firstCall.args[2] as { shell: boolean }; assert.ok(opts.shell === true, 'shell should be true for a .cmd executable'); @@ -822,21 +880,21 @@ describe('automationTools', () => { it('passes shell: false when sf_path is a .exe binary', () => { // .exe has an explicit extension that is neither .cmd nor .bat, so no // shell is required even on Windows. - spawnStub.returns(makeSpawnResult('ok', '', 0)); + spawnStub.callsFake(makeSpawnResult('ok', '', 0)); server.call('provar_automation_compile', { flags: [], sf_path: 'C:\\Program Files\\sf.exe' }); const opts = spawnStub.firstCall.args[2] as { shell: boolean }; assert.ok(opts.shell === false, 'shell should be false for a .exe executable'); }); it('passes shell: false for a .js node script path', () => { - spawnStub.returns(makeSpawnResult('ok', '', 0)); + spawnStub.callsFake(makeSpawnResult('ok', '', 0)); server.call('provar_automation_testrun', { flags: [], sf_path: 'C:\\npm\\sf.js' }); const opts = spawnStub.firstCall.args[2] as { shell: boolean }; assert.ok(opts.shell === false, 'shell should be false for a .js script'); }); it('passes shell: true when sf_path is a .bat file', () => { - spawnStub.returns(makeSpawnResult('ok', '', 0)); + spawnStub.callsFake(makeSpawnResult('ok', '', 0)); server.call('provar_automation_testrun', { flags: [], sf_path: 'C:\\tools\\sf.bat' }); const opts = spawnStub.firstCall.args[2] as { shell: boolean }; assert.ok(opts.shell === true, 'shell should be true for a .bat executable'); @@ -858,8 +916,8 @@ describe('automationTools', () => { }); it('phase 1 uses shell:false for the --version probe', () => { - spawnStub.onFirstCall().returns(makeSpawnResult('sf/2.0.0 linux-x64 node-v18', '', 0)); // probe - spawnStub.onSecondCall().returns(makeSpawnResult('testrun ok', '', 0)); // actual command + spawnStub.onFirstCall().callsFake(makeSpawnResult('sf/2.0.0 linux-x64 node-v18', '', 0)); // probe + spawnStub.onSecondCall().callsFake(makeSpawnResult('testrun ok', '', 0)); // actual command server.call('provar_automation_testrun', { flags: [] }); const probeArgs = spawnStub.firstCall.args as [string, string[], { shell: boolean }]; assert.deepEqual(probeArgs[1], ['--version']); @@ -867,8 +925,8 @@ describe('automationTools', () => { }); it('phase 1 success — does not attempt phase 2', () => { - spawnStub.onFirstCall().returns(makeSpawnResult('sf/2.0.0', '', 0)); // probe succeeds - spawnStub.onSecondCall().returns(makeSpawnResult('ok', '', 0)); // actual command + spawnStub.onFirstCall().callsFake(makeSpawnResult('sf/2.0.0', '', 0)); // probe succeeds + spawnStub.onSecondCall().callsFake(makeSpawnResult('ok', '', 0)); // actual command server.call('provar_automation_testrun', { flags: [] }); // Exactly 2 spawns: phase 1 probe + actual command; no phase 2 retry const versionProbes = Array.from({ length: spawnStub.callCount }, (_, i) => spawnStub.getCall(i)).filter((c) => @@ -879,9 +937,9 @@ describe('automationTools', () => { it('win32 ENOENT on phase 1 triggers a shell:true phase 2 probe', () => { setSfPlatformForTesting('win32'); - spawnStub.onFirstCall().returns(makeEnoentResult()); // phase 1 ENOENT - spawnStub.onSecondCall().returns(makeSpawnResult('sf/2.0.0 win32', '', 0)); // phase 2 success - spawnStub.onThirdCall().returns(makeSpawnResult('testrun ok', '', 0)); // actual command + spawnStub.onFirstCall().callsFake(makeEnoentResult()); // phase 1 ENOENT + spawnStub.onSecondCall().callsFake(makeSpawnResult('sf/2.0.0 win32', '', 0)); // phase 2 success + spawnStub.onThirdCall().callsFake(makeSpawnResult('testrun ok', '', 0)); // actual command server.call('provar_automation_testrun', { flags: [] }); assert.ok(spawnStub.callCount >= 2, 'phase 2 probe should have been called'); const phase2Args = spawnStub.secondCall.args as [string, string[], { shell: boolean }]; @@ -891,9 +949,9 @@ describe('automationTools', () => { it('non-win32 ENOENT on phase 1 does not trigger a phase 2 probe', () => { setSfPlatformForTesting('linux'); - spawnStub.onFirstCall().returns(makeEnoentResult()); // probe ENOENT + spawnStub.onFirstCall().callsFake(makeEnoentResult()); // probe ENOENT // Safe default for any subsequent calls (e.g. if sf is found via common-path fallback) - spawnStub.returns(makeSpawnResult('ok', '', 0)); + spawnStub.callsFake(makeSpawnResult('ok', '', 0)); server.call('provar_automation_testrun', { flags: [] }); // Only one --version probe; no shell:true retry on non-win32 const versionProbes = Array.from({ length: spawnStub.callCount }, (_, i) => spawnStub.getCall(i)).filter((c) => @@ -930,14 +988,14 @@ describe('automationTools', () => { }); it('accepts a clean Windows .cmd path', () => { - spawnStub.returns(makeSpawnResult('ok', '', 0)); + spawnStub.callsFake(makeSpawnResult('ok', '', 0)); const result = server.call('provar_automation_testrun', { flags: [], sf_path: 'C:\\npm\\sf.cmd' }); assert.ok(!isError(result)); }); it('does not check path safety on non-Windows (shell:false, no injection risk)', () => { setSfPlatformForTesting('linux'); - spawnStub.returns(makeSpawnResult('ok', '', 0)); + spawnStub.callsFake(makeSpawnResult('ok', '', 0)); // On linux needsWindowsShell returns false → assertShellSafePath is never called const result = server.call('provar_automation_testrun', { flags: [], sf_path: '/usr/bin/sf' }); assert.ok(!isError(result)); @@ -948,19 +1006,19 @@ describe('automationTools', () => { describe('sf_path explicit executable', () => { it('testrun uses sf_path as the executable', () => { - spawnStub.returns(makeSpawnResult('ok', '', 0)); + spawnStub.callsFake(makeSpawnResult('ok', '', 0)); server.call('provar_automation_testrun', { flags: [], sf_path: '/opt/sf/bin/sf' }); assert.equal(spawnStub.firstCall.args[0], '/opt/sf/bin/sf'); }); it('compile uses sf_path as the executable', () => { - spawnStub.returns(makeSpawnResult('ok', '', 0)); + spawnStub.callsFake(makeSpawnResult('ok', '', 0)); server.call('provar_automation_compile', { flags: [], sf_path: '/opt/sf/bin/sf' }); assert.equal(spawnStub.firstCall.args[0], '/opt/sf/bin/sf'); }); it('SF_NOT_FOUND message names the explicit path when sf_path is provided and missing', () => { - spawnStub.returns(makeEnoentResult()); + spawnStub.callsFake(makeEnoentResult()); const result = server.call('provar_automation_compile', { flags: [], sf_path: '/missing/sf' }); const body = parseBody(result); assert.equal(body.error_code, 'SF_NOT_FOUND'); @@ -968,7 +1026,7 @@ describe('automationTools', () => { }); it('SF_NOT_FOUND message mentions sf_path option when no explicit path was given', () => { - spawnStub.returns(makeEnoentResult()); + spawnStub.callsFake(makeEnoentResult()); const result = server.call('provar_automation_testrun', { flags: [] }); const body = parseBody(result); assert.equal(body.error_code, 'SF_NOT_FOUND'); @@ -1066,7 +1124,7 @@ describe('filterTestRunOutput', () => { }); it('config_load surfaces PROVAR_PLUGIN_NOT_FOUND with remediation when the provar topic is missing', () => { - spawnStub.returns(makeSpawnResult('', 'Warning: provar is not a sf command.\nDid you mean a typo?', 1)); + spawnStub.callsFake(makeSpawnResult('', 'Warning: provar is not a sf command.\nDid you mean a typo?', 1)); const result = server.call('provar_automation_config_load', { properties_path: '/proj/provardx-properties.json', }); @@ -1080,7 +1138,7 @@ describe('filterTestRunOutput', () => { }); it('detects the "command provar not found" phrasing', () => { - spawnStub.returns(makeSpawnResult('', 'command provar not found', 1)); + spawnStub.callsFake(makeSpawnResult('', 'command provar not found', 1)); const result = server.call('provar_automation_config_load', { properties_path: '/proj/provardx-properties.json', }); @@ -1088,7 +1146,7 @@ describe('filterTestRunOutput', () => { }); it('does NOT fire for a generic config-load failure (stays AUTOMATION_CONFIG_LOAD_FAILED)', () => { - spawnStub.returns(makeSpawnResult('', 'Error: properties file not found at /proj/x.json', 1)); + spawnStub.callsFake(makeSpawnResult('', 'Error: properties file not found at /proj/x.json', 1)); const result = server.call('provar_automation_config_load', { properties_path: '/proj/provardx-properties.json', }); @@ -1096,13 +1154,13 @@ describe('filterTestRunOutput', () => { }); it('maps a missing provar topic on testrun too', () => { - spawnStub.returns(makeSpawnResult('', 'command provar:automation:test:run not found', 1)); + spawnStub.callsFake(makeSpawnResult('', 'command provar:automation:test:run not found', 1)); const result = server.call('provar_automation_testrun', { flags: [] }); assert.equal(parseBody(result).error_code, 'PROVAR_PLUGIN_NOT_FOUND'); }); it('does not fire on a successful command (exit 0)', () => { - spawnStub.returns(makeSpawnResult('properties file loaded successfully', '', 0)); + spawnStub.callsFake(makeSpawnResult('properties file loaded successfully', '', 0)); const result = server.call('provar_automation_config_load', { properties_path: '/proj/provardx-properties.json', }); diff --git a/test/unit/mcp/defectTools.test.ts b/test/unit/mcp/defectTools.test.ts index f99ae6d..6eb7c6d 100644 --- a/test/unit/mcp/defectTools.test.ts +++ b/test/unit/mcp/defectTools.test.ts @@ -7,6 +7,7 @@ /* eslint-disable camelcase */ import { strict as assert } from 'node:assert'; +import fs from 'node:fs'; import sinon from 'sinon'; import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js'; import { sfSpawnHelper } from '../../../src/mcp/tools/sfSpawn.js'; @@ -46,13 +47,40 @@ type SpawnResult = { signal: NodeJS.Signals | null; }; -function makeSpawnResult(stdout: string, status = 0): SpawnResult { - return { stdout, stderr: '', status, error: undefined, pid: 1, output: [null, stdout, ''], signal: null }; +type SpawnFake = (...callArgs: unknown[]) => SpawnResult; + +// runSfCommand streams the child's stdout/stderr to temp files rather than +// buffering them in memory, so the stub for sfSpawnHelper.spawnSync mimics a real +// child: it writes the captured output to the file descriptors passed via +// `stdio: ['ignore', outFd, errFd]`, then returns the spawn result. +function spawnFake(result: SpawnResult): SpawnFake { + return (...callArgs) => { + const stdio = (callArgs[2] as { stdio?: unknown[] } | undefined)?.stdio; + const outFd = stdio?.[1]; + const errFd = stdio?.[2]; + if (typeof outFd === 'number' && typeof errFd === 'number') { + fs.writeSync(outFd, result.stdout); + fs.writeSync(errFd, result.stderr); + } + return result; + }; +} + +function makeSpawnResult(stdout: string, status = 0): SpawnFake { + return spawnFake({ stdout, stderr: '', status, error: undefined, pid: 1, output: [null, stdout, ''], signal: null }); } -function makeEnoentResult(): SpawnResult { +function makeEnoentResult(): SpawnFake { const err = Object.assign(new Error('spawn sf ENOENT'), { code: 'ENOENT' }); - return { stdout: '', stderr: '', status: null, error: err, pid: undefined, output: [null, '', ''], signal: null }; + return spawnFake({ + stdout: '', + stderr: '', + status: null, + error: err, + pid: undefined, + output: [null, '', ''], + signal: null, + }); } function queryResult(records: object[], totalSize?: number): string { @@ -94,9 +122,9 @@ const ORG = 'my-qh-org'; function makeHappyPathStub(stub: sinon.SinonStub): void { // Call 0: job query - stub.onCall(0).returns(makeSpawnResult(queryResult([{ Id: JOB_ID }]))); + stub.onCall(0).callsFake(makeSpawnResult(queryResult([{ Id: JOB_ID }]))); // Call 1: cycle query - stub.onCall(1).returns( + stub.onCall(1).callsFake( makeSpawnResult( queryResult([ { @@ -109,7 +137,7 @@ function makeHappyPathStub(stub: sinon.SinonStub): void { ) ); // Call 2: failed executions query - stub.onCall(2).returns( + stub.onCall(2).callsFake( makeSpawnResult( queryResult([ { @@ -121,7 +149,7 @@ function makeHappyPathStub(stub: sinon.SinonStub): void { ) ); // Call 3: failed step query - stub.onCall(3).returns( + stub.onCall(3).callsFake( makeSpawnResult( queryResult([ { @@ -135,11 +163,11 @@ function makeHappyPathStub(stub: sinon.SinonStub): void { ) ); // Call 4: create Defect__c - stub.onCall(4).returns(makeSpawnResult(createResult(DEFECT_ID))); + stub.onCall(4).callsFake(makeSpawnResult(createResult(DEFECT_ID))); // Call 5: create Test_Case_Defect__c - stub.onCall(5).returns(makeSpawnResult(createResult(TC_DEFECT_ID))); + stub.onCall(5).callsFake(makeSpawnResult(createResult(TC_DEFECT_ID))); // Call 6: create Test_Execution_Defect__c - stub.onCall(6).returns(makeSpawnResult(createResult(EXEC_DEFECT_ID))); + stub.onCall(6).callsFake(makeSpawnResult(createResult(EXEC_DEFECT_ID))); } // ── Tests: createDefectsForRun (unit) ───────────────────────────────────────── @@ -178,8 +206,8 @@ describe('createDefectsForRun', () => { }); it('returns empty result when no failed executions exist', () => { - stub.onCall(0).returns(makeSpawnResult(queryResult([{ Id: JOB_ID }]))); - stub.onCall(1).returns( + stub.onCall(0).callsFake(makeSpawnResult(queryResult([{ Id: JOB_ID }]))); + stub.onCall(1).callsFake( makeSpawnResult( queryResult([ { @@ -191,7 +219,7 @@ describe('createDefectsForRun', () => { ]) ) ); - stub.onCall(2).returns(makeSpawnResult(queryResult([], 0))); + stub.onCall(2).callsFake(makeSpawnResult(queryResult([], 0))); const result = createDefectsForRun(RUN_ID, ORG); assert.equal(result.created.length, 0); @@ -200,7 +228,7 @@ describe('createDefectsForRun', () => { }); it('throws when job not found by tracking ID', () => { - stub.onCall(0).returns(makeSpawnResult(queryResult([], 0))); + stub.onCall(0).callsFake(makeSpawnResult(queryResult([], 0))); assert.throws( () => createDefectsForRun(RUN_ID, ORG), @@ -209,8 +237,8 @@ describe('createDefectsForRun', () => { }); it('throws when no Test_Cycle__c found for job', () => { - stub.onCall(0).returns(makeSpawnResult(queryResult([{ Id: JOB_ID }]))); - stub.onCall(1).returns(makeSpawnResult(queryResult([], 0))); + stub.onCall(0).callsFake(makeSpawnResult(queryResult([{ Id: JOB_ID }]))); + stub.onCall(1).callsFake(makeSpawnResult(queryResult([], 0))); assert.throws( () => createDefectsForRun(RUN_ID, ORG), @@ -221,8 +249,8 @@ describe('createDefectsForRun', () => { it('filters executions by failedTestFilter substring', () => { // Two failed executions; filter keeps only the one matching TC_ID const OTHER_TC = 'a0t000000000OTH'; - stub.onCall(0).returns(makeSpawnResult(queryResult([{ Id: JOB_ID }]))); - stub.onCall(1).returns( + stub.onCall(0).callsFake(makeSpawnResult(queryResult([{ Id: JOB_ID }]))); + stub.onCall(1).callsFake( makeSpawnResult( queryResult([ { @@ -234,7 +262,7 @@ describe('createDefectsForRun', () => { ]) ) ); - stub.onCall(2).returns( + stub.onCall(2).callsFake( makeSpawnResult( queryResult([ { Id: EXEC_ID, provar__Test_Case__c: TC_ID, provar__Tester__c: 'tester@example.com' }, @@ -243,7 +271,7 @@ describe('createDefectsForRun', () => { ) ); // step query for the one kept execution - stub.onCall(3).returns( + stub.onCall(3).callsFake( makeSpawnResult( queryResult([ { @@ -256,9 +284,9 @@ describe('createDefectsForRun', () => { ]) ) ); - stub.onCall(4).returns(makeSpawnResult(createResult(DEFECT_ID))); - stub.onCall(5).returns(makeSpawnResult(createResult(TC_DEFECT_ID))); - stub.onCall(6).returns(makeSpawnResult(createResult(EXEC_DEFECT_ID))); + stub.onCall(4).callsFake(makeSpawnResult(createResult(DEFECT_ID))); + stub.onCall(5).callsFake(makeSpawnResult(createResult(TC_DEFECT_ID))); + stub.onCall(6).callsFake(makeSpawnResult(createResult(EXEC_DEFECT_ID))); const result = createDefectsForRun(RUN_ID, ORG, [TC_ID]); assert.equal(result.created.length, 1); @@ -267,8 +295,8 @@ describe('createDefectsForRun', () => { }); it('handles missing step gracefully (no step found for execution)', () => { - stub.onCall(0).returns(makeSpawnResult(queryResult([{ Id: JOB_ID }]))); - stub.onCall(1).returns( + stub.onCall(0).callsFake(makeSpawnResult(queryResult([{ Id: JOB_ID }]))); + stub.onCall(1).callsFake( makeSpawnResult( queryResult([ { @@ -280,7 +308,7 @@ describe('createDefectsForRun', () => { ]) ) ); - stub.onCall(2).returns( + stub.onCall(2).callsFake( makeSpawnResult( queryResult([ { @@ -291,10 +319,10 @@ describe('createDefectsForRun', () => { ]) ) ); - stub.onCall(3).returns(makeSpawnResult(queryResult([], 0))); // no steps - stub.onCall(4).returns(makeSpawnResult(createResult(DEFECT_ID))); - stub.onCall(5).returns(makeSpawnResult(createResult(TC_DEFECT_ID))); - stub.onCall(6).returns(makeSpawnResult(createResult(EXEC_DEFECT_ID))); + stub.onCall(3).callsFake(makeSpawnResult(queryResult([], 0))); // no steps + stub.onCall(4).callsFake(makeSpawnResult(createResult(DEFECT_ID))); + stub.onCall(5).callsFake(makeSpawnResult(createResult(TC_DEFECT_ID))); + stub.onCall(6).callsFake(makeSpawnResult(createResult(EXEC_DEFECT_ID))); const result = createDefectsForRun(RUN_ID, ORG); assert.equal(result.created.length, 1); @@ -302,7 +330,7 @@ describe('createDefectsForRun', () => { }); it('throws SfNotFoundError (SF_NOT_FOUND) when sf is not in PATH', () => { - stub.returns(makeEnoentResult()); + stub.callsFake(makeEnoentResult()); assert.throws( () => createDefectsForRun(RUN_ID, ORG), @@ -352,7 +380,7 @@ describe('provar_qualityhub_defect_create (MCP tool)', () => { }); it('returns isError with DEFECT_CREATE_FAILED on job-not-found error', () => { - stub.onCall(0).returns(makeSpawnResult(queryResult([], 0))); + stub.onCall(0).callsFake(makeSpawnResult(queryResult([], 0))); const result = server.call('provar_qualityhub_defect_create', { run_id: RUN_ID, target_org: ORG, @@ -363,7 +391,7 @@ describe('provar_qualityhub_defect_create (MCP tool)', () => { }); it('returns isError with SF_NOT_FOUND when sf CLI is missing', () => { - stub.returns(makeEnoentResult()); + stub.callsFake(makeEnoentResult()); const result = server.call('provar_qualityhub_defect_create', { run_id: RUN_ID, target_org: ORG, @@ -375,8 +403,8 @@ describe('provar_qualityhub_defect_create (MCP tool)', () => { it('passes failed_tests filter through to createDefectsForRun', () => { // Return empty executions - filtered out - stub.onCall(0).returns(makeSpawnResult(queryResult([{ Id: JOB_ID }]))); - stub.onCall(1).returns( + stub.onCall(0).callsFake(makeSpawnResult(queryResult([{ Id: JOB_ID }]))); + stub.onCall(1).callsFake( makeSpawnResult( queryResult([ { @@ -388,7 +416,7 @@ describe('provar_qualityhub_defect_create (MCP tool)', () => { ]) ) ); - stub.onCall(2).returns( + stub.onCall(2).callsFake( makeSpawnResult( queryResult([ { diff --git a/test/unit/mcp/qualityHubTools.test.ts b/test/unit/mcp/qualityHubTools.test.ts index edbe480..f5ac623 100644 --- a/test/unit/mcp/qualityHubTools.test.ts +++ b/test/unit/mcp/qualityHubTools.test.ts @@ -7,6 +7,7 @@ /* eslint-disable camelcase */ import { strict as assert } from 'node:assert'; +import fs from 'node:fs'; import path from 'node:path'; import sinon from 'sinon'; import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js'; @@ -36,25 +37,42 @@ class MockMcpServer { // ── Helpers ─────────────────────────────────────────────────────────────────── -function makeSpawnResult( - stdout: string, - stderr: string, - status: number -): { stdout: string; stderr: string; status: number; error: undefined; pid: number; output: never[]; signal: null } { - return { stdout, stderr, status, error: undefined, pid: 1, output: [], signal: null }; -} - -function makeEnoentResult(): { +// runSfCommand streams the child's stdout/stderr to temp files rather than +// buffering them in memory, so the stub for sfSpawnHelper.spawnSync mimics a real +// child: it writes the captured output to the file descriptors passed via +// `stdio: ['ignore', outFd, errFd]`, then returns the spawn result. The in-memory +// probe path (encoding/maxBuffer, no fd stdio) just receives the object. +type FakeSpawnResult = { stdout: string; stderr: string; - status: null; - error: Error & { code: string }; - pid: undefined; + status: number | null; + error: (Error & { code?: string }) | undefined; + pid: number | undefined; output: never[]; signal: null; -} { +}; +type SpawnFake = (...callArgs: unknown[]) => FakeSpawnResult; + +function spawnFake(result: FakeSpawnResult): SpawnFake { + return (...callArgs) => { + const stdio = (callArgs[2] as { stdio?: unknown[] } | undefined)?.stdio; + const outFd = stdio?.[1]; + const errFd = stdio?.[2]; + if (typeof outFd === 'number' && typeof errFd === 'number') { + fs.writeSync(outFd, result.stdout); + fs.writeSync(errFd, result.stderr); + } + return result; + }; +} + +function makeSpawnResult(stdout: string, stderr: string, status: number): SpawnFake { + return spawnFake({ stdout, stderr, status, error: undefined, pid: 1, output: [], signal: null }); +} + +function makeEnoentResult(): SpawnFake { const err = Object.assign(new Error('spawn sf ENOENT'), { code: 'ENOENT' }); - return { stdout: '', stderr: '', status: null, error: err, pid: undefined, output: [], signal: null }; + return spawnFake({ stdout: '', stderr: '', status: null, error: err, pid: undefined, output: [], signal: null }); } function parseBody(result: unknown): Record { @@ -91,7 +109,7 @@ describe('qualityHubTools', () => { describe('provar_qualityhub_connect', () => { it('passes correct args to sf and returns stdout on success', () => { - spawnStub.returns(makeSpawnResult('{"status":0}', '', 0)); + spawnStub.callsFake(makeSpawnResult('{"status":0}', '', 0)); const result = server.call('provar_qualityhub_connect', { target_org: 'myorg', flags: [] }); const body = parseBody(result); @@ -105,14 +123,14 @@ describe('qualityHubTools', () => { }); it('forwards extra flags', () => { - spawnStub.returns(makeSpawnResult('ok', '', 0)); + spawnStub.callsFake(makeSpawnResult('ok', '', 0)); server.call('provar_qualityhub_connect', { target_org: 'myorg', flags: ['--json'] }); const args = spawnStub.firstCall.args[1] as string[]; assert.ok(args.includes('--json')); }); it('returns isError when exit code is non-zero', () => { - spawnStub.returns(makeSpawnResult('', 'bad credentials', 1)); + spawnStub.callsFake(makeSpawnResult('', 'bad credentials', 1)); const result = server.call('provar_qualityhub_connect', { target_org: 'myorg', flags: [] }); assert.ok(isError(result)); const body = parseBody(result); @@ -120,7 +138,7 @@ describe('qualityHubTools', () => { }); it('returns SF_NOT_FOUND when sf is not in PATH', () => { - spawnStub.returns(makeEnoentResult()); + spawnStub.callsFake(makeEnoentResult()); const result = server.call('provar_qualityhub_connect', { target_org: 'myorg', flags: [] }); assert.ok(isError(result)); const body = parseBody(result); @@ -133,7 +151,7 @@ describe('qualityHubTools', () => { describe('provar_qualityhub_display', () => { it('calls sf with display args on success', () => { - spawnStub.returns(makeSpawnResult('display output', '', 0)); + spawnStub.callsFake(makeSpawnResult('display output', '', 0)); const result = server.call('provar_qualityhub_display', { target_org: 'myorg', flags: [] }); const body = parseBody(result); assert.equal(body.exitCode, 0); @@ -143,20 +161,20 @@ describe('qualityHubTools', () => { }); it('omits --target-org when target_org not provided', () => { - spawnStub.returns(makeSpawnResult('ok', '', 0)); + spawnStub.callsFake(makeSpawnResult('ok', '', 0)); server.call('provar_qualityhub_display', { target_org: undefined, flags: [] }); const args = spawnStub.firstCall.args[1] as string[]; assert.ok(!args.includes('--target-org')); }); it('returns isError on non-zero exit', () => { - spawnStub.returns(makeSpawnResult('', 'error', 1)); + spawnStub.callsFake(makeSpawnResult('', 'error', 1)); const result = server.call('provar_qualityhub_display', { flags: [] }); assert.ok(isError(result)); }); it('returns SF_NOT_FOUND on ENOENT', () => { - spawnStub.returns(makeEnoentResult()); + spawnStub.callsFake(makeEnoentResult()); const result = server.call('provar_qualityhub_display', { flags: [] }); const body = parseBody(result); assert.equal(body.error_code, 'SF_NOT_FOUND'); @@ -167,7 +185,7 @@ describe('qualityHubTools', () => { describe('provar_qualityhub_display — detail param', () => { it('standard (default) returns requestId, exitCode, stdout, stderr', () => { - spawnStub.returns(makeSpawnResult('display output', '', 0)); + spawnStub.callsFake(makeSpawnResult('display output', '', 0)); const result = server.call('provar_qualityhub_display', { flags: [] }); const body = parseBody(result); assert.ok('requestId' in body); @@ -177,7 +195,7 @@ describe('qualityHubTools', () => { }); it('summary returns only requestId and exitCode', () => { - spawnStub.returns(makeSpawnResult('display output', '', 0)); + spawnStub.callsFake(makeSpawnResult('display output', '', 0)); const result = server.call('provar_qualityhub_display', { flags: [], detail: 'summary' }); const body = parseBody(result); assert.ok('requestId' in body, 'summary must include requestId'); @@ -187,7 +205,7 @@ describe('qualityHubTools', () => { }); it('full returns same fields as standard', () => { - spawnStub.returns(makeSpawnResult('display output', '', 0)); + spawnStub.callsFake(makeSpawnResult('display output', '', 0)); const full = parseBody(server.call('provar_qualityhub_display', { flags: [], detail: 'full' })); const std = parseBody(server.call('provar_qualityhub_display', { flags: [], detail: 'standard' })); assert.deepEqual(Object.keys(full).sort(), Object.keys(std).sort()); @@ -196,7 +214,7 @@ describe('qualityHubTools', () => { describe('provar_qualityhub_display — fields param', () => { it('retains only specified keys', () => { - spawnStub.returns(makeSpawnResult('display output', '', 0)); + spawnStub.callsFake(makeSpawnResult('display output', '', 0)); const result = server.call('provar_qualityhub_display', { flags: [], fields: 'exitCode,stdout' }); const body = parseBody(result); assert.ok('exitCode' in body); @@ -206,7 +224,7 @@ describe('qualityHubTools', () => { }); it('silently ignores unknown fields', () => { - spawnStub.returns(makeSpawnResult('ok', '', 0)); + spawnStub.callsFake(makeSpawnResult('ok', '', 0)); const result = server.call('provar_qualityhub_display', { flags: [], fields: 'exitCode,ghost' }); assert.equal(isError(result), false); const body = parseBody(result); @@ -219,7 +237,7 @@ describe('qualityHubTools', () => { describe('provar_qualityhub_testrun', () => { it('passes correct args and returns success', () => { - spawnStub.returns(makeSpawnResult('run started', '', 0)); + spawnStub.callsFake(makeSpawnResult('run started', '', 0)); const result = server.call('provar_qualityhub_testrun', { target_org: 'myorg', flags: [] }); const body = parseBody(result); assert.equal(body.exitCode, 0); @@ -228,20 +246,43 @@ describe('qualityHubTools', () => { }); it('returns QH_TESTRUN_FAILED on non-zero exit', () => { - spawnStub.returns(makeSpawnResult('', 'run failed', 1)); + spawnStub.callsFake(makeSpawnResult('', 'run failed', 1)); const result = server.call('provar_qualityhub_testrun', { target_org: 'myorg', flags: [] }); assert.ok(isError(result)); assert.equal(parseBody(result).error_code, 'QH_TESTRUN_FAILED'); }); it('returns SF_NOT_FOUND on ENOENT', () => { - spawnStub.returns(makeEnoentResult()); + spawnStub.callsFake(makeEnoentResult()); const result = server.call('provar_qualityhub_testrun', { target_org: 'myorg', flags: [] }); assert.equal(parseBody(result).error_code, 'SF_NOT_FOUND'); }); + it('translates a residual ENOBUFS into an actionable error with a suggestion', () => { + // Quality Hub shares the same runSfCommand + handleSpawnError as automation, + // so a residual ENOBUFS here must get the same actionable translation. + const enobufs = Object.assign(new Error('spawnSync C:\\WINDOWS\\system32\\cmd.exe ENOBUFS'), { + code: 'ENOBUFS', + }); + spawnStub.callsFake(() => ({ + stdout: '', + stderr: '', + status: null, + error: enobufs, + pid: undefined, + output: [], + signal: null, + })); + const result = server.call('provar_qualityhub_testrun', { target_org: 'myorg', flags: [] }); + assert.ok(isError(result)); + const body = parseBody(result); + assert.equal(body.error_code, 'ENOBUFS'); + assert.match(String(body.message), /written to disk|--json|verbosity/); + assert.ok((body.details as { suggestion?: string }).suggestion, 'should include an actionable suggestion'); + }); + it('adds wildcard warning when flags contain * glob pattern', () => { - spawnStub.returns(makeSpawnResult('run started', '', 0)); + spawnStub.callsFake(makeSpawnResult('run started', '', 0)); const result = server.call('provar_qualityhub_testrun', { target_org: 'myorg', flags: ['--plan-name', 'Suite/E2E*'], @@ -254,7 +295,7 @@ describe('qualityHubTools', () => { }); it('adds wildcard warning when flags contain ? pattern', () => { - spawnStub.returns(makeSpawnResult('run started', '', 0)); + spawnStub.callsFake(makeSpawnResult('run started', '', 0)); const result = server.call('provar_qualityhub_testrun', { target_org: 'myorg', flags: ['--plan-name', 'Suite?Test'], @@ -265,7 +306,7 @@ describe('qualityHubTools', () => { }); it('does not add warning for exact plan name flags', () => { - spawnStub.returns(makeSpawnResult('run started', '', 0)); + spawnStub.callsFake(makeSpawnResult('run started', '', 0)); const result = server.call('provar_qualityhub_testrun', { target_org: 'myorg', flags: ['--plan-name', 'SmokeTests'], @@ -279,7 +320,7 @@ describe('qualityHubTools', () => { describe('provar_qualityhub_testrun_report', () => { it('passes run_id in args', () => { - spawnStub.returns(makeSpawnResult('{"status":"running"}', '', 0)); + spawnStub.callsFake(makeSpawnResult('{"status":"running"}', '', 0)); server.call('provar_qualityhub_testrun_report', { target_org: 'myorg', run_id: 'abc-123', flags: [] }); const args = spawnStub.firstCall.args[1] as string[]; assert.ok(args.includes('--run-id')); @@ -287,7 +328,7 @@ describe('qualityHubTools', () => { }); it('returns QH_REPORT_FAILED on non-zero exit', () => { - spawnStub.returns(makeSpawnResult('', 'not found', 1)); + spawnStub.callsFake(makeSpawnResult('', 'not found', 1)); const result = server.call('provar_qualityhub_testrun_report', { target_org: 'myorg', run_id: 'abc-123', @@ -298,7 +339,7 @@ describe('qualityHubTools', () => { }); it('returns SF_NOT_FOUND on ENOENT', () => { - spawnStub.returns(makeEnoentResult()); + spawnStub.callsFake(makeEnoentResult()); const result = server.call('provar_qualityhub_testrun_report', { target_org: 'myorg', run_id: 'abc-123', @@ -309,7 +350,7 @@ describe('qualityHubTools', () => { describe('failure detection', () => { it('sets suggestion when JSON result.status is "FAILED"', () => { - spawnStub.returns(makeSpawnResult(JSON.stringify({ result: { status: 'FAILED' } }), '', 0)); + spawnStub.callsFake(makeSpawnResult(JSON.stringify({ result: { status: 'FAILED' } }), '', 0)); const result = server.call('provar_qualityhub_testrun_report', { target_org: 'myorg', run_id: 'abc-123', @@ -320,7 +361,7 @@ describe('qualityHubTools', () => { }); it('sets suggestion when JSON result.status is "FAIL"', () => { - spawnStub.returns(makeSpawnResult(JSON.stringify({ result: { status: 'FAIL' } }), '', 0)); + spawnStub.callsFake(makeSpawnResult(JSON.stringify({ result: { status: 'FAIL' } }), '', 0)); const result = server.call('provar_qualityhub_testrun_report', { target_org: 'myorg', run_id: 'abc-123', @@ -331,7 +372,7 @@ describe('qualityHubTools', () => { }); it('does NOT set suggestion when status is "RUNNING"', () => { - spawnStub.returns(makeSpawnResult(JSON.stringify({ result: { status: 'RUNNING' } }), '', 0)); + spawnStub.callsFake(makeSpawnResult(JSON.stringify({ result: { status: 'RUNNING' } }), '', 0)); const result = server.call('provar_qualityhub_testrun_report', { target_org: 'myorg', run_id: 'abc-123', @@ -342,7 +383,7 @@ describe('qualityHubTools', () => { }); it('does NOT set suggestion when status is "PASSED"', () => { - spawnStub.returns(makeSpawnResult(JSON.stringify({ result: { status: 'PASSED' } }), '', 0)); + spawnStub.callsFake(makeSpawnResult(JSON.stringify({ result: { status: 'PASSED' } }), '', 0)); const result = server.call('provar_qualityhub_testrun_report', { target_org: 'myorg', run_id: 'abc-123', @@ -355,7 +396,7 @@ describe('qualityHubTools', () => { it('does NOT false-positive on "failure" in plain text output (word in non-status context)', () => { // Before PR #110 the check was /fail/i which would match "failure" anywhere in output; // now it only matches the "status" field value. - spawnStub.returns( + spawnStub.callsFake( makeSpawnResult('{"message": "No failure detected in this output", "result": {"status": "PASSED"}}', '', 0) ); const result = server.call('provar_qualityhub_testrun_report', { @@ -369,7 +410,7 @@ describe('qualityHubTools', () => { it('falls back to regex extraction when stdout is not valid JSON', () => { // Non-JSON output with "status": "FAILED" substring - spawnStub.returns(makeSpawnResult('"status": "FAILED"', '', 0)); + spawnStub.callsFake(makeSpawnResult('"status": "FAILED"', '', 0)); const result = server.call('provar_qualityhub_testrun_report', { target_org: 'myorg', run_id: 'abc-123', @@ -388,7 +429,7 @@ describe('qualityHubTools', () => { describe('provar_qualityhub_testrun_abort', () => { it('passes run_id and abort subcommand', () => { - spawnStub.returns(makeSpawnResult('aborted', '', 0)); + spawnStub.callsFake(makeSpawnResult('aborted', '', 0)); server.call('provar_qualityhub_testrun_abort', { target_org: 'myorg', run_id: 'abc-123', flags: [] }); const args = spawnStub.firstCall.args[1] as string[]; assert.ok(args.includes('abort')); @@ -396,7 +437,7 @@ describe('qualityHubTools', () => { }); it('returns QH_ABORT_FAILED on non-zero exit', () => { - spawnStub.returns(makeSpawnResult('', 'abort failed', 1)); + spawnStub.callsFake(makeSpawnResult('', 'abort failed', 1)); const result = server.call('provar_qualityhub_testrun_abort', { target_org: 'myorg', run_id: 'abc-123', @@ -407,7 +448,7 @@ describe('qualityHubTools', () => { }); it('returns SF_NOT_FOUND on ENOENT', () => { - spawnStub.returns(makeEnoentResult()); + spawnStub.callsFake(makeEnoentResult()); const result = server.call('provar_qualityhub_testrun_abort', { target_org: 'myorg', run_id: 'abc-123', @@ -421,7 +462,7 @@ describe('qualityHubTools', () => { describe('provar_qualityhub_testcase_retrieve', () => { it('calls sf with testcase retrieve args', () => { - spawnStub.returns(makeSpawnResult('[]', '', 0)); + spawnStub.callsFake(makeSpawnResult('[]', '', 0)); const result = server.call('provar_qualityhub_testcase_retrieve', { target_org: 'myorg', flags: ['--user-story', 'US-1'], @@ -435,14 +476,14 @@ describe('qualityHubTools', () => { }); it('returns QH_RETRIEVE_FAILED on non-zero exit', () => { - spawnStub.returns(makeSpawnResult('', 'no records', 1)); + spawnStub.callsFake(makeSpawnResult('', 'no records', 1)); const result = server.call('provar_qualityhub_testcase_retrieve', { target_org: 'myorg', flags: [] }); assert.ok(isError(result)); assert.equal(parseBody(result).error_code, 'QH_RETRIEVE_FAILED'); }); it('returns SF_NOT_FOUND on ENOENT', () => { - spawnStub.returns(makeEnoentResult()); + spawnStub.callsFake(makeEnoentResult()); const result = server.call('provar_qualityhub_testcase_retrieve', { target_org: 'myorg', flags: [] }); assert.equal(parseBody(result).error_code, 'SF_NOT_FOUND'); }); @@ -452,7 +493,7 @@ describe('qualityHubTools', () => { describe('provar_qualityhub_testcase_retrieve — detail param', () => { it('standard (default) returns requestId, exitCode, stdout, stderr', () => { - spawnStub.returns(makeSpawnResult('[]', '', 0)); + spawnStub.callsFake(makeSpawnResult('[]', '', 0)); const result = server.call('provar_qualityhub_testcase_retrieve', { target_org: 'myorg', flags: [] }); const body = parseBody(result); assert.ok('requestId' in body); @@ -462,7 +503,7 @@ describe('qualityHubTools', () => { }); it('summary returns only requestId and exitCode', () => { - spawnStub.returns(makeSpawnResult('[]', '', 0)); + spawnStub.callsFake(makeSpawnResult('[]', '', 0)); const result = server.call('provar_qualityhub_testcase_retrieve', { target_org: 'myorg', flags: [], @@ -478,7 +519,7 @@ describe('qualityHubTools', () => { describe('provar_qualityhub_testcase_retrieve — fields param', () => { it('retains only specified keys', () => { - spawnStub.returns(makeSpawnResult('[]', '', 0)); + spawnStub.callsFake(makeSpawnResult('[]', '', 0)); const result = server.call('provar_qualityhub_testcase_retrieve', { target_org: 'myorg', flags: [], @@ -491,7 +532,7 @@ describe('qualityHubTools', () => { }); it('silently ignores unknown field names', () => { - spawnStub.returns(makeSpawnResult('[]', '', 0)); + spawnStub.callsFake(makeSpawnResult('[]', '', 0)); const result = server.call('provar_qualityhub_testcase_retrieve', { target_org: 'myorg', flags: [], @@ -508,7 +549,7 @@ describe('qualityHubTools', () => { describe('sf_path threading', () => { it('provar_qualityhub_connect uses explicit sf_path as the executable', () => { - spawnStub.returns(makeSpawnResult('ok', '', 0)); + spawnStub.callsFake(makeSpawnResult('ok', '', 0)); server.call('provar_qualityhub_connect', { target_org: 'myorg', flags: [], @@ -519,14 +560,14 @@ describe('qualityHubTools', () => { }); it('provar_qualityhub_display uses explicit sf_path as the executable', () => { - spawnStub.returns(makeSpawnResult('ok', '', 0)); + spawnStub.callsFake(makeSpawnResult('ok', '', 0)); server.call('provar_qualityhub_display', { flags: [], sf_path: '/custom/bin/sf' }); const [cmd] = spawnStub.firstCall.args as [string, string[]]; assert.equal(cmd, '/custom/bin/sf'); }); it('returns SF_NOT_FOUND with path hint when explicit sf_path gives ENOENT', () => { - spawnStub.returns(makeEnoentResult()); + spawnStub.callsFake(makeEnoentResult()); const result = server.call('provar_qualityhub_connect', { target_org: 'myorg', flags: [], diff --git a/test/unit/mcp/sfSpawn.test.ts b/test/unit/mcp/sfSpawn.test.ts index 34bcc9c..ed709e6 100644 --- a/test/unit/mcp/sfSpawn.test.ts +++ b/test/unit/mcp/sfSpawn.test.ts @@ -6,6 +6,7 @@ */ import { strict as assert } from 'node:assert'; +import fs from 'node:fs'; import os from 'node:os'; import path from 'node:path'; import sinon from 'sinon'; @@ -26,31 +27,45 @@ import { // ── Helpers ─────────────────────────────────────────────────────────────────── -function makeSpawnOk( - stdout = '', - stderr = '' -): { stdout: string; stderr: string; status: number; error: undefined; pid: number; output: never[]; signal: null } { - return { stdout, stderr, status: 0, error: undefined, pid: 1, output: [], signal: null }; -} - -function makeSpawnFail( - exitCode = 1, - stdout = '', - stderr = '' -): { stdout: string; stderr: string; status: number; error: undefined; pid: number; output: never[]; signal: null } { - return { stdout, stderr, status: exitCode, error: undefined, pid: 1, output: [], signal: null }; -} - -function makeEnoent(): { +// runSfCommand streams the child's stdout/stderr to temp files instead of +// buffering them in memory, so the stub for sfSpawnHelper.spawnSync must mimic a +// real child: write the captured output to the file descriptors passed via +// `stdio: ['ignore', outFd, errFd]`, then return the spawn result. The in-memory +// probe path (encoding/maxBuffer, no fd stdio) simply receives the object. +type FakeSpawnResult = { stdout: string; stderr: string; - status: null; - error: Error & { code: string }; - pid: undefined; + status: number | null; + error: (Error & { code?: string }) | undefined; + pid: number | undefined; output: never[]; signal: null; -} { - return { +}; +type SpawnFake = (...callArgs: unknown[]) => FakeSpawnResult; + +function spawnFake(result: FakeSpawnResult): SpawnFake { + return (...callArgs) => { + const stdio = (callArgs[2] as { stdio?: unknown[] } | undefined)?.stdio; + const outFd = stdio?.[1]; + const errFd = stdio?.[2]; + if (typeof outFd === 'number' && typeof errFd === 'number') { + fs.writeSync(outFd, result.stdout); + fs.writeSync(errFd, result.stderr); + } + return result; + }; +} + +function makeSpawnOk(stdout = '', stderr = ''): SpawnFake { + return spawnFake({ stdout, stderr, status: 0, error: undefined, pid: 1, output: [], signal: null }); +} + +function makeSpawnFail(exitCode = 1, stdout = '', stderr = ''): SpawnFake { + return spawnFake({ stdout, stderr, status: exitCode, error: undefined, pid: 1, output: [], signal: null }); +} + +function makeEnoent(): SpawnFake { + return spawnFake({ stdout: '', stderr: '', status: null, @@ -58,7 +73,7 @@ function makeEnoent(): { pid: undefined, output: [], signal: null, - }; + }); } // ── SfNotFoundError ─────────────────────────────────────────────────────────── @@ -205,7 +220,7 @@ describe('runSfCommand', () => { }); it('returns stdout, stderr, and exitCode on success', () => { - spawnStub.returns(makeSpawnOk('output', 'warn')); + spawnStub.callsFake(makeSpawnOk('output', 'warn')); const result = runSfCommand(['data', 'query']); assert.equal(result.stdout, 'output'); assert.equal(result.stderr, 'warn'); @@ -213,7 +228,7 @@ describe('runSfCommand', () => { }); it('passes args array to spawnSync', () => { - spawnStub.returns(makeSpawnOk()); + spawnStub.callsFake(makeSpawnOk()); runSfCommand(['provar', 'quality-hub', 'connect', '--target-org', 'myorg']); const [cmd, args] = spawnStub.firstCall.args as [string, string[]]; assert.equal(cmd, 'sf'); @@ -221,7 +236,7 @@ describe('runSfCommand', () => { }); it('uses explicit sfPath instead of cached path', () => { - spawnStub.returns(makeSpawnOk()); + spawnStub.callsFake(makeSpawnOk()); runSfCommand(['--version'], '/custom/path/sf'); const [cmd] = spawnStub.firstCall.args as [string, string[]]; assert.equal(cmd, '/custom/path/sf'); @@ -229,14 +244,14 @@ describe('runSfCommand', () => { it('falls through to auto-discovery when sfPath is empty string', () => { // Empty string is not a valid path — should behave as if sfPath was absent - spawnStub.returns(makeSpawnOk('ok')); + spawnStub.callsFake(makeSpawnOk('ok')); runSfCommand(['--version'], ''); const [cmd] = spawnStub.firstCall.args as [string, string[]]; assert.equal(cmd, 'sf'); // uses the cached path, not the empty string }); it('falls through to auto-discovery when sfPath is whitespace only', () => { - spawnStub.returns(makeSpawnOk('ok')); + spawnStub.callsFake(makeSpawnOk('ok')); runSfCommand(['--version'], ' '); const [cmd] = spawnStub.firstCall.args as [string, string[]]; assert.equal(cmd, 'sf'); @@ -255,7 +270,7 @@ describe('runSfCommand', () => { }); it('throws SfNotFoundError with path hint when explicit sfPath gives ENOENT', () => { - spawnStub.returns(makeEnoent()); + spawnStub.callsFake(makeEnoent()); assert.throws( () => runSfCommand(['--version'], '/bad/path/sf'), (err) => { @@ -287,12 +302,30 @@ describe('runSfCommand', () => { }); it('returns non-zero exitCode from failed command', () => { - spawnStub.returns(makeSpawnFail(2, '', 'error text')); + spawnStub.callsFake(makeSpawnFail(2, '', 'error text')); const result = runSfCommand(['bad', 'command']); assert.equal(result.exitCode, 2); assert.equal(result.stderr, 'error text'); }); + // Regression (PDX-513): pre-streaming, spawnSync({ maxBuffer: 50 MB }) aborted + // the whole call with ENOBUFS the moment combined child output crossed the cap. + // Streaming stdout/stderr to temp files removes the in-memory ceiling entirely, + // so an over-cap payload must round-trip with no throw. makeSpawnOk writes the + // payload to the inherited fd, exercising the exact file-backed capture path. + it('captures output larger than the retired 50 MB cap without ENOBUFS (file-backed)', function () { + this.timeout(15_000); + const huge = 'x'.repeat(51 * 1024 * 1024); // 51 MB — comfortably over the old 50 MB cap + spawnStub.callsFake(makeSpawnOk(huge)); + + let result: ReturnType | undefined; + assert.doesNotThrow(() => { + result = runSfCommand(['provar', 'automation', 'test', 'run']); + }); + assert.equal(result?.exitCode, 0); + assert.equal(result?.stdout.length, huge.length, 'full over-cap output should round-trip through the temp file'); + }); + describe('Windows shell path injection guard', () => { beforeEach(() => { setSfPlatformForTesting('win32'); @@ -308,7 +341,7 @@ describe('runSfCommand', () => { }); it('accepts clean .cmd path on win32', () => { - spawnStub.returns(makeSpawnOk('ok')); + spawnStub.callsFake(makeSpawnOk('ok')); // Should not throw — clean path const result = runSfCommand(['--version'], 'C:\\Program Files\\sf\\bin\\sf.cmd'); assert.equal(result.exitCode, 0); @@ -322,7 +355,7 @@ describe('runSfCommand', () => { }); it('caches "sf" when shell:false probe succeeds', () => { - spawnStub.returns(makeSpawnOk('sf/2.0.0')); + spawnStub.callsFake(makeSpawnOk('sf/2.0.0')); runSfCommand(['--version']); const [, args, opts] = spawnStub.firstCall.args as [string, string[], { shell: boolean }]; assert.deepEqual(args, ['--version']); // probe call @@ -331,7 +364,7 @@ describe('runSfCommand', () => { it('returns SF_NOT_FOUND when probe fails and no common path exists', () => { // Both probe attempts fail with ENOENT, no common paths match - spawnStub.returns(makeEnoent()); + spawnStub.callsFake(makeEnoent()); assert.throws( () => runSfCommand(['--version']), (err) => { @@ -344,11 +377,11 @@ describe('runSfCommand', () => { it('Windows two-phase probe: retries with shell:true on ENOENT', () => { setSfPlatformForTesting('win32'); // First call (shell:false) → ENOENT - spawnStub.onFirstCall().returns(makeEnoent()); + spawnStub.onFirstCall().callsFake(makeEnoent()); // Second call (shell:true) → success - spawnStub.onSecondCall().returns(makeSpawnOk('sf/2.0.0')); + spawnStub.onSecondCall().callsFake(makeSpawnOk('sf/2.0.0')); // Third call → the actual command - spawnStub.onThirdCall().returns(makeSpawnOk('result')); + spawnStub.onThirdCall().callsFake(makeSpawnOk('result')); const result = runSfCommand(['--version']); assert.equal(result.stdout, 'result'); @@ -362,7 +395,7 @@ describe('runSfCommand', () => { describe('Windows shell quoting (spaced executable & args not split)', () => { beforeEach(() => { setSfPlatformForTesting('win32'); - spawnStub.returns(makeSpawnOk('ok')); + spawnStub.callsFake(makeSpawnOk('ok')); }); it('(a) does not split an auto-resolved Program Files path', () => { @@ -402,13 +435,45 @@ describe('runSfCommand', () => { it('still rejects a user sf_path with shell metacharacters (spaces are now allowed)', () => { assert.throws(() => runSfCommand(['--version'], 'C:\\sf & evil\\sf.cmd'), /unsafe for shell execution/); // A spaced-but-clean user path is accepted and quoted. - spawnStub.returns(makeSpawnOk('ok')); + spawnStub.callsFake(makeSpawnOk('ok')); const result = runSfCommand(['--version'], 'C:\\Program Files\\sf\\bin\\sf.cmd'); assert.equal(result.exitCode, 0); }); }); }); +// ── runSfCommand — real subprocess (integration) ────────────────────────────── +// The unit tests above stub sfSpawnHelper.spawnSync, so they exercise the temp-file +// capture but not real OS process stdio. This block spawns a real `node` child that +// writes well over the retired 50 MB cap, proving the streaming fix end-to-end: real +// fd inheritance, the child's own flush-on-exit, and read-back — the exact path that +// used to abort with ENOBUFS under the in-memory maxBuffer. +describe('runSfCommand — real subprocess (integration)', () => { + beforeEach(() => { + setSfPathCacheForTesting(undefined); + setSfPlatformForTesting(undefined); + }); + afterEach(() => { + setSfPathCacheForTesting(undefined); + setSfPlatformForTesting(undefined); + }); + + it('streams >50 MB of real child stdout through temp files without ENOBUFS', function () { + this.timeout(30_000); + const bytes = 60 * 1024 * 1024; // 60 MB — well over the retired 50 MB maxBuffer cap + // Pass `node` as the executable explicitly (bypasses sf discovery) and have it + // write a raw 60 MB buffer to stdout, which runSfCommand inherits onto the + // temp-file descriptor. With the old maxBuffer capture this aborted with ENOBUFS. + const script = `process.stdout.write(Buffer.alloc(${bytes}, 0x78));`; + let result: ReturnType | undefined; + assert.doesNotThrow(() => { + result = runSfCommand(['-e', script], process.execPath); + }); + assert.equal(result?.exitCode, 0); + assert.equal(result?.stdout.length, bytes, 'full over-cap output must round-trip from the real child'); + }); +}); + // ── ProvarPluginNotFoundError ─────────────────────────────────────────────── describe('ProvarPluginNotFoundError', () => { @@ -466,12 +531,12 @@ describe('probeProvarTopic', () => { }); it('returns true when `sf provar --help` succeeds', () => { - spawnStub.returns(makeSpawnOk('USAGE\n $ sf provar COMMAND')); + spawnStub.callsFake(makeSpawnOk('USAGE\n $ sf provar COMMAND')); assert.equal(probeProvarTopic(), true); }); it('returns false when the provar topic is missing', () => { - spawnStub.returns(makeSpawnFail(1, '', 'command provar not found')); + spawnStub.callsFake(makeSpawnFail(1, '', 'command provar not found')); assert.equal(probeProvarTopic(), false); });