Skip to content

test(harness): silence the :: runner-command output of the unit suite's call-through @actions/core spies - #605

Merged
jeefy merged 3 commits into
mainfrom
quality/test-quiet-actions-core
Oct 29, 2026
Merged

jeefy merged 3 commits into
mainfrom
quality/test-quiet-actions-core

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Test Improvement

Stops the unit suite's call-through @actions/core from emitting workflow commands. The shared setup file mocks @actions/core with spy: true, so core.debug/info/warning/error/setFailed/setSecret still run for real and write ::debug::, ::warning::, ::error:: and ::add-mask:: lines straight to process.stdout, bypassing vitest's console capture. The GitHub runner parses them: on a green build-test run (job 121828647038, head of #591) the test:coverage step held 61 ##[warning] and 18 ##[error] lines and the check run carried 10 failure + 10 warning annotations, all test fixtures (Codertocat is not a org member or collaborator, status=500), while ::add-mask::secret-key redacted a fixture string and setFailed set process.exitCode inside the worker. Locally those lines were ~3k of the ~12.8k a run printed, hiding the one genuine warning in the output (a MaxListenersExceededWarning from @mswjs/interceptors in meow.test.ts).

Files / cluster claimed — one new file plus one line; disjoint from every held PR (#595 edits __tests__/setup.ts and #595/#598/#599/#603 edit docs/contributing.md, so neither is touched here; no held PR edits vitest.config.mjs):

file change
__tests__/setupQuietActionsCore.ts (new) beforeEach installs a no-op implementation on the 14 @actions/core output functions (debug, info, warning, error, notice, setFailed, setSecret, setOutput, exportVariable, addPath, saveState, startGroup, endGroup, setCommandEcho). Per test because mockReset: true restores a spy's original implementation. Spies stay — the 36 expect(core.warning/setFailed/debug/info) assertions in the suite are untouched — and getInput etc. remain real. Header comment carries the rationale.
vitest.config.mjs adds the file to setupFiles after __tests__/setup.ts (the e2e-coverage and mutation configs inherit it via mergeConfig; the bundle suite runs dist/index.js in a child process and was never affected)

Testing

  • npx vitest run — 191 files, 2473 passed, 75 skipped; the full run now prints no :: line (before: 2946 ::debug::, 57 ::warning::, 17 ::error::, 1 ::add-mask::), so the MaxListenersExceededWarning is the only stray output left
  • npx vitest run __tests__/label/kind.test.ts __tests__/issueCommentTest/handleIssueComment.test.ts __tests__/utils/authGuards.test.ts __tests__/plugins/approveReview.test.ts __tests__/issueCommentTest/meow.test.ts (the files asserting on core.* calls) — 164 passed
  • npx eslint __tests__/setupQuietActionsCore.ts vitest.config.mjs — clean; npx tsc --noEmit — clean

Related Issue

Closes #604


Filed by quality agent (hold-gated mode). Human review required.

— hive: agent=quality backend=copilot model=claude-fable-5.1 copilot=1.0.88

…emitting workflow commands

setup.ts mocks @actions/core with spy: true, so the real implementation still
runs and core.debug/info/warning/error/setFailed/setSecret write ::debug::,
::warning::, ::error:: and ::add-mask:: lines straight to process.stdout,
bypassing vitest's console capture. Under GitHub Actions the runner parses
them as workflow commands: the test:coverage step of a green build-test run
(job 121828647038) contained 61 ##[warning] and 18 ##[error] lines and the
check run carried ten failure and ten warning annotations, all test fixtures
("Codertocat is not a org member or collaborator", "status=500"), while
::add-mask::secret-key redacted a fixture string from the rest of the log and
setFailed set process.exitCode inside the worker. Locally the same lines were
~3k of the ~12.8k lines a run printed, burying the one genuine warning in the
output (a MaxListenersExceededWarning from @mswjs/interceptors in meow.test.ts).

A second setup file re-installs a no-op implementation on the output
functions before every test (mockReset: true restores the originals), keeping
the spies tests assert on while dropping the writes. 191 files / 2473 tests
pass unchanged and a full run now prints no :: line.

Signed-off-by: quality <280983584+hivecommons-hive[bot]@users.noreply.github.com>
@hivecommons-hive
hivecommons-hive Bot requested a review from jpmcb as a code owner October 28, 2026 12:03
@hivecommons-hive hivecommons-hive Bot added the hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 28, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

Important

Held for human review by the hive's ACMM level gate.

This PR was opened by the "quality" agent while Hive policy required a human checkpoint for that agent. Agent PRs are held at ACMM L3–L5; an outreach PR that targets a repository this hive does not manage is held at every level because it publishes project-facing communication into someone else's project.

Hive will keep the hold label until a human removes it. Operators can make a deliberate one-off release during an ACMM level change with release_level_holds=true, but level changes never release this hold automatically.

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the tree at b7ae47a and reproduced the mechanism rather than taking the body's word for it:

  • __tests__/setup.ts:4 mocks @actions/core with { spy: true }, so the real output functions run and write ::-prefixed workflow commands to process.stdout. Confirmed locally: with main's vitest.config.mjs, npx vitest run __tests__/label/kind.test.ts alone emits 64 raw :: lines; with this PR's config, the same files emit zero.
  • The per-test beforeEach reinstall (__tests__/setupQuietActionsCore.ts:30-32) is required, not defensive: vitest.config.mjs:13 sets mockReset: true, which restores a spy's original implementation between tests. A one-time install would silence only the first test per file.
  • Spy assertions survive: a no-op mockImplementation still records calls, and __tests__/issueCommentTest/meow.test.ts:249-260 (setSecret spy + toHaveBeenCalledWith('secret-key')) passes — ran meow.test.ts, authGuards.test.ts, kind.test.ts: 62/62 green.
  • The __tests__/bundle suite asserts on literal ::warning:: stdout (__tests__/bundle/meow.test.ts:142 etc.), but it runs dist/index.js in a child process (vitest.e2e-coverage.config.mjs:19: "The bundle only ever runs in a child process; vitest never imports it"), so the in-process mock cannot reach it. Not affected.
  • No test reads process.exitCode or GITHUB_OUTPUT, so neutering setFailed/setOutput side effects breaks nothing.

One deliberate trade-off worth naming, not a defect: no-oping setSecret means fixture strings are no longer ::add-mask::-ed in CI logs. Everything the suite passes to it is a hard-coded fixture (secret-key), so nothing real loses masking.

Looks correct to me — small, well-scoped, and it removes the false failure/warning annotations the runner was attaching to green runs. A human should do the final approve/merge.

— hive: agent=reviewer backend=copilot model=claude-fable-5 copilot=1.0.88

@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

/kind cleanup


🐝 Hive Agent: quality | Instance: hosted-available-lke648397-260827-5q9t | SHA: 9acb046

— hive: agent=quality backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

⚠️ Hold-gate mergeability: this PR has a real conflict with its base

This PR is hold-gated and has fallen behind its base branch. The hive tried to merge the base in to keep it mergeable while it waits on review, but that merge conflicts and the hive cannot resolve it automatically.

It will stay held and will NOT merge on its own even once reviewed — removing hold alone will not be enough. Someone will need to merge or rebase the base branch in by hand (or resolve the conflict another way) before this PR can merge.

@jeefy jeefy left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes please — every CI log this week carried the ::error:: fixture noise and the add-mask redactions. Approving on content; conflicting with main after #607 (setup.ts beforeEach) — rebase and I will drop hold.

Keeps #615's globalSetup entry in vitest.config.mjs alongside the new
setup file. Unit suite on main writes 3218 '::' workflow-command lines to
stdout; with setupQuietActionsCore.ts it writes none, 110 files pass.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mrbobbytables

Copy link
Copy Markdown
Member

Re-merged main (a499eb2): kept #615's globalSetup entry in vitest.config.mjs next to the new setup file.

Verified locally: the unit suite on main writes 3218 ::debug/warning/error/add-mask lines to stdout; with this branch it writes 0, 110 files pass and coverage stays above the floors. Note #611 also edits vitest.config.mjs and will need a trivial rebase after this merges.

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified against the tree and by running the suite — looks correct to me.

The red build-test is not this PR's fault. Run 40905577513 fails only __tests__/adapters/app/webhookGuards.test.ts:126 ("expected { jobs: 'lgtm', 'dry-run': '' } to strictly equal { 'dry-run': '' }") — the pre-existing main breakage from #612+#614, which #616 fixes. Once #616 merges, a rebase/rerun should go green; nothing in this diff needs to change.

What I checked:

  • __tests__/setup.ts:7 is indeed vi.mock('@actions/core', { spy: true }), so the real output functions run and write :: workflow commands to stdout — the mechanism the body describes is real.
  • vitest.config.mjs:13-15 sets clearMocks/mockReset/restoreMocks, and vitest's mockReset restores a spy's original implementation, so the per-test beforeEach reinstall in __tests__/setupQuietActionsCore.ts:30-32 is required, not defensive.
  • No test asserts process.exitCode (only src/server/main.ts:7 sets it), so no-oping setFailed loses nothing.
  • Ran __tests__/label/kind.test.ts, handleIssueComment.test.ts, authGuards.test.ts, approveReview.test.ts with this PR applied: 124/124 pass and the output contains zero :: lines; the expect(core.warning/...) call assertions still work because mockImplementation(() => {}) keeps the spy recording.

Suggestion for whoever lands the queue: merge #616 first, then rerun this PR's checks.

— hive: agent=reviewer backend=copilot model=claude-fable-5 copilot=1.0.88

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@jeefy jeefy removed the hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 29, 2026
@jeefy
jeefy merged commit 333ba1f into main Oct 29, 2026
6 checks passed
@jeefy
jeefy deleted the quality/test-quiet-actions-core branch October 29, 2026 17:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[quality] Unit suite's call-through @actions/core emits workflow commands: green build-test runs carry 10 failure + 10 warning fixture annotations

2 participants