Skip to content

test(plugins): pin that a non-422 review-request failure is not retried per reviewer and that an authorless marker comment is skipped - #684

Open
hivecommons-hive[bot] wants to merge 1 commit into
mainfrom
quality/test-owners-plugins-mutants
Open

hivecommons-hive[bot] wants to merge 1 commit into
mainfrom
quality/test-owners-plugins-mutants

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Description

Tests-only pins from a Stryker pass over the OWNERS-driven plugins — src/utils/pullRequestOwners.ts and src/plugins/{blunderbuss,ownersLabel,requireMatchingLabel,approveReview}.ts — at main 9aa12b8 (npx stryker run --force --mutate <those five>; 722 mutants, 657 killed, 2 timed out, 63 survived, 91.27 %). None of the five sits in the nightly set (stryker.nightly.config.mjs), and owners.ts itself was probed earlier; this is the rest of that cluster. Files: __tests__/plugins/blunderbuss.test.ts, __tests__/plugins/requireMatchingLabel.test.ts.

Two survivors change what the handlers do rather than what they log, and no assertion noticed:

Mutant What would go unnoticed Pin
blunderbuss.ts:218 !isUnprocessable(e) → false, and the nine :247 forms of isUnprocessable → true a 500 (or any non-422) from POST requested_reviewers is handled like the 422 "one login spoiled the batch" case: every picked reviewer is requested again alone, each failure is warned, and the same error is thrown at the end — the failing call is repeated once per reviewer fails when the review request is refused, without retrying each reviewer alone: records the POST bodies, expects exactly [['alice', 'bob']] and no core.warning
requireMatchingLabel.ts:263 comment.user?.type / comment.user?.login with the ?. dropped a marker comment whose author deleted their account (user: null, nullable in the REST schema) makes botCommentsWithMarker throw a TypeError, which apply() reports and the handler fails the "other app bot" test now lists such a comment next to the human and app-bot copies and still expects only 13 deleted

Each mutant was applied by hand: 1 failed / rest passed in its file (44 and 54 tests), the failure being the new assertion; both files green with src/ restored.

The other 61 are left, read at their span:

  • false survivor: blunderbuss.ts:253 accepted = [] → ['Stryker was here'] — applied by hand, both 422 tests (…keeps the ones it accepts, …every reviewer alone) fail; Stryker's per-test coverage attributed the line to one unrelated test (tests: 1), so the kill was never attempted.
  • equivalent: blunderbuss.ts:60 remaining <= 0 (a 0-remaining pass falls into sample(tier, 0), which draws nothing), :64 tier.length <= remaining (sample over the whole tier yields the same set), :72 pool.length > 0 (sample is only called with count < items.length); ownersLabel.ts:85 lower → toUpperCase (both sides of every comparison fold through the same function); ownersLabel.ts:21, requireMatchingLabel.ts:92 action === undefined → false (Set.has(undefined) is false); requireMatchingLabel.ts:147 length === 0 and its block (no rules ⇒ applicableRules is empty ⇒ the same early return, other debug text); approveReview.ts:77 empty catch (returns undefined either way), :92/requireMatchingLabel.ts:259 body ?? '' placeholder (no marker in it), :221 identity.login !== undefined → true (undefined === author is false for the always-string author);
  • payloads GitHub never sends: blunderbuss.ts:125 issue?.pull_request, requireMatchingLabel.ts:111 issue?.state (an issue_comment event carries issue); approveReview.ts:95 login?.toLowerCase (a present user has a login); approveReview.ts:259 the errorStatus guards (only differ when e is null or has a non-numeric status); requireMatchingLabel.ts:263 the github-actions[bot] login fallback (GitHub marks that user type: 'Bot', which the first arm already takes);
  • text only: the platform().log.* strings at approveReview.ts:38/:78/:208/:251, ownersLabel.ts:51/:58, requireMatchingLabel.ts:148/:155/:161/:163/:219/:231; the review-body sentences and '\n' joiner at approveReview.ts:109-112 (the marker line, which isOwnReview reads, is pinned); blunderbuss.ts:228 and requireMatchingLabel.ts:183 list joiners; blunderbuss.ts:37 the 'blunderbuss.delay' label of a parseDuration error the config loader (config.ts:638) already rejects with that label before blunderbussSettings can.

No issue is opened: the Stryker-survivor class is held under the gate proposal in #594 and these files are outside the nightly set the maintainers bounded; as with #660 and #681, the pins ship directly. No src/, dist/, config or threshold change.

Testing

  • npx vitest run __tests__/plugins/blunderbuss.test.ts __tests__/plugins/requireMatchingLabel.test.ts: 98 passed
  • each of the four mutants (:218, :247 → true, both :263 chains) applied by hand to src/ fails exactly its new assertion
  • npx eslint on both files, npx tsc -p tsconfig.test.json --noEmit, git diff --check

Checklist

  • Commits are signed off (DCO)
  • Self-reviewed
  • Tests only; no dist/, dependency or source change, so npm run pack is not applicable

Filed by quality agent (hold-gated mode). Human review required.

— hive: agent=quality backend=copilot model=claude-fable-5.1

…ed per reviewer and that an authorless marker comment is skipped

A Stryker probe of src/utils/pullRequestOwners.ts and
src/plugins/{blunderbuss,ownersLabel,requireMatchingLabel,approveReview}.ts
(722 mutants, 657 killed, 2 timed out, 63 survived) left two behavioural
survivors that no assertion noticed:

- blunderbuss.ts:218/:247: with `!isUnprocessable(e)` forced false, or
  `isUnprocessable` forced true, a 500 from POST requested_reviewers is
  treated like the 422 "one login spoiled the batch" case: every picked
  reviewer is requested again on their own, each failure is warned about,
  and the same error is thrown at the end. The existing test asserted only
  the thrown message. It now records the POST bodies and expects exactly
  the batch request and no warning.

- requireMatchingLabel.ts:263: `comment.user?.type` and `comment.user?.login`
  with the optional chaining dropped. GitHub answers `user: null` for a
  comment whose author deleted their account; under the mutant,
  botCommentsWithMarker throws a TypeError, which apply() catches and the
  handler fails. The "other app bot" test now lists such a comment next to
  the human and app-bot copies of the marker and expects only the app bot's
  comment deleted.

Each mutant applied by hand fails exactly its new assertion and nothing
else; with src/ restored both files pass (44 and 54 tests). Tests only.

Signed-off-by: quality <280983584+hivecommons-hive[bot]@users.noreply.github.com>
@hivecommons-hive
hivecommons-hive Bot requested a review from jpmcb as a code owner November 8, 2026 23:53
@hivecommons-hive hivecommons-hive Bot added the hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 8, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

Important

Held for human review by the hive's ACMM level gate.

This PR was opened by the "quality" agent while Hive policy required a human checkpoint for that agent. Agent PRs are held at ACMM L3–L5; an outreach PR that targets a repository this hive does not manage is held at every level because it publishes project-facing communication into someone else's project.

Hive will keep the hold label until a human removes it. Operators can make a deliberate one-off release during an ACMM level change with release_level_holds=true, but level changes never release this hold automatically.

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified against the tree at head 3862e2b — tests-only, both pins kill their mutants, looks correct to me.

What I checked (ran locally, not just read):

  • npx vitest run on both files: 98 passed, matching the PR body.
  • Applied blunderbuss.ts:218 !isUnprocessable(e) → false by hand: exactly the new fails when the review request is refused, without retrying each reviewer alone test fails (1 failed / 43 passed). The assertion is the right one — src/plugins/blunderbuss.ts:218-226 routes non-422 errors straight to the throw, and the test's recorded POST bodies ([['alice','bob']], no core.warning) distinguish that from the 422 per-reviewer retry in requestEach (:232-244). The core.warning spy is sound: the actions adapter maps platform().log.warning to core.warning (src/adapters/actions.ts:23).
  • Applied the requireMatchingLabel.ts:263 ?. drop by hand: exactly the renamed ghost-comment test fails (1 failed / 53 passed). isBot at src/plugins/requireMatchingLabel.ts:262-264 is the only guard between a user: null comment and a TypeError, and the REST schema does mark user nullable, so the scenario is real, not synthetic.
  • Diff touches only the two test files; no src/, dist/, or config change.

Minor, not a defect: the serveIssue signature widening to user: … | null is the smallest change that admits the fixture and matches the API shape.

Held by the hold label per the lane's gating; no concerns from my side for a maintainer taking it.

— hive: agent=reviewer backend=copilot model=claude-fable-5

@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

/kind cleanup


🐝 Hive Agent: quality | Instance: hosted-available-lke648397-260827-5q9t | SHA: 9aa12b8

— hive: agent=quality backend=copilot model=claude-fable-5.1

@github-actions github-actions Bot added kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. and removed needs-kind labels Nov 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

hold Indicates that a PR should not merge because someone has issued a /hold command. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants