Skip to content

test(lgtm): pin the refusal error log, the no-label cancel debug, the token guard, the missing-head error and isForbidden's non-object arms - #582

Closed
hivecommons-hive[bot] wants to merge 1 commit into
mainfrom
quality/test-lgtm-guards
Closed

hivecommons-hive[bot] wants to merge 1 commit into
mainfrom
quality/test-lgtm-guards

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Test Improvement

A Stryker probe of the lgtm label-gating cluster — src/labels/lgtm.ts and src/plugins/lgtmBinding.ts (npx stryker run --mutate src/labels/lgtm.ts,src/plugins/lgtmBinding.ts on #572's harness, main @ e3119ad, Stryker 10.0.0 / Vitest 5.0.4, 8 workers, 6 m 36 s) left 24 of 202 mutants alive (88.12 %). Fourteen flip a guard or drop a log line the suite reaches but never distinguishes from its mutation. This PR adds two test files that kill all fourteen; re-running the probe with them reports 192 killed / 10 survived (95.05 %). The lgtm label is the automatic-merge gate, so this is the "selected high-risk behavior" #554 asks to probe.

Files / cluster claimed: __tests__/label/lgtmGuards.test.ts (new), __tests__/plugins/lgtmBindingGuards.test.ts (new). No src/, dist/ or existing-test change. Disjoint from every held PR on this cluster: #571 edits the existing __tests__/label/lgtm.test.ts / __tests__/plugins/lgtmBinding.test.ts (helper call only); #557 is tide's lgtm re-check; #573 is cronJobs/lgtm.ts. Same pattern as #575 (ownersGuards), #578 (tideGuards), #579 (authGuards), #580, #581.

Mutants pinned (span → test)

Span Mutation Test
lgtm.ts:29:47-65 { required: true } → {} / true → false the handler would proceed with an empty token requires the github-token input before reading anything
lgtm.ts:140:3 core.error(msg) → ; the refusal loses its ::error:: annotation a refusal is logged as an error with the same message it replies with
lgtm.ts:96:16 debug string → '' a cancel on an issue without the label says so in the debug log and removes nothing (also pins that no DELETE follows)
lgtmBinding.ts:171:60-78 { required: true } (2 mutants) requires the github-token input before reading the configuration
lgtmBinding.ts:166:24-63 pull_request?.head?.sha → head.sha a pull_request without a head throws "Cannot read properties of undefined" instead of the missing pull request head TypeError throws the missing-head error when the pull request has no head, not a property read failure
lgtmBinding.ts:173:16 debug string → '' bind_to_commit: false says so in the debug log
lgtmBinding.ts:182:10-74 isForbidden conjunction — 6 mutants (true × 4, || × 2) 'status' in 'boom' / null.status throws a TypeError out of the catch instead of the wrapped "could not bind lgtm to …" / "could not read the prow/lgtm status of …" error bindLgtm / isLgtmBound wraps null / a string / a number without the permission hint (an octokit stub whose statuses calls reject with a bare value — the same precedent #579 uses for getContent)

Reviewed survivors, not pinned (10)

I have not edited docs/contributing.md's survivor table because that file is #572's claimed ground; this list is the record until #572 lands.

Validation

  • npx vitest run __tests__/label/lgtmGuards.test.ts __tests__/plugins/lgtmBindingGuards.test.ts → 14 passed
  • npx eslint on both files → clean
  • Stryker on the two files before: 178 killed / 24 survived (88.12 %); after: 192 / 10 (95.05 %), 2 m 45 s
  • Unit coverage on main (npm run test:coverage, 167 files / 2330 tests, 2026-10-25): 99.92 % stmts / 99.35 % branches / 100 % funcs / 99.91 % lines; both files are already at 100 % lines. E2E (npm run test:coverage:e2e, 86 files / 528 tests): 100 % lines / 99.46 % branches. This PR adds assertion strength, not line coverage.

Related Issue

Refs #554 — ticks a slice of the "Strengthen assertions and the test harness" section (reviewed mutation survivors on the lgtm gate); the tracker's CI-enforcement, release-delivery and routing sections stay open and are owned elsewhere.


Filed by quality agent (hold-gated mode). Human review required.

— hive: agent=quality backend=copilot model=claude-fable-5.1 copilot=1.0.88 requested_by=@mrbobbytables

… token guard, the missing-head error and isForbidden's non-object arms

A Stryker probe of src/labels/lgtm.ts and src/plugins/lgtmBinding.ts
(npx stryker run --mutate src/labels/lgtm.ts,src/plugins/lgtmBinding.ts
on the harness of #572, main @ e3119ad, Stryker 10.0.0 / Vitest 5.0.4,
8 workers, 6 m 36 s) left 24 of 202 mutants alive (88.12 %). Fourteen
flip a guard or drop a log line the suite reaches but never distinguishes
from its mutation. Two new test files kill all fourteen; the re-run
reports 192 killed / 10 survived (95.05 %).

__tests__/label/lgtmGuards.test.ts (src/labels/lgtm.ts):
- 29 `getInput('github-token', { required: true })` (2 mutants): the
  handler fails fast with "Input required and not supplied" and posts
  nothing
- 140 `core.error(msg)` in refuse: the refusal is logged as an error with
  the same text it replies with
- 96 the "could not find lgtm to remove" debug line of a cancel on an
  issue without the label, and that no DELETE follows

__tests__/plugins/lgtmBindingGuards.test.ts (src/plugins/lgtmBinding.ts):
- 171 `getInput('github-token', { required: true })` (2 mutants) in
  lgtmOnPullRequest
- 166 `pull_request?.head?.sha` -> `head.sha`: a pull_request without a
  head throws the "missing pull request head" TypeError, not a property
  read failure
- 173 the "lgtm: bind_to_commit is false" debug line
- 182 isForbidden `typeof error === 'object' && error !== null &&
  'status' in error` (6 mutants: `true` x4, `||` x2): bindLgtm and
  isLgtmBound wrap a null, string or number rejection in their own error
  instead of throwing from `'status' in <primitive>` / `null.status`

Reviewed survivors, not pinned (10): lgtm.ts 33, 34 (x2), 35, 65 and
lgtmBinding.ts 162, 177, 178 are optional chains on fields every
issue_comment / pull_request payload carries (comment.body, comment.user,
issue.user, pull_request after the head check, sender once isBotUser is
true); lgtmBinding.ts 66 `targetUrl === undefined` -> false spreads
`target_url: undefined`, which JSON serialisation drops, so the request
body is identical; lgtmBinding.ts 156 `label?.name ?? ''` -> any other
string still fails the `!== 'lgtm'` comparison and returns early.

Refs #554

Signed-off-by: quality <280983584+hivecommons-hive[bot]@users.noreply.github.com>
@hivecommons-hive
hivecommons-hive Bot requested a review from jpmcb as a code owner October 25, 2026 19:11
@hivecommons-hive hivecommons-hive Bot added the hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 25, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

Important

Held for human review by the hive's ACMM level gate.

This PR was opened by the "quality" agent while Hive policy required a human checkpoint for that agent. Agent PRs are held at ACMM L3–L5; an outreach PR that targets a repository this hive does not manage is held at every level because it publishes project-facing communication into someone else's project.

Hive will keep the hold label until a human removes it. Operators can make a deliberate one-off release during an ACMM level change with release_level_holds=true, but level changes never release this hold automatically.

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read both new test files against the tree at 2dd1629 — this looks correct to me, and the mutant→test mapping in the body checks out.

Verified against the source:

  • src/labels/lgtm.ts:30 — getInput('github-token', { required: true }) is the first statement; the test deletes INPUT_GITHUB-TOKEN and asserts no comment POST follows, which kills the { required: true } → {} mutants.
  • src/labels/lgtm.ts:140 (refuse) — core.error(msg) is asserted exactly once with the same string the reply carries; the fixture's commenter is the issue author (Codertocat on both sides of issueCommentEvent.json), so a plain /lgtm genuinely reaches refuse before any authorization read.
  • src/labels/lgtm.ts:96 — the could not find lgtm to remove debug plus the notCalled() assertion on the DELETE pins both the log and the early return.
  • src/plugins/lgtmBinding.ts:166-168 — passing { pull_request: { number: 1 } } makes the optional chain yield undefined; the /^github context payload missing pull request head: / match distinguishes the TypeError from the property-read crash the ?.-removal mutants would produce.
  • src/plugins/lgtmBinding.ts:186 (isForbidden) — the bare-value rejections (null, 'boom', 403) each short-circuit a different conjunct; replacing any arm with true makes 'status' in null / null.status throw out of the catch instead of the wrapped message the tests assert. The octokit stub rejecting with a bare value mirrors the getContent precedent in authGuards (#579).

Scope claim holds: two new test files only, disjoint from #571's edits to the existing lgtm.test.ts/lgtmBinding.test.ts and from #573 (cronJobs/lgtm.ts). The ten unpinned survivors listed in the body are correctly classed — the optional chains sit behind guards every delivered payload satisfies, and the targetUrl === undefined → false mutant produces an identical serialized request body.

No defects found. Refs #554 (non-closing) is appropriate: the tracker's other sections stay open.

— hive: agent=reviewer backend=copilot model=claude-fable-5 copilot=1.0.88

@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

/kind cleanup


🐝 Hive Agent: quality | Instance: hosted-available-lke648397-260827-5q9t | SHA: e3119ad

— hive: agent=quality backend=copilot model=claude-fable-5.1 copilot=1.0.88

@github-actions github-actions Bot added kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. and removed needs-kind labels Oct 26, 2026
@mrbobbytables

Copy link
Copy Markdown
Member

Consolidated into #588 together with the other eight open mutation-survivor pin PRs (commit cherry-picked unchanged, DCO preserved). Closing in favour of that PR.

@mrbobbytables
mrbobbytables deleted the quality/test-lgtm-guards branch October 26, 2026 20:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

hold Indicates that a PR should not merge because someone has issued a /hold command. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant