testbot: bigger PRs, Slack PR card, trimmed coverage report

[jiaenren]

Description

Three testbot improvements.

1. Target 3 files per run instead of 1

• max_targets 1 → 3, on both the dispatch input default and the schedule-path fallback. Nothing hardcodes the count: SELECT_TARGETS_PROMPT.md renders pick at most {max_targets} and the generator prompt already loops per target.
• max_turns 200 → 400. The generator runs the full read/write/verify/coverage loop per target, and a single target has already exhausted the limit once (runs/26536045087, 101/100, after context auto-compaction).
• max_uncovered stays 500. It is a per-target cap (_cap_ranges runs inside the per-file loop, remaining reset per file), and recent targets carried only 35–231 uncovered lines — so it is not the binding constraint. The per-PR ceiling still rises 500 → 1500 as a consequence of targeting 3 files.
• README table refreshed; it documented max_turns=100 / timeout_minutes=30 against the workflow's actual 200 / 60.

Schedule and timeout_minutes deliberately unchanged. Note the workflow is hourly with concurrency.cancel-in-progress: true, so a run is cancelled when the next hourly trigger fires — the effective ceiling is ~60 min regardless of timeout_minutes. If 3-target runs start getting cancelled mid-generation, the fix is slowing the cron rather than raising the timeout.

2. Let Slack unfurl the PR link into a GitHub card

The review request posted as plain text with no preview card.

Two things in the payload prevented it: Slack unfurls links from a message's text, not from links inside Block Kit blocks; and bot-posted messages do not unfurl unless unfurl_links is set. The blocks array duplicated text verbatim, so dropping it changes nothing about how the message reads.

3. Drop the per-range checklist from the PR body

The coverage report listed every targeted range as its own bullet with a ✅/❌ and hit count — 35 bullets on #1253, all ✅, under a summary line that already said 61/61.

Now: the per-target summary line, plus only the ranges still missing coverage, capped at MAX_REPORTED_RANGES with an "and N more" tail. The JSON sidecar keeps full per-range detail for the generator's self-iteration loop, which is what actually consumes it.

## Coverage gain on listed uncovered ranges

✅ **`src/lib/data/storage/uploading.py`** — 61/61 listed lines hit (100%)

⚠️ **`src/lib/foo.py`** — 8/40 listed lines hit (20%)
  - still uncovered: lines 10-11, lines 12-13, ..., and 7 more

Verification

bazel test //src/scripts/testbot:all //src/scripts/testbot/tests:all — 19 targets pass, including every -pylint sibling. Workflow YAML parses with the expected defaults. Added tests covering the Slack payload's unfurl flags and the three report-rendering cases (fully covered, partially covered, capped overflow).

The Slack card itself can only be confirmed once this runs with the testbot bot token, since posting as a user unfurls regardless and would not exercise the bot path.

Issue - None

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features

  • Test generation workflows now support up to 3 targets and 400 agent turns by default.
  • Coverage reports show only remaining under-covered ranges and summarize additional ranges beyond the first 8.
  • Slack review notifications now use clearer plain-text formatting with link and media previews enabled.

• Documentation

  • Updated test-generation guidance and workflow timeout documentation to reflect the expanded limits.

[coderabbitai]

📝 Walkthrough

Walkthrough

The PR increases testbot execution defaults, condenses Markdown coverage reports to uncovered ranges, and changes Slack review messages from Block Kit sections to plain text with link and media unfurling.

Changes

Testbot updates

Layer / File(s)	Summary
Workflow execution defaults .github/workflows/testbot.yaml, src/scripts/testbot/README.md	The workflow supports up to 3 targets and 400 agent turns. The documented workflow timeout increases to 60 minutes.
Coverage report output src/scripts/testbot/verify_coverage.py, src/scripts/testbot/tests/test_verify_coverage.py	Markdown reports show only uncovered ranges, cap the list at 8 entries, and report omitted ranges. Tests cover full, partial, and truncated output.
Slack review payload src/scripts/testbot/create_pr.py, src/scripts/testbot/tests/test_create_pr.py	Slack review payloads use plain text with link and media unfurling. Tests verify that Block Kit blocks are absent.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

• NVIDIA/OSMO#1287: Contains the same Slack payload and test changes.
• NVIDIA/OSMO#1289: Contains the same testbot target and turn configuration changes.

Suggested reviewers: adelbertc, aruns-nv

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the three main changes: larger testbot runs, Slack PR cards, and condensed coverage reports.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch jiaenr/testbot-improvements

---

_{Comment @coderabbitai help to get the list of available commands.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 70.95%. Comparing base (566e2e8) to head (d657c5d).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1290      +/-   ##
==========================================
+ Coverage   70.82%   70.95%   +0.12%     
==========================================
  Files         239      239              
  Lines       28501    28501              
  Branches     4298     4298              
==========================================
+ Hits        20186    20222      +36     
+ Misses       7491     7456      -35     
+ Partials      824      823       -1     

Flag	Coverage Δ
backend	73.65% <ø> (+0.13%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 2 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/testbot.yaml:
- Line 214: Replace direct interpolation of inputs.max_targets and
inputs.max_uncovered in the workflow command with validated environment
variables; validate both values as permitted numeric inputs before use, then
reference the environment variables in the shell command to prevent command
injection while preserving their defaults.

In `@src/scripts/testbot/tests/test_verify_coverage.py`:
- Around line 359-372: Update test_still_uncovered_ranges_are_capped to assert
that “line 8” is present and “line 9” is absent in the rendered markdown,
verifying exactly MAX_REPORTED_RANGES entries are included while retaining the
existing overflow and boundary assertions.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 55373ed9-efa8-4d2f-a05f-f7c4b0890985

📥 Commits

Reviewing files that changed from the base of the PR and between 5dc4c8f and d657c5d.

📒 Files selected for processing (6)

• .github/workflows/testbot.yaml
• src/scripts/testbot/README.md
• src/scripts/testbot/create_pr.py
• src/scripts/testbot/tests/test_create_pr.py
• src/scripts/testbot/tests/test_verify_coverage.py
• src/scripts/testbot/verify_coverage.py