develop: add PAT permission guidance and verification by efraespada · Pull Request #400 · vypdev/copilot

efraespada · 2026-09-20T17:32:57Z

This change adds evidence-based PAT permission guidance and verification to setup, while strengthening Bugbot’s exhaustive review flow. It helps repository owners configure only the permissions needed by enabled workflows and makes review results more resilient to incomplete or malformed analysis.

What changed

Setup now inspects repository and organization permission evidence, reports actionable PAT guidance, and fails closed when required checks are inconclusive.
Workflow PAT recommendations are scoped to enabled capabilities, with examples and documentation updated to explain permission requirements and recovery steps.
Bugbot partitions large reviews, validates partition assignments and output, retries transient analysis failures, and reconciles review state in bounded batches.
Documentation, specifications, generated bundles, and tests are updated for the setup and Bugbot behavior changes.

docs-page · 2026-09-20T17:32:59Z

To preview the documentation for this pull request, visit the following URL:

docs.page/vypdev/copilot~400

codecov-commenter · 2026-09-20T17:37:53Z

Codecov Report

❌ Patch coverage is 98.99699% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.77%. Comparing base (1ad0adc) to head (7261967).

Files with missing lines	Patch %	Lines
...lication/policies/setup_token_permission_policy.ts	97.84%	1 Missing and 1 partial ⚠️
...tructure/setup_remote_credential_health_adapter.ts	94.44%	1 Missing and 1 partial ⚠️
...astructure/setup_token_permission_query_adapter.ts	98.49%	1 Missing and 1 partial ⚠️
...ication/usecases/actions/initial_setup_workflow.ts	94.11%	0 Missing and 1 partial ⚠️
...on/usecases/actions/setup_resource_provisioning.ts	95.23%	0 Missing and 1 partial ⚠️
...pplication/usecases/setup/setup_wizard_use_case.ts	95.23%	0 Missing and 1 partial ⚠️
...cases/steps/commit/bugbot/query_bugbot_findings.ts	94.44%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #400      +/-   ##
==========================================
+ Coverage   91.16%   91.77%   +0.61%     
==========================================
  Files         676      686      +10     
  Lines       19635    20484     +849     
  Branches     5257     5582     +325     
==========================================
+ Hits        17900    18799     +899     
+ Misses        721      690      -31     
+ Partials     1014      995      -19

Files with missing lines	Coverage Δ
src/actions/github_action.ts	`95.37% <100.00%> (+3.21%)`	⬆️
...tion/policies/application_error_message_catalog.ts	`100.00% <ø> (ø)`
...application/policies/bounded_concurrency_policy.ts	`100.00% <100.00%> (ø)`
...plication/policies/bugbot_diff_partition_policy.ts	`100.00% <100.00%> (ø)`
...ion/policies/bugbot_partition_completion_policy.ts	`100.00% <100.00%> (ø)`
src/application/policies/file_ignore_policy.ts	`96.55% <100.00%> (ø)`
...ion/policies/setup_configuration_storage_policy.ts	`93.69% <100.00%> (+6.55%)`	⬆️
...policies/setup_token_permission_evidence_policy.ts	`100.00% <100.00%> (ø)`
...pplication/usecases/comment_automation_use_case.ts	`90.90% <100.00%> (ø)`
src/application/usecases/commit_use_case.ts	`100.00% <100.00%> (ø)`
... and 49 more

... and 2 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

vypbot

🤖 Bugbot review snapshot

Bugbot reported 1 potential problem when commit 185e2cd was analyzed. This snapshot is historical; use the status block above for current state. 1 finding is linked to changed code.

Findings

medium: Final setup PAT permission audit runs after remote preflight — src/cli/commands/setup.ts:152

vypbot · 2026-09-20T17:42:59Z

Bugbot: review complete

Pull request · Verified commit · Workflow run

vypbot

🤖 Bugbot review snapshot

Bugbot reported 1 potential problem when commit b0977e7 was analyzed. This snapshot is historical; use the status block above for current state. 1 finding is linked to changed code.

Findings

medium: Unavailable repository Secret inventory is treated as empty — src/data/repository/repository_variables_repository.ts:48

vypbot

🤖 Bugbot review snapshot

Bugbot reported 1 potential problem when commit a8ac8f6 was analyzed. This snapshot is historical; use the status block above for current state. 1 finding is linked to changed code.

Findings

medium: Organization-only storage is blocked by unrelated repository inventory access — src/application/policies/setup_configuration_storage_policy.ts:130

vypbot

🤖 Bugbot review snapshot

Bugbot reported 2 potential problems when commit 021c842 was analyzed. This snapshot is historical; use the status block above for current state. 2 findings are linked to changed code.

Findings

medium: Rate-limit responses are reported as missing permissions — src/infrastructure/setup_token_permission_query_adapter.ts:57
medium: Final setup-PAT permission report can be skipped — src/cli/commands/setup.ts:158

vypbot

🤖 Bugbot review snapshot

Bugbot reported 1 potential problem when commit fd095f2 was analyzed. This snapshot is historical; use the status block above for current state. 1 finding is linked to changed code.

Findings

medium: Unvalidated provider head SHA is interpolated into trusted prompt text — src/application/policies/bugbot_diff_partition_policy.ts:149

vypbot

🤖 Bugbot review snapshot

Bugbot reported 5 potential problems when commit 4c7edc3 was analyzed. This snapshot is historical; use the status block above for current state. 5 findings are linked to changed code.

Findings

high: Members-only authorization is bypassed for dynamic locale catalogs — src/actions/github_action.ts:98
medium: Organization member checks make valid workflow PATs unverifiable — src/application/policies/setup_token_permission_policy.ts:183
medium: Raw diff budget does not bound normalized prompt size — src/application/policies/bugbot_diff_partition_policy.ts:85
medium: Installed selected-branch health workflow is treated as unavailable — src/data/repository/github/credential_health_workflow_visibility.ts:35
low: Disabled assignment can still fail on a missing target number — src/application/usecases/steps/issue/assign_members_workflow.ts:34

efraespada · 2026-09-23T13:42:06Z

/copilot recheck

vypbot

🤖 Bugbot review snapshot

Bugbot reported 1 potential problem when commit b859c44 was analyzed. This snapshot is historical; use the status block above for current state. 1 finding is linked to changed code.

Findings

low: Dynamic catalog incorrectly requires file-modification permission — src/actions/github_action.ts:95

vypbot · 2026-09-23T13:45:28Z

Request could not be completed

Action: Retry when the provider is available.

Retained state: Existing persisted state and completed external effects were preserved.

Error code: provider.unavailable

Retryable: Yes

Reference: d38fc8fb-13b6-48ec-853b-0d259fd71f60

vypbot

🤖 Bugbot review snapshot

Bugbot reported 7 potential problems when commit 593b549 was analyzed. This snapshot is historical; use the status block above for current state. 7 findings are linked to changed code.

Findings

medium: Organization Members operational evidence is contradicted by the canonical rule — specs/setup-pat-permission-guidance-and-verification.md:409
medium: Issue permission is required when issue automation is disabled — src/application/policies/setup_token_permission_policy.ts:76
medium: Do not dispatch a workflow that exists only on the selected ref — src/infrastructure/setup_remote_credential_health_adapter.ts:102
low: Definition of Done reports the wrong test count — specs/setup-pat-permission-guidance-and-verification.md:896
low: Definition of Done cites the wrong test-budget floor — specs/bugbot-exhaustive-partitioned-analysis.md:697
low: Fenced code text can satisfy the PAT prerequisite check — scripts/documentation_pat_exception_policy.cjs:3
low: Credential keep rule contradicts the mandatory workflow PAT re-entry — specs/setup-configuration-credentials-and-doctor.md:144

vypbot

🤖 Bugbot review snapshot

Bugbot reported 3 potential problems when commit 2cced48 was analyzed. This snapshot is historical; use the status block above for current state. 3 findings are linked to changed code.

Findings

medium: Temporary bootstrap can delete a workflow created by another actor — specs/setup-pat-permission-guidance-and-verification.md:259
medium: Validate remote inventory before copying setup files — src/application/usecases/actions/initial_setup_workflow.ts:90
low: Indented shell examples bypass PAT exception validation — scripts/validate-documentation-contract.cjs:270

vypbot

🤖 Bugbot review snapshot

Bugbot reported 1 potential problem when commit 0ee8e03 was analyzed. This snapshot is historical; use the status block above for current state. 1 finding is linked to changed code.

Findings

low: README shell examples bypass the PAT exception check — scripts/validate-documentation-contract.cjs:269

vypbot

🤖 Bugbot review snapshot

Bugbot reported 1 potential problem when commit dc33c34 was analyzed. This snapshot is historical; use the status block above for current state. 1 finding is linked to changed code.

Findings

medium: Unverifiable write permissions can bypass required acknowledgement — specs/setup-pat-permission-guidance-and-verification.md:727

vypbot

🤖 Bugbot review snapshot

Bugbot reported 2 potential problems when commit a5c1aba was analyzed. This snapshot is historical; use the status block above for current state. 2 findings are linked to changed code.

Findings

medium: Any repository read can be marked operationally available — src/application/policies/setup_token_permission_evidence_policy.ts:49
medium: Organization targets can be masked by unseen repository resources — src/application/usecases/actions/setup_resource_provisioning.ts:153

vypbot

🤖 Bugbot review snapshot

Bugbot reported 1 potential problem when commit 6fa92ef was analyzed. This snapshot is historical; use the status block above for current state. 1 finding is linked to changed code.

Findings

medium: Public members listing can falsely satisfy the Members read requirement — specs/setup-pat-permission-guidance-and-verification.md:411

vypbot

🤖 Bugbot review snapshot

Bugbot reported 1 potential problem when commit fb9a447 was analyzed. This snapshot is historical; use the status block above for current state. 1 finding is linked to changed code.

Findings

low: Partition test budget conflicts with companion specifications — specs/bugbot-exhaustive-partitioned-analysis.md:564

vypbot

🤖 Bugbot review snapshot

Bugbot reported 1 potential problem when commit e16e3db was analyzed. This snapshot is historical; use the status block above for current state. 1 finding is linked to changed code.

Findings

low: Definition of Done uses the wrong test budget — specs/agent-runtime-provider-and-model-routing.md:358

vypbot

🤖 Bugbot review snapshot

Bugbot reported 1 potential problem when commit cca631a was analyzed. This snapshot is historical; use the status block above for current state. 1 finding is linked to changed code.

Findings

low: Blockquoted shell examples bypass the PAT acknowledgement check — scripts/documentation_pat_exception_policy.cjs:49

develop: add PAT permission guidance and verification

61db6ea

vypbot added the state:ai-processing label Sep 20, 2026

vypbot assigned efraespada Sep 20, 2026

vypbot self-requested a review September 20, 2026 17:33

vypbot added this to vypdev Sep 20, 2026

vypbot moved this to In Progress in vypdev Sep 20, 2026

Merge remote-tracking branch 'origin/master' into develop

185e2cd

vypbot reviewed Sep 20, 2026

View reviewed changes

Comment thread src/cli/commands/setup.ts Outdated

vypbot added state:blocked state:awaiting-maintainer and removed state:ai-processing labels Sep 20, 2026

develop: close PAT permission coverage gaps

be48110

vypbot added state:ai-processing and removed state:ai-processing labels Sep 20, 2026

develop: bound unavailable setup inventory reads

b0977e7

vypbot added the state:ai-processing label Sep 20, 2026

vypbot reviewed Sep 20, 2026

View reviewed changes

Comment thread src/data/repository/repository_variables_repository.ts

vypbot removed the state:ai-processing label Sep 20, 2026

develop: fail closed on unavailable setup inventory

a8ac8f6

vypbot added the state:ai-processing label Sep 20, 2026

vypbot reviewed Sep 20, 2026

View reviewed changes

Comment thread src/application/policies/setup_configuration_storage_policy.ts Outdated

vypbot removed the state:ai-processing label Sep 20, 2026

develop: scope setup inventory validation

021c842

vypbot added the state:ai-processing label Sep 20, 2026

vypbot reviewed Sep 20, 2026

View reviewed changes

Comment thread src/infrastructure/setup_token_permission_query_adapter.ts Outdated

Comment thread src/cli/commands/setup.ts Outdated

vypbot removed the state:ai-processing label Sep 20, 2026

develop: classify setup permission failures safely

952b0d8

develop: close review identity and admission findings

fd095f2

vypbot reviewed Sep 23, 2026

View reviewed changes

Comment thread src/application/policies/bugbot_diff_partition_policy.ts Outdated

develop: validate Bugbot review head identities

4c7edc3

vypbot reviewed Sep 23, 2026

View reviewed changes

develop: close remaining review findings

b859c44

vypbot reviewed Sep 23, 2026

View reviewed changes

Comment thread src/actions/github_action.ts

develop: clarify member-only planner authorization

593b549