test(arsenal): lock the advertised tool count to the real callable surface#41
Open
Pazificateur69 wants to merge 1 commit into
Open
test(arsenal): lock the advertised tool count to the real callable surface#41Pazificateur69 wants to merge 1 commit into
Pazificateur69 wants to merge 1 commit into
Conversation
…rface
The arsenal advertises "83 tools" and verify-claims defends it by counting
`id:`/`name:` source lines — a source-line count that would keep passing even
if an entry became uncallable. Add a guard that ties the number to the real
registered arrays (TOOL_ADAPTERS + BUILTIN_TOOLS + EXTERNAL_TOOLS === 83) and
pins the invariant that makes the count honest:
an adapter is off the generic callable surface (adapterToCustomTool → null)
IFF it is explicitly gated by execution mode (catalog_only / import_only).
So a tool can be counted-but-not-generically-callable only on purpose —
metasploit / hydra (catalog_only, reachable solely via the approval-gated
post-ex drivers) and bloodhound (import_only) — never through a silent DEFAULT
or mint slip. Also asserts isMintable agrees with actual callability, the gated
set is exactly {metasploit, hydra, bloodhound}, and every minted adapter has a
real handler and a unique name. Sibling of no-phantom-tools / stub-honesty.
Contributor
|
The invariant tests here (mintable ⇔ callable, no silently-dead / wrongly-gated) are a genuinely good guard. But the two hardcoded pins are already stale against current
Fix: rebase and update both constants — or better, derive the count and gated set from the array lengths / a flag predicate so they don't need hand-updating every time a tool is added (which is the brittleness this test is otherwise trying to prevent). |
Contributor
|
Please rebase this branch onto current This should be a tiny test-focused PR, but the branch currently carries unrelated stale-base deletions of benchmark corpora, scripts, provenance docs, and tests. Suggested agent repair path:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The arsenal advertises "83 tools" and
verify-claimsdefends it by countingid:/name:source lines — a source-line count that would keep passing even if an entry became uncallable (e.g.bloodhoundis counted butimport_only, never a generic callable tool).Change
Adds
src/__tests__/arsenal-count-honesty.test.tsthat ties the number to the real registered arrays (TOOL_ADAPTERS + BUILTIN_TOOLS + EXTERNAL_TOOLS === 83) and pins the invariant that makes the count honest:So a tool can be counted-but-not-generically-callable only on purpose —
metasploit/hydra(catalog_only, reachable solely via the approval-gated post-ex drivers) andbloodhound(import_only) — never through a silent DEFAULT / mint slip. Also assertsisMintableagrees with actual callability, the gated set is exactly{metasploit, hydra, bloodhound}, and every minted adapter has a real handler + a unique name. Sibling ofno-phantom-tools/stub-honesty.Pure test — no behavior change.
Review standard (all green locally)
npm run typecheck·npm test·npm run doctor·npm run test:gate·npm run smoke