Skip to content

fix(security): repair chained-cd gate test after #9089 helper removal - #9182

Merged
chenmingwei23 merged 1 commit into
mainfrom
fix/heal-chained-cd-gate-test
Sep 7, 2026
Merged

fix(security): repair chained-cd gate test after #9089 helper removal#9182
chenmingwei23 merged 1 commit into
mainfrom
fix/heal-chained-cd-gate-test

Conversation

@pepmach

@pepmach pepmach commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Broken main

d4cb9afc7 (#9089) removed the bash gate's cd/variable normalizer and the ~4.2k lines behind it, including _dir_holds_sensitive_leaf, _remember_bases and _MAX_TRACKED_BASES. test/test_security.py::TestIsSensitiveBashCommand::test_chained_cd_expansions_do_not_blow_up_the_gate still mock-patched the first and read the third, so it raised on every merge-ref run repo-wide:

AttributeError: <module 'kiro_crew.security'> does not have the attribute '_dir_holds_sensitive_leaf'

Reproduced on pristine origin/main before the fix (1 failed), and it is the only failure in its class (42 passed alongside it) — the break is this one test, not the removal.

The fix

The deleted helper is not restored. #9089 removed the walk deliberately, and TestTraversalSimulationIsGone.test_the_simulation_helpers_are_absent already pins that machinery absent by name. This extends the same philosophy instead of reintroducing a symbol only a mock needs.

The test is rewritten against the mechanism that survived_check_native_home_entry_then_fenced_read: one linear scan of the command's words for a home entry, then one per-word leaf decision (_fenced_relative_prefix, the successor to the deleted per-directory leaf probe) after it.

The old property was "the tracked base set is capped, so probe counts grow linearly rather than doubling per segment". There is no longer a set to cap, so three properties replace it — none of them a wall-clock budget, which is what killed the shape before this one on a loaded Windows runner:

  1. Behaviour, nothing patched. The same cd ${D:-foo} chain reads clean at 1/10/20/200 segments, and the surviving pass is still live on that exact shape — a fenced leaf named after a home entry through the identical chain (cd ~; cd ${D:-foo}; …; cat .aws/credentials) is still refused. That second half is what stops the test degrading into an "everything is allowed now" tautology.
  2. The passes are per-SUBJECT, not per-resolved-base. _sensitive_pattern_hit, _fence_hit_in_collapsed and _native_words are each invoked exactly (1, 1, 1) times, identically at 10 and at 200 segments. A reinstated per-segment resolver shows up here as a count that grows.
  3. The per-word leaf decision is linear. Measured 2n + 3 (10→23, 20→43, 40→83, 200→403), asserted as 0 < ten < twenty <= 2*ten + 4 plus a 200-segment ceiling.

Plus the absent-by-name assertion for _dir_holds_sensitive_leaf / _remember_bases / _MAX_TRACKED_BASES, so re-adding a base tracker fails loudly here rather than quietly reintroducing the doubling underneath the behavioural assertions.

The spies wrap the real functions rather than stubbing them, so every verdict asserted is the real verdict, and no filesystem probe and no clock is involved. Call time is ~0.13s.

Verification

Gate Result
the failing test on pristine origin/main 1 failed (AttributeError — break proven first)
the failing test after the fix 1 passed
test/test_security.py 1501 passed, 1 skipped
test/test_security_posture.py + test/test_spawn_audit.py 65 passed
all three together, post-format 1566 passed, 1 skipped
black --check --target-version py310, flake8, isort --check-only clean

Test-only diff (+80 / −25, one file, one commit). .kiro/settings/cli.json checked byte-clean against HEAD — no config side effect this run.

No visual surface: the change is confined to one unit test asserting invocation counts and verdicts of is_sensitive_bash_command. No gateway endpoint, MCP tool leg, CLI verb or frontend component is touched.

Pattern harvest

CI could not catch this. A mock.patch.object(mod, "gone") against a deleted attribute is not a static reference — no linter, type checker or import-time check sees it, and AttributeError only surfaces when that one test body executes. #9089 was green on its own PR because its merge ref predated nothing; the break appeared repo-wide the moment it landed, in a file the PR had already edited heavily.

Rule candidate: a PR that deletes a module attribute must grep tests for mock.patch references to that attribute's name — a mock of a deleted symbol is invisible to every static gate and only fails at test runtime.

Rule candidate: a PR deleting a helper MUST grep the test tree for mock.patch/patch.object/getattr/hasattr references to the removed name before merge — CI cannot catch a mock of a deleted symbol until that specific test runs, so a targeted or diff-scoped test selection will pass while main breaks.

Cheap enforcement: for every symbol removed from a src/ module in the diff, grep -rn "\"<name>\"\|'<name>'" test/ and require zero hits (or hits that assert absence, which is the legitimate case — this PR adds three of those).

Secondary observation: two consecutive rewrites of this same test (elapsed-seconds → mocked-probe cap) both pinned the property through internals, and both broke — once on a runner's I/O latency, once on a refactor. The shape that survives a refactor asserts the gate's verdict on the input plus a growth relation, with spies that wrap rather than replace.

#9089 removed the bash gate's cd/variable normalizer along with
_dir_holds_sensitive_leaf, _remember_bases and _MAX_TRACKED_BASES, but
test_chained_cd_expansions_do_not_blow_up_the_gate still mock.patch.object'd
_dir_holds_sensitive_leaf and read _MAX_TRACKED_BASES, so it raised
AttributeError on every merge-ref run repo-wide.

The deleted helper is NOT restored -- #9089 removed the walk deliberately, and
TestTraversalSimulationIsGone pins the same machinery absent by name. The test
is rewritten against the mechanism that survived,
_check_native_home_entry_then_fenced_read: one linear scan of the command's
words for a home entry, then one per-word leaf decision
(_fenced_relative_prefix) after it.

Three properties replace the deleted cap, none of them a wall-clock budget:
the same chained-cd inputs read clean at 1/10/20/200 segments while a fenced
leaf named after a home entry through the identical chain is still refused;
the pass-level matchers run once per SUBJECT, identically at 10 and 200
segments, so no per-segment path resolution survives to blow up; and the
per-word leaf decision grows linearly in the segment count (measured 2n+3).
The spies wrap the real functions rather than stubbing them, so every verdict
asserted is the real one and no filesystem probe is involved -- the test runs
in ~0.13s.
@pepmach
pepmach requested a review from a team as a code owner September 7, 2026 02:23
@pepmach
pepmach requested a review from CrysisDeu September 7, 2026 02:23
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Sep 7, 2026
@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — ✅ PASS

Design-level review of e63aab685688b4cfa7daffef8235e0cbdbedf436 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Test-only repair asserts the boundedness property against the surviving mechanism with real (wrapped, not stubbed) verdicts, plus absent-by-name pins per existing convention — sound and proportionate.

[DESIGN-REVIEWED] e63aab6

@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of e63aab685688b4cfa7daffef8235e0cbdbedf436 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] e63aab6

False positive or not applicable? A repository writer can comment:
/ai-review override gpt e63aab685688b4cfa7daffef8235e0cbdbedf436: <one-sentence reason>

@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Opus 4.8 Review — ✅ no blocking findings

Reviewed e63aab685688b4cfa7daffef8235e0cbdbedf436 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] e63aab6

Verdict parsed from the review's SHA-scoped output markers for commit e63aab685688b4cfa7daffef8235e0cbdbedf436.

False positive or not applicable? A repository writer can comment:
/ai-review override fable e63aab685688b4cfa7daffef8235e0cbdbedf436: <one-sentence reason>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Sep 7, 2026
@chenmingwei23
chenmingwei23 enabled auto-merge (squash) September 7, 2026 03:04

@chenmingwei23 chenmingwei23 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: test (1 file). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, SAST annotations clean, security checklist all-NO, AI reviewers green. Category: test-only change to test/test_security.py, repairing the chained-cd gate test after #9089 removed the walk helpers it mocked; no production code touched.

@chenmingwei23
chenmingwei23 merged commit cbdd4a5 into main Sep 7, 2026
71 of 72 checks passed
@chenmingwei23
chenmingwei23 deleted the fix/heal-chained-cd-gate-test branch September 7, 2026 03:04
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Sep 7, 2026

@dwu96 dwu96 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: test (1 file). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, SAST annotations clean, security checklist all-NO, AI reviewers green. Category: test-files-only repair of test/test_security.py after #9089 removed _dir_holds_sensitive_leaf / _remember_bases / _MAX_TRACKED_BASES (absence verified against current main source, and every function the new spies wrap is present); no production code touched and the gate still refuses a fenced .aws/credentials read.

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: test (1 file). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, SAST annotations clean, security checklist all-NO, AI reviewers green. Category: test-only edit to test/test_security.py -- rewrites test_chained_cd_expansions_do_not_blow_up_the_gate, which mock-patched _dir_holds_sensitive_leaf / _remember_bases / _MAX_TRACKED_BASES after #9089 deleted them, to assert the surviving _check_native_home_entry_then_fenced_read scan with wrapped (real-verdict) spies plus a name-absence pin; no production code touched.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants