[KatalKavya96]

Summary

• Fixes security-issue-import: Step 2 thread-ID de-dup misses tracked threads (backtick-wrapped ids not found by search) #1730: GitHub issue search can miss security mailing-list thread IDs wrapped in Markdown backticks, causing previously imported reports to be treated as new.
• Replace search-based exact-ID deduplication with deterministic matching against an authorized, bounded tracker issue-body index, covering Gmail IDs, PonyMail archive IDs, root Message-IDs, and secondary threads.
• Fail closed on incomplete or ambiguous results, preserve least-privilege caller permissions, and add regression tests and updated evaluation fixtures.

Type of change

• Skill change (.claude/skills/<name>/) — eval fixtures updated below
• Tool / bridge contract (tools/<system>/*.md)
• Python package (tools/*/ with pyproject.toml)
• Groovy reference impl
• Cross-cutting (RFC, AGENTS.md, sandbox, privacy-LLM)
• Documentation (docs/, README.md, CONTRIBUTING.md)
• Project template (projects/_template/)
• CI / dev loop (prek, workflows, validators)
• Other:

Test plan

• prek run --all-files passes — all hooks green.
• For Python packages touched: uv run pytest / ruff check / mypy passes — verified through workspace hooks and targeted package tests.
• For Groovy bridges touched: command-line invocation tested end-to-end
• For skill changes: eval suite passes for the affected skill.
  • security-issue-import/step-2-dedup: 4/4 passed, 0 failed, 0 manual, 0 errored.
• For skill behaviour changes: a new or updated eval fixture is included in this PR.
  • Updated three existing fixtures to use dedup-index results.
  • Added case-4-backtick-threadid-match to reproduce the reported failure.
• Other:
  • Tested exact identifier matching, PonyMail URLs, secondary threads, missing issue bodies, ambiguous identities, and truncated indexes.
  • Verified unauthorized callers are refused, explicitly authorized read operations are allowed, and the ordinary issue-list operation does not expose bodies.
  • Verified generated vetted-ops plugin mirrors match their source files.

RFC-AI-0004 compliance

• HITL — any required caller-policy change requires explicit user approval; no new mutation is introduced.
• Sandbox — tracker access remains restricted to the vetted-ops execution boundary; no unrestricted host access is added.
• Vendor neutrality — existing adopter placeholders are preserved in skill and tool instructions; check-placeholders passes.
• Conversational + correctable — agentic-override path documented if behaviour is adopter-tunable
• Write-access discipline — the new operation is read-only; it does not create tracker issues or send outbound messages.
• Privacy LLM — private tracker issue bodies are saved in the controlled workspace rather than printed into model context; matching results expose only issue summaries.

Linked issues

Fixes #1730.

Notes for reviewers (optional)

Implementation: The new issue-import dedup-index command reads a saved tracker index and compares identifiers against the authoritative Security mailing list thread field. A dedicated issue-list-with-bodies vetted operation requires explicit caller authorization.

Safety: A missing body, ambiguous mailing-thread identity, or potentially truncated 1000-issue index prevents negative deduplication claims. Jira and local-files backends are explicitly unsupported by this new body-index operation rather than silently treated as having no matches.

Trade-off: The bounded index avoids relying on GitHub's inconsistent search indexing, but trackers with 1000 or more returned issues require a further completeness-safe solution. This PR prioritizes preventing duplicate security imports over automatically proceeding when completeness cannot be established.

Validation scope: The four model evaluations use mocked tool outputs; deterministic Python tests separately verify the matching logic and vetted-ops authorization boundaries.

[KatalKavya96]

@potiuk ,

Implemented the fix for #1730 and completed validation.

The security-import workflow now uses deterministic matching against an authorized tracker body index instead of relying on GitHub issue search, which can miss backtick-wrapped thread IDs.

Validation:

• prek run --all-files — all hooks passed.
• Full workspace tests, Ruff, and mypy — passed.
• Security-import Step 2 model evaluation — 4/4 passed.
• Added regression coverage for backtick-wrapped Gmail IDs, archive identifiers, secondary threads, incomplete indexes, and caller authorization.

The implementation also fails closed when tracker completeness cannot be established, preventing unsafe duplicate imports.

Thanks!

[potiuk]

Replacing body search with a deterministic match on the authoritative field is the right fix for #1730, and the fail-closed handling of missing bodies, ambiguous fields and a full index is good — but the matcher still misses three identifier shapes the field legitimately records, which reintroduces the same duplicate-import false negative.

The index matcher misses identifiers the field records (tools/security-management/src/security_management/issue_import/dedup.py:166)

identity = threads.classify(field_value)

threads.classify picks the primary reply thread; it does not enumerate every identifier in the field, so three recorded shapes come back complete: true, untracked: [<id>] — the false negative this PR exists to prevent.

Three cases, all reproducible with the PR's own field() helper:

1. Maildir ids. The maildir backend's file name goes through the same Gmail thread `<threadId>` template (outcomes/mailing-thread.md), but threads._THREAD_ID is [A-Za-z0-9_-]{6,} and stops at the first . — 1700000000.M1P2.host,U=12:2,S is never matched, though valid_ids accepts it.
2. Sentinel first line. classify returns early on N/A — opened from public PR / N/A — imported from markdown file without reading the lines below, so a tracker issue-deduplicate merged from a PR import and a mail import loses its mail thread ids.
3. Several roots. classify keeps only the first Root Message-ID; merge.py unions both trackers' fields, so the second root is dropped.

Rather than reuse classify, scan every line of the field for every identifier shape valid_ids accepts (Gmail/maildir token with the _OPAQUE_ID character set, every /thread/<id>, every Root Message-ID), regardless of sentinels — and add a test for each case.

Flagged by codex (adversarial review); cross-checked.

Smaller observations

• tools/security-management/README.md:122 — nothing calls dedup-queries / dedup-attribute after this PR; remove them (code, README rows, tests) here, or name the follow-up in the PR body
• See inline comments on plugins/magpie-security/skills/issue-import/existing-tracker-dedup.md:18.

---

This review was drafted by an AI-assisted tool and
confirmed by an Apache Magpie maintainer. The findings
below are observations, not blockers; an Apache Magpie
maintainer — a real person — will take the next look at the
PR. If you think a finding is mis-applied, please reply on
the PR and a maintainer will weigh in.

More on how Apache Magpie handles maintainer review:
Contributing guide.

[potiuk]

threads.classify picks the primary reply thread rather than listing every identifier in the field, so a maildir id (stops at the first .), threads recorded below a N/A — opened from public PR / markdown sentinel, and any second Root Message-ID from a merged tracker come back complete: true, untracked: [<id>]. Scanning every line for every shape valid_ids accepts would close all three; details in the review body.

[potiuk]

Please spell this as uv run --project ~/.claude/magpie/vetted-ops vetted-op-read --caller security-issue-import …, as the other runnable blocks do (AGENTS.md → Local setup, golden rule): a bare vetted-op-read is not on PATH and is not what the sandbox exclusion matches. Also, <saved-index-path> below is never defined — it is <workspace>/saved/security-import-index.json.

[potiuk]

Marking this as changes requested, so the PR's status matches where it stands: the points in my earlier review (#1750 (review)) are still open and need a change, or a reply on their threads. This review adds nothing new to them.

---

This review was drafted by an AI-assisted tool and
confirmed by an Apache Magpie maintainer. After you've
addressed the points above and pushed an update, an Apache Magpie
maintainer — a real person — will take the next look
at the PR. The findings cite the project's review criteria;
if you think one of them is mis-applied, please reply on the
PR and a maintainer will weigh in.

More on how Apache Magpie handles maintainer review:
Contributing guide.