feat(project-graph): LLM-grounded extractor with enforced segment citations (Phase 2)#931
Open
seonghobae wants to merge 1 commit into
Open
Conversation
…ations Phase 2 of the project-graph activation plan (stacked on the Phase 1 import wiring): upgrades extraction from the deterministic keyword baseline to an LLM-backed extractor that cannot fabricate. - services/project_graph/llm_extractor.py: structured-output extraction over the imported email's content segments. Hard grounding rule: every object must cite content_segment_uids present in the input; objects with unknown, missing, or partial-unknown citations are DROPPED. Unknown object types dropped; confidence clamped to [0,1]; segment text passed as JSON data (prompt-injection hardening, mirrors api/llm.py); base_url goes through the SSRF-validated provider client; segments capped (40 x 2000 chars) to bound tokens. - Selection: PROJECT_GRAPH_EXTRACTOR=keyword|llm (default keyword -> zero behavior change). The llm path reuses the import's OpenAI-compatible provider credentials and falls back to the keyword extractor on any failure, so the projection is never lost to a provider outage. Tests (7): grounded mapping + evidence edges; hallucinated/empty citations dropped; unknown type dropped + confidence clamped; empty segments short- circuit without an LLM call; selection uses llm when configured, falls back to keyword on llm failure, defaults to keyword. Regressions green (18). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017RkKdtHRLG4wSLh6PVsp8J
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What (stacked on #927 — Phase 2 of #925)
Upgrades project-graph extraction from the deterministic keyword baseline to an LLM-grounded extractor that cannot fabricate:
content_segment_uids present in the input segments — objects with unknown/missing/partially-unknown citations are dropped. The model cannot introduce uncited domain claims into the graph.object_typedropped;confidenceclamped to [0,1]; titles bounded to the DB column limit.api/llm.py);base_urlgoes through the SSRF-validated provider client; request bounded to 40 segments × 2000 chars.PROJECT_GRAPH_EXTRACTOR=keyword|llm(defaultkeyword→ zero behavior change). Thellmpath reuses the import's OpenAI-compatible provider credentials and falls back to keyword on any failure — the projection is never lost to a provider outage.Verification (local — 7 new tests + 18 regressions green)
grounded mapping + evidence edges · hallucinated/empty citations dropped · unknown type dropped + confidence clamped · empty segments short-circuit (no LLM call) · selection: llm-when-configured / keyword-fallback-on-failure / keyword-default.
Follow-up (Phase 3, per #925)
Corrections → per-tenant eval set + calibrated confidence.
🤖 Generated with Claude Code