fix(ingest): preserve origin title provenance - #3892
Conversation
Assign typed origin provenance only to provider-authored titles from Grok, Antigravity, browser capture, and Hermes. Keep heuristic filename and page-title fallbacks untrusted. Add parser assertions and a shared parser-to-public-surface regression route with a provenance-removal mutation.\n\nRef polylogue-o5smo.
Record polylogue-o5smo as satisfied by the parser-to-public-surface implementation, focused production-route coverage, red provenance mutation, and clean quick gate.
|
Important Review skippedNo new commits to review since the last review. ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughThe change records title provenance from browser capture and origin parsers through storage and public session conversion. Provider titles are trusted and marked separately from page and session-ID fallbacks. Schema version 67 requires semantic reparse. ChangesTitle provenance
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant OriginParser
participant BrowserCaptureParser
participant ArchiveStorage
participant PublicSession
OriginParser->>BrowserCaptureParser: title and TitleSource.ORIGIN
BrowserCaptureParser->>ArchiveStorage: parsed session with title provenance
ArchiveStorage->>PublicSession: stored title and provenance
PublicSession-->>ArchiveStorage: public session summary
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fe8c89666b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| source_name=Provider.GROK, | ||
| provider_session_id=fallback_id, | ||
| title=title, | ||
| title_source=TitleSource.ORIGIN if provider_title else None, |
There was a problem hiding this comment.
When upgrading an existing v66 archive whose raw records have already been parsed, these parser-only assignments leave INDEX_SCHEMA_VERSION unchanged, so normal startup accepts the existing index.db and the parse backlog excludes rows with parsed_at_ms; consequently the stored title_source remains NULL and the public title stays suppressed for all previously ingested Grok, Antigravity, browser-capture, and Hermes sessions. Increment the index version and declare the delta as SEMANTIC_REPARSE so those sessions are rebuilt from their retained raw evidence rather than fixing only new or explicitly reprocessed imports.
AGENTS.md reference: AGENTS.md:L193-L200
Useful? React with 👍 / 👎.
| source_name=provider, | ||
| provider_session_id=provider_session_id, | ||
| title=envelope.session.title or envelope.provenance.page_title or provider_session_id, | ||
| title_source=TitleSource.ORIGIN if envelope.session.title else None, |
There was a problem hiding this comment.
When a browser adapter cannot extract a provider title, the actual extension producer still sets session.title to document.title or the native session ID in browser-extension/src/common.js; therefore this condition marks exactly those page-title/identifier fallbacks as TitleSource.ORIGIN, bypassing the public suppression policy. The new test avoids the production shape by deleting session.title manually, but a real no-title capture will retain it, so provenance must be carried separately from the producer or compared against the known page-title/ID fallbacks before assigning ORIGIN.
Useful? React with 👍 / 👎.
| _apply_browser_capture_session_kind( | ||
| _merge_envelope_attachments(parse_chatgpt(raw_provider_payload, provider_session_id), envelope), | ||
| _merge_envelope_attachments( | ||
| _merge_envelope_native_metadata(parse_chatgpt(raw_provider_payload, provider_session_id), envelope), |
There was a problem hiding this comment.
When a native ChatGPT payload omits update_time, parse_chatgpt correctly leaves updated_at unset, but this newly enabled helper also copies envelope.session.updated_at; the production extension always constructs that field as updatedAt || now, so the session is now falsely stamped with browser capture time and downstream timeline/date ordering treats it as provider evidence. Limit the ChatGPT merge to the trusted title fields, or preserve a distinct capture-time field instead of passing the whole metadata merge.
Useful? React with 👍 / 👎.
Problem Browser envelopes carried fallback page titles and session ids in the same field as provider-authored titles. Native ChatGPT parsing also merged the envelope capture timestamp as provider update time. Existing derived rows could not be repaired without replay. What changed Add a typed title-source field to live and backfill producers, trust only provider evidence in the parser, and keep native ChatGPT timestamp authority inside its native payload. Declare the semantic parser change as index v67 and refresh generated demo evidence. Pin the v64-to-v65 executor fixture to its stated target so later semantic versions do not invalidate it. Verification - 157 focused Python tests passed - 65 browser-extension tests passed - browser-extension ESLint passed - devtools lab policy schema-versioning passed - devtools verify --quick passed all 24 steps Ref polylogue-o5smo. Co-Authored-By: Codex <noreply@openai.com>
|
@coderabbitai review |
|
@codex review |
|
|
Codex Review: Didn't find any major issues. 🎉 Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
1 participant
Summary
Preserve provider-authored titles from Grok, Antigravity, browser capture, and Hermes by assigning the existing typed origin provenance at each parser boundary. Heuristic identifiers, artifact names, and browser page titles remain untrusted fallbacks.
Problem
The public title-presence policy intentionally suppresses untyped title text. Four active parsers supplied genuine provider-authored titles without
ParsedSession.title_source, so archive summaries and display labels degraded to generated structural labels after otherwise successful ingest. Refpolylogue-o5smo.Solution
The Grok, Antigravity, and Hermes parsers now assign
TitleSource.ORIGINonly to provider-authored values. Live and backfill browser producers emit a typedprovider,page, orsession-idtitle source; the parser trusts onlyproviderand handles old envelopes conservatively. Native ChatGPT may consume that trusted title without inheriting the browser capture timestamp as provider update time. Index v67 declares the stored parser-semantic change as requiring raw replay. A shared production-route test carries each repaired origin through materialization,ArchiveStore,SessionFilter, and the public summary, while a provenance-removal mutation proves that the public title degrades.Verification
devtools test tests/unit/sources/test_browser_capture.py tests/unit/storage/test_title_provenance_origins.py tests/unit/sources/parsers/test_antigravity.py tests/unit/sources/parsers/test_grok.py tests/unit/sources/parsers/test_hermes_state.py tests/unit/sources/parsers/test_origin_regression_pack.py: 122 passed in 26.50 seconds.devtools test tests/unit/storage/test_index_fast_forward_lifecycle.py tests/unit/storage/test_index_fast_forward_executor.py tests/unit/devtools/test_verify_schema_upgrade_lane.py: 35 passed in 2.55 seconds.npm test -- --run tests/common_envelope_boundary.test.js tests/backfill.test.js: 65 passed.npm run lint -- --quiet: passed.devtools lab policy schema-versioning: schema evolution policy intact.devtools verify --quick: all 24 checks passed in 97.09 seconds.No live archive or production state was mutated.
Summary by CodeRabbit
New Features
Bug Fixes
Documentation