Skip to content

improvement(search): shorter live document IDs - #9794

Merged
waleedlatif1 merged 3 commits into
stagingfrom
improvement/live-search-short-ids-v2
Nov 8, 2026
Merged

waleedlatif1 merged 3 commits into
stagingfrom
improvement/live-search-short-ids-v2

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

  • Live search document IDs were live: + base64 of a JSON object naming the reader, scope, account and document (~270-375 chars, ~175-245 tokens). Clients re-read them every turn and copy them verbatim into reads
  • New searches mint live2: IDs: provider~account~id~container~kind~thread~revision~binding~checksum, ~65-135 chars (~47-64 tokens)
    • Reader and scope come from the request: the old fields were never an authorization boundary (unsigned), reads still re-resolve the account among the reader's own accounts in scope and re-verify the document
    • A 6-char checksum makes a truncated or edited copy ask for a new search instead of naming another document
    • Only the separator, % and control characters are escaped, so an ID is never longer than the old form and any string encodes
  • Old live: IDs still read, with their reader/scope check unchanged
  • Every ID comparison (filters, read check, continuation fingerprint, MCP batch dedupe, citation IDs) goes through a canonical document key, so old and new forms of one document match

Type of Change

  • Improvement

Testing

  • New reference.test.ts + updated application/filters/MCP server tests; 6 new tests seen failing on the previous encoding (truncation, edit, unpaired surrogate, length, mixed-form batch dedupe) and passing now
  • Targeted vitest: lib/sim-search/live, lib/knowledge/search, lib/knowledge/mcp, mothership knowledge tools, chat citations, message-content citations (1,356 passed); biome on changed files; type-check, audits and integration in CI
  • Paired Claude-as-MCP-client benchmark (Sonnet 5.5, prompt caching, 83 cases × 2, judged; vs two control runs of staging): input tokens per question −15k to −19k, model time −1.0 to −1.5 s, tool and schema errors to zero, correctness and completeness flat or up on both dev and held-out splits

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing (new tests pass the test-audit authoring gate)
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

🤖 Generated with Claude Code

Live search now mints live2: IDs: readable, ~-separated, percent-escaped provider fields with no reader or scope, about a quarter the length and tokens of the base64 JSON form. Reads derive the reader and scope from the request; account resolution and provider verification remain the authorization. Earlier live: IDs still read with their reader and scope checks. Document ID filters, Assistant scope intersection, and citation IDs compare by the decoded document identity, so both forms of one document match.
@vercel

vercel Bot commented Nov 7, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Nov 8, 2026 12:10am UTC

Request Review

@greptile-apps

greptile-apps Bot commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no new actionable issues were found.

Summary

This PR replaces long live document references with shorter live2: IDs.

  • Live search returns shorter document IDs that still work for reads.
  • Older and newer live IDs match as the same document.

No new actionable issues were found.

Reviews (3) · Last reviewed commit: "fix(search): bind live IDs to their read..." · Reviewed by Greptile

Comment thread apps/sim/lib/knowledge/mcp/server.ts Resolved

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 10 files

Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic

Comment thread apps/sim/lib/knowledge/mcp/server.ts Resolved
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 11 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Turn on auto-fix | Re-trigger cubic

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 8, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 9 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Turn on auto-fix | Re-trigger cubic

@waleedlatif1
waleedlatif1 merged commit 40d7474 into staging Nov 8, 2026
83 checks passed
@waleedlatif1
waleedlatif1 deleted the improvement/live-search-short-ids-v2 branch November 8, 2026 02:28

This branch was previously deployed

1 inactive deployment
Preview — 49c06dd1 Deployed Nov 8, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant