Skip to content

Add read-only embedding inspection and population planning #55

Description

@gkennos

Pre-flight

  • I searched existing issues and this has not been requested before.

Problem or motivation

Add supported APIs for inspecting an embedding store and planning vocabulary population without changing the store.

This allows setup tools to show what is configured, which embeddings already exist, and what needs to be populated or refreshed.

Proposed solution

  • Inspect whether an embedding store is initialise & registered embedding models
  • Read stored concept IDs
  • Inspect existing PostgreSQL indexes
  • Generate reviewed index-removal statements
  • Explicitly initialize an embedding store as a separate writable operation
  • Plan population by vocabulary and domain
  • Stream large CDM and embedding-store reads in bounded batches.
  • Reprocess concepts when their stored metadata no longer matches the CDM.

Alternatives considered

No response

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions