KnowledgeGraph.get_num_ancestors crashes on large candidate sets - needs batching of the IN clause

[gkennos]

Pre-flight

• I searched existing issues and this is not a duplicate.

Summary

get_num_ancestors builds a single SQL query with one bound parameter per input concept_id, via Concept.concept_id.in_(concept_ids).

When the candidate set passed in is large enough (tens of thousands of concepts), the resulting query exceeds PostgreSQL/psycopg's hard limit of 65,535 bound parameters per statement, and the call fails outright rather than degrading gracefully.

Reproduction

groundworkers.adapters.omop_graph.run_ground_tier
→ omop_graph.reasoning.grounding.ground_term
→ omop_graph.graph.scoring.score_standard_concepts
→ omop_graph.graph.kg.KnowledgeGraph.get_num_ancestors(concept_ids)
→ omop_graph.graph.queries.q_concept_num_ancestors(concept_ids)
→ session.execute(select(...).where(Concept.concept_id.in_(concept_ids))...)
→ psycopg.OperationalError (>65,535 parameters)

Actual behaviour

get_num_ancestors (kg.py:769) passes the full concept_ids tuple through to this query in one call, with no chunking

Expected behaviour

Batch get_num_ancestors: chunk concept_ids into groups safely under the parameter limit (e.g. 20,000) and merge the resulting dicts.

Error output

psycopg.OperationalError: sending query and params failed: number of parameters must be between 0 and 65535

System info