Skip to content

feat: cygnet.follow_many — batched FK navigation (N+1 → 1 round-trip) - #16

Merged
Xof merged 1 commit into
mainfrom
feat/follow-many
Jun 30, 2026
Merged

feat: cygnet.follow_many — batched FK navigation (N+1 → 1 round-trip)#16
Xof merged 1 commit into
mainfrom
feat/follow-many

Conversation

@Xof

@Xof Xof commented Jun 30, 2026

Copy link
Copy Markdown
Owner

What

Adds cygnet.follow_many(db, objs, fk_column) — the batched sibling of cygnet.follow. Following an FK for a whole collection previously meant looping follow, one round-trip per object (the classic N+1). follow_many does it in one WHERE pk = ANY($1) query and re-associates the targets back to the inputs.

logs = await cygnet.SELECT(db).FROM(LogTable)
accounts = await cygnet.follow_many(db, logs, LogTable.account_id)
# accounts[i] is logs[i]'s Account — or None if the FK is NULL / row missing.

Why this, and why now

This is the driver-agnostic read-throughput lever. A prior real-PG spike measured that end-to-end read time is ~85–92% driver decode + network round-trip and the positional hydrator already captured the hydration slice — so no faster hydrator (or Rust driver) moves the ceiling. Fewer round-trips is where the win is, and N+1 FK-loading is the most common source of extra round-trips. follow_many is the explicit cure (it fits the "littlest ORM" stance — a helper, not magic lazy-loading).

The whole FK-value list binds as a single array parameter (= ANY($1)), so there's no IN-list length limit and it's plan-cache friendly.

Behavior

  • Returns a list aligned to objs (input order); None for a NULL FK or a missing target row.
  • Deduplicates FK values into one query; objects sharing a value share the returned instance (a batching artifact — Cygnet has no identity map).
  • No query when objs is empty or every FK is None.
  • Validation mirrors follow() (FK column, object types), with object-type errors taking precedence (matched to follow).

Tests & verification

  • 11 unit tests (FakeDB): alignment, NULL→None, missing→None, dedup/shared-instance, empty + all-None no-query short-circuits, all validation paths, and the error-precedence guarantee.
  • 1 real-PG integration test in TestFollowRoundtrip — passed locally on Dockerised PG.
  • Full unit suite green (523), ruff + mypy clean.
  • Adversarial review found no correctness bugs; its validation-parity finding (FK-check vs type-check ordering) is fixed in this PR and pinned by a test.

Docs

README recipe (features follow_many, plus the underlying = ANY pattern and .stream() note), THEORY synthesis paragraph (round-trips are the read ceiling; batching is the lever), ARCHITECTURE component-map symbol.

…trip)

Following a foreign key for a whole collection meant looping cygnet.follow,
one round-trip per object (the classic N+1). Against real PG the per-query
round-trip dominates read cost (a measured spike put hydration at <15% of
end-to-end), so collapsing N lookups into one is the high-leverage,
driver-agnostic read-throughput win.

follow_many(db, objs, fk_column) issues a single WHERE pk = ANY($1) query
(the whole FK-value list bound as one array parameter -- no IN-length limit)
and returns the targets aligned to objs: input order, None for a NULL FK or
a missing row, and a shared instance for a shared FK value. It issues no
query when objs is empty or every FK is None. Validation mirrors follow()
(FK column, object types), with object-type errors taking precedence.

Tests: 11 unit (FakeDB) covering alignment, NULL/missing -> None, dedup,
the no-query short-circuits, validation + error precedence; 1 real-PG
integration test (passed on Dockerised PG). Docs: README recipe, THEORY
(round-trips are the read ceiling; batching is the lever), ARCHITECTURE
component map.
@Xof
Xof merged commit 30007d5 into main Jun 30, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant