perf: parallelize page-memory title_detection and node_assembly VLM calls

[suguanYang]

Summary

• Logfire staging analysis showed v2 (page-memory) ingestion jobs running ~7x slower than v1 (chunk) on average (185s vs 25s avg, 821s vs 110s p95). Two stages dominate: title_detection (avg 236s/job) and node_assembly (avg 143s/job).
• Root cause: both stages ran their per-page VLM (vision-language model) calls in a plain serial for loop with zero concurrency, unlike sibling stages in the same codebase (tag_pages, hierarchy-scope processing, asset summaries) that already use a bounded GeventPool idiom and get real parallelism out of it. This worker runs Celery with --pool=gevent, so the same idiom genuinely overlaps VLM round-trips instead of just interleaving.
• Changes:
  • PageMemoryConfig gains title_detection_concurrency and node_assembly_concurrency (default 4, mirroring scope_concurrency/tag_concurrency), threaded into the two stage call sites in memory_service.py.
  • page_tagger.tag_page_titles now runs its VLM calls through a GeventPool, mirroring tag_pages in the same file. Results are merged by page index after gevent.joinall, so there's no concurrent mutation of shared tag objects.
  • node_assembler.build_node_rows's two loops (page-text resolution, then node summaries) each get their own bounded pool. The second pool still runs strictly after the first joins, since node summaries read resolved page text. Node-summary results are written into an index-addressed list so row order is preserved regardless of greenlet completion order (existing tests assert row order).
  • _vlm_node_summary and resolve_page_text (node_assembler's two VLM call sites) now retry once on an empty result (5s backoff) before falling back to the existing silent-degradation behavior — previously a single transient failure would silently produce a degraded summary/empty OCR text with no retry at all.
• No new abstraction introduced: the GeventPool(size=min(n, len(items))) → pool.spawn → gevent.joinall pattern was already duplicated at 8 call sites with no shared helper; this adds 2 more inline copies rather than extracting a generic utility.
• Worst-case in-flight VLM calls stays at 4 (never stacked, since these stages run sequentially relative to each other and to per-scope work) — well under the httpx max_keepalive_connections=20 ceiling and DashScope's per-key RPM=300 limit.

Verification

• cd apps/worker && python -m pytest tests/contract/ -k "page_memory" -v → 68 passed (58 pre-existing + 10 new)
• cd apps/worker && python -m pytest tests/ -q → 134 passed, no regressions
• Manually verified PageMemoryConfig defaults, from_mapping overrides, and to_dict() round-trip for the two new fields
• New tests added:
  • test_page_memory_node_assembler_contract.py: row-order preservation under concurrent node summaries, retry-then-succeed and retry-exhausted cases for both _vlm_node_summary and resolve_page_text
  • test_page_memory_page_tagger_contract.py (new file): order preservation under concurrent title detection, image-missing skip behavior, no-op when no fat-leaf pages
• Not yet verified: actual wall-clock improvement in staging. Next step after merge/deploy is re-running the Logfire stage-timing queries used during this investigation (Stage completed: page_memory.title_detection / page_memory.node_assembly) to confirm the expected ~4x reduction, and watching for any new rate-limit warning volume from the added retries.

Deployment Notes

• No new environment variables — the two new concurrency knobs are PageMemoryConfig fields resolved from job metadata (default 4 when absent), consistent with existing sibling knobs.
• No database migrations, queue changes, or storage changes.
• Fully backwards compatible: existing jobs with no title_detection_concurrency/node_assembly_concurrency in their metadata fall back to the default of 4, preserving current behavior modulo the concurrency and retry changes.

Checklist

• Tests were added or updated when behavior changed
• Public docs, examples, or OpenAPI contracts were updated when needed (n/a — internal worker config, not part of the public API)
• Database migrations are idempotent and safe to deploy (n/a — no migrations)
• Logs, errors, and validation paths avoid leaking secrets or user data
• The pull request description explains any breaking or user-visible change (none — internal performance change only)