fix: isolate citation registries per pipeline

[ump45nose]

Summary

• isolate citation state behind a unique registry ID for each LightResearch pipeline run
• pass that ID through the generated tool metadata and release the registry when the pipeline completes
• add a focused regression test that interleaves two runs and verifies their citation counters remain independent

Root cause

init_citation_registry() reset a class-level dictionary shared by every request. Starting a second pipeline therefore erased the first pipeline's in-progress citation mappings.

Validation

• pytest -p no:cacheprovider tests/test_citation_registry.py -q
• ultrarag build examples/demos/LightResearch.yaml
• generated MCP metadata smoke for init, assign, and clear tool contracts
• ruff check --select I,F,E9 tests/test_citation_registry.py
• ruff check --select I servers/custom/src/custom.py

Fixes #394

[ump45nose]

@xhd0728 This focused LightResearch concurrency fix is ready for review. It scopes citation registries by pipeline-run ID, carries that ID through generated tool metadata, and releases the registry on completion; the regression interleaves two runs and verifies independent counters. The focused pytest, pipeline build, MCP metadata smoke, and targeted Ruff checks pass. Since you maintain and review servers/custom/src/custom.py and the related demo pipelines, could you take a look when convenient?

[xhd0728]

Thanks for the fix! The registry isolation works in our interleaved-request tests, but further testing found two issues:

1. The final answer is lost from the pipeline return value. Adding clear_citation_registry as the last step makes PipelineCall.final_result return None, even though the answer was generated successfully.
2. Cleanup is skipped on failure. If generation or retrieval raises before the last step, the registry remains in memory. Repeated failures in a reused server can accumulate these entries.

Could you move cleanup into a guaranteed lifecycle/finally path that preserves the generated result and handles failures and cancellation? Please also add regression tests for these cases before merging.

[MestreY0d4-Uninter]

Chiming in from #408 (the thread-local attempt this PR supersedes) — the two issues @xhd0728 found both trace to the same design choice: the registry state lives in a class-level global (CitationRegistry._instances) while only the small citation_registry_id handle flows through the pipeline context. That forces cleanup to be expressed as a pipeline step, which is exactly what breaks final_result (issue 1) and gets skipped on mid-pipeline failure (issue 2).

Two ways to remove the global-lifecycle problem entirely:

Option A — carry the state in the pipeline context (no global at all). The registry state is just {doc_hash: id, counter} per query index — JSON-serializable. assign_citation_ids_stateful could return the state as an output variable (citation_state) that the next call feeds back in via the YAML input mapping. Per-request isolation then comes for free from the pipeline's own per-run variable dict: nothing shared across requests, nothing to clean up on failure, and no trailing step so generation.generate stays last and final_result is preserved.

Option B — keep the registry-id design, replace the trailing step with allocation-time GC. Add a created_at timestamp per registry in create(), purge entries older than a TTL (and/or cap the number of live registries) at the top of create(), and drop custom.clear_citation_registry from the YAML. Cleanup then runs on a guaranteed path (every run allocates, so every run collects), memory stays bounded even when generation raises mid-pipeline, and the pipeline's return value is untouched.

Either way, the regression tests that would pin this down: (1) interleaved runs keep independent counters (already present), (2) a pipeline that raises after assign_citation_ids_stateful leaves no unbounded growth in a reused server, (3) final_result still carries the generation output on the happy path, (4) cancellation mid-run behaves like failure.

Happy to test either approach against the interleaved MCP-client scenario from the maintainer testing on this thread, or to send a patch if useful — we have the failing repro from #394 wired up locally.