Silence ~50 spurious test warnings that were being blamed on unrelated tests

[nilsmechtel]

Running the test suite printed about fifty "coroutine was never awaited" warnings, and they were blamed on tests that had nothing to do with the cause — a CLI test, a mock-library internal, an asyncio scheduling callback. One of them landed inside a command-line test's captured output and broke its assertion, which is how a flaky failure with no apparent connection to proxy deployments turned out to be a proxy-deployment cleanup problem. This makes the noise go away, and adds a check that fails the suite if it ever comes back, without changing any shipped behaviour.

Problem

ProxyDeployment.__del__ is async def, and that is not an accident. A Ray Serve deployment class has exactly one per-replica shutdown hook — the destructor — and Serve awaits it. From ray/serve/_private/replica.py (Ray 2.55.1):

if hasattr(self._callable, "__del__"):
    # Make sure to accept `async def __del__(self)` as well.
    await self._call_func_or_gen(
        self._callable.__del__,
        run_sync_methods_in_threadpool_override=False,
    )

That is the path every real replica teardown takes, and it is where the Hypha deregistration, the False registration report and the server.disconnect() that frees the client_id actually run.

CPython's collector does not await destructors. It calls __del__, gets a coroutine object back and discards it, which raises RuntimeWarning: coroutine 'ProxyDeployment.__del__' was never awaited — attributed to wherever the collector happened to trip, not to the code that built the object.

Five test modules build ProxyDeployment stand-ins with object.__new__ (or the real constructor) and drop them: test_proxy_entry_saturation_tolerance, test_proxy_hypha_decoupled_health, test_service_id_registration_gate, test_usage_ledger, test_peer_connection_sweep. Together they accounted for 50–52 of the suite’s 66–68 warnings — the range is real, not measurement slop, because how many objects the collector finalises inside the session depends on where it trips.

Note that explicitly awaiting the teardown in a test does not help. Calling obj.__del__() by hand is an ordinary method call; it does not mark the object finalised, so the collector still calls the destructor afterwards and still discards the coroutine. Measured:

A: plain, dropped                          -> 1 never-awaited warning
B: plain, awaited __del__ then dropped     -> 1 never-awaited warning
C: subclass with sync no-op __del__        -> 0

Solution

A shared tests/apps/_proxy_double.py exporting PROXY_CLS (the production class, for source assertions and behaviour tests) and ProxyDouble — that same class minus the Serve-only destructor. The five modules construct their stand-ins from ProxyDouble. Nothing else about the class changes, so the registration-gate tests still exercise the real __init__, the real claim path and the real _deregister_services. One cosmetic consequence: proxy_deployment.py logs self.__class__.__name__ during construction, so those tests now log "ProxyDouble". Nothing asserts on it.

The stand-ins hold no resource the destructor would have released: BIOENGINE_APP_DIR is unset under test so _usage_ledger is None, server is None, and no maintenance task is ever started — the suite reports no lingering pending asyncio tasks before or after. The un-awaited coroutine was the entire leak.

A session-wide hook in tests/conftest.py keeps it that way. pytest_warning_recorded collects any un-awaited-destructor record, and pytest_sessionfinish fails the run, naming ProxyDouble and the nodeids the records were charged to. It has to be session-wide precisely because the warning does not land on the test that caused it.

-W error::pytest.PytestUnraisableExceptionWarning would also turn these into failures — fourteen of them. It is the wrong tool rather than an ineffective one: the failures land on whichever unrelated test the collector tripped in (test_proxy_counts_a_raising_call_as_failed took the blame in one run), which reintroduces exactly the misattribution this PR removes. A plain -W error::RuntimeWarning on its own fails nothing, because the warning is raised inside a coroutine's finaliser.

Production code is deliberately unchanged

Moving the teardown out of the destructor into an explicit async shutdown hook is the textbook fix, and Ray Serve does not offer one. call_destructor reaches user code in exactly two places: __del__, and __serve_multiplex_wrapper.shutdown(), which is Serve's own multiplexing internals rather than a user hook. Renaming the body to _shutdown() and delegating from __del__ would change nothing observable and would not remove the destructor.

The two alternatives both regress behaviour the service-registration record depends on:

• A synchronous destructor doing the work is impossible — the body awaits throughout (_flush_usage, _deregister_services, _reset_server_connection all perform I/O with timeouts).
• A synchronous destructor scheduling the async work on the running loop compiles, but Serve awaits call_destructor and then tears the replica down. A scheduled task has nobody waiting on it, so the deregistration and the disconnect would race replica death — precisely the client_id release that avoids "Client already exists and is active" on the successor, and the tagged False report the replica-tag guard arbitrates.

So the async destructor stays. What changes is that the constraint is now enforced rather than implied: test_the_production_destructor_stays_awaitable fails if anyone makes it synchronous, and the module docstring records why the shape is what it is.

Test plan

Full suite in the worker image: 525 passed, 25 skipped, 67–68 warnings before, 528 passed, 25 skipped, 16 warnings after. Un-awaited ProxyDeployment.__del__ records: 50–52 before (52, 52, 51, 52, 52 across five runs here; an independent check saw 51, 52, 50 across three, so 50 is the observed floor), 0 after. Those are exact per-record tallies from the pytest_warning_recorded hook, not counts read off pytest's warnings summary — the summary compresses records into location-grouped blocks and cannot be totalled reliably. The 16 warnings that remain are pydantic and Ray deprecation notices from imports.

All 27 tests in test_service_id_registration_gate.py pass, including the four replica-tag invariants: a live replica deregistering itself, a departed replica's late deregistration being dropped, an untagged successor claiming from a tagged predecessor, and the crash-restart case.

Mutation-proven:

Mutation	Result
ProxyDouble no longer overrides the destructor	test_dropping_the_test_double_is_silent fails
Production destructor made synchronous (scheduling teardown on the loop)	test_the_production_destructor_stays_awaitable and test_dropping_the_production_class_warns fail
Probe keeps the instance alive instead of dropping it	test_dropping_the_production_class_warns fails
One module reverted to the raw class	session hook exits 1: "14 un-awaited ProxyDeployment destructor(s). Build test proxies from tests.apps._proxy_double.ProxyDouble."

The hook is silent and the suite exits 0 on the unmutated tree.

Files

File	Change
tests/apps/_proxy_double.py	New. PROXY_CLS and ProxyDouble.
tests/apps/test_proxy_teardown_hook.py	New. Three tests: the destructor stays awaitable, the raw class warns (positive control), the double does not.
tests/conftest.py	Session hook that fails the run if any proxy is left for the collector.
tests/apps/test_proxy_entry_saturation_tolerance.py	_bare_proxy builds a ProxyDouble.
tests/apps/test_proxy_hypha_decoupled_health.py	_bare_proxy builds a ProxyDouble.
tests/apps/test_service_id_registration_gate.py	_bare_proxy and _construct_proxy build a ProxyDouble.
tests/apps/test_usage_ledger.py	Both construction sites build a ProxyDouble.
tests/apps/test_peer_connection_sweep.py	_make_instance builds a ProxyDouble.

🤖 Generated with Claude Code