[Bugfix][EPD] Fix Mooncake transfer lifecycle cleanup

[jiaran-king]

Summary

• wait for every consumer-shard reservation future before rolling back every shard by transfer ID
• wait for every started shard write before releasing producer staging memory or abandoning consumer allocations
• attempt cancellation on every shard and keep cancelled-transfer tombstones when an RPC fails, so healthy shards are released and late ready events cannot recreate a cancelled transfer
• explicitly unregister the producer staging pool during connector shutdown

Why

The Mooncake connector fans reservation and write work out across consumer TP shards. Previously, the result loop stopped at the first exception. Other shard operations could still be running while source staging memory or destination allocations were released, and successful reservations from the same fan-out could remain allocated until lease expiry.

A cancel RPC can also fail after some remote shards have already processed the cancellation. Removing the scheduler tombstone in that case lets a queued late ready event recreate a transfer whose remote state is already partially released.

Tests

• .venv/bin/python -m pytest -q tests/v1/ec_connector/unit/test_ec_mooncake_connector.py (49 passed)
• .venv/bin/python -m pytest -q tests/v1/ec_connector/unit/test_worker_ec_connector.py (6 passed)
• pre-commit run ruff-check --files vllm/distributed/ec_transfer/ec_connector/mooncake_ec_connector.py tests/v1/ec_connector/unit/test_ec_mooncake_connector.py
• pre-commit run ruff-format --files vllm/distributed/ec_transfer/ec_connector/mooncake_ec_connector.py tests/v1/ec_connector/unit/test_ec_mooncake_connector.py

The added fault-injection coverage verifies partial TP-shard reservation failure, partial TP-shard write failure, continued cancellation after a shard RPC failure, cancel-RPC failure with a late ready event, and producer-pool unregistration at shutdown.

Duplicate-work check

I searched the open PRs in vllm-project/vllm and kvcache-ai/vllm for the same Mooncake shard-failure and reservation-failure fixes and found no existing implementation.

AI assistance

AI assistance was used to trace the failure paths, prepare the implementation and tests, and run the checks above. The human submitter reviewed the change and its test evidence before submission.