Improve Spark cursor session ownership and lifecycle handling #791

Use case

Track focused improvements to Spark session and calculation lifecycle behavior across SparkCursor, AsyncSparkCursor, and AioSparkCursor.
The current implementation has a confirmed readiness-state comparison defect and exception-path cleanup gaps, alongside ownership and connection-lifecycle decisions that need an agreed contract.

The findings were rechecked at a03bc2b4ea5dddcbf6230aafda0a359dce12ef0e.
Deterministic offline probes confirmed failure-state polling, termination of supplied sessions, missing initialization rollback, and skipped executor shutdown after termination errors.
No live AWS calls were made; cancellation behavior remains a separate investigation because AWS cancellation is best effort.

Proposed change

• Spark: stop readiness polling on failed, terminated, or degraded session states #792 — Spark: stop readiness polling on failed, terminated, or degraded session states
• Spark: define ownership when reusing externally supplied or shared sessions #793 — Spark: define ownership when reusing externally supplied or shared sessions
• Spark: clean up newly created sessions when cursor initialization fails #794 — Spark: clean up newly created sessions when cursor initialization fails
• AsyncSparkCursor: handle executor cleanup when session termination fails #795 — AsyncSparkCursor: handle executor cleanup when session termination fails
• Spark: define cleanup on Connection and AioConnection context exit #796 — Spark: define cleanup on Connection and AioConnection context exit
• Spark: verify best-effort calculation cancellation without mandatory session termination #797 — Spark: verify best-effort calculation cancellation without mandatory session termination
• Spark: an interrupt during StartCalculationExecution leaves the calculation running #841 — Spark: an interrupt during StartCalculationExecution leaves the calculation running (follow-up found while fixing Spark: verify best-effort calculation cancellation without mandatory session termination #797)

Fix the readiness comparison independently.
Agree on ownership before changes that affect borrowed/shared sessions or connection cleanup.
Implement and review each remaining concern in a focused PR, preserving the documented behavior or documenting the agreed compatibility change.
This tracking issue does not mandate a connection-wide reference-counting design or a new public close API.

Validation plan (if implementing)

Each sub-issue records its source evidence, reproduction or open question, scope, and acceptance criteria.
Use deterministic tests for state transitions and fault paths, and relevant real-AWS tests for session ownership, calculation cancellation, and cleanup.
Cover all three Spark cursor variants; a successful mocked test does not establish service behavior.
Serialize live validation with isolated resources and explicit final cleanup.
Close this parent when the sub-issues are resolved or explicitly dispositioned with the remaining contract documented.