gh-153713: Fix cross-interpreter memory leak in _xidata_release via atomic claim-and-free

[imabd645]

• Fixes Cross-interpreter memory leak in _xidata_release when originating interpreter is destroyed #153713

Description

This PR resolves the intentional memory leak in Python/crossinterp.c where xidata->obj is leaked if the originating interpreter is destroyed before the receiving interpreter calls _PyXIData_Release.

Instead of introducing a heavy interpreter-level registry with a blocking teardown that could cause deadlocks (especially in the PEP 703 free-threaded build), this PR implements a lock-free atomic claim-and-free protocol.

Implementation Details

1. Per-Item Atomic Status: Introduced an _Py_atomic_int status field to _PyXIData_t (ACTIVE, CLAIMED_BY_SENDER_TEARDOWN, RELEASED).
2. Intrusive Linked List: Added xid_next and xid_prev pointers to _PyXIData_t, allowing them to be linked directly to PyInterpreterState->xidata_list_head via an O(1) splice mutex.
3. Lock-Free CAS in _xidata_release: The receiver attempts a compare-and-swap from ACTIVE to RELEASED. If it wins, it proceeds exactly as before. If it loses to the sender's teardown, it skips the free and discards its reference, avoiding a double-free.
4. Safe Teardown: Py_EndInterpreter (via PyInterpreterState_Delete) detaches the list under the splice lock, then lock-freely iterates the orphaned data, running a CAS from ACTIVE to CLAIMED_BY_SENDER_TEARDOWN and safely deallocating them.

This guarantees exactly one side ever frees the data, and ensures no locks are held across arbitrary free() calls that might trigger re-entrant Python code.

Verification

• Added a NEWS.d blurb file.
• Verified that the by-value primitive transfers and existing interpreter teardown paths do not cause regressions.

• Issue: Cross-interpreter memory leak in _xidata_release when originating interpreter is destroyed #153713

[ZeroIntensity]

It seems you mixed two fixes in this PR. Please fix that.

[bedevere-app]

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

[imabd645]

I have addressed the CI failures and pushed the fixes for the macOS race conditions and linting issues. Ready for review!

[imabd645]

I have made the requested changes; please review again. (I have rebased the branch to drop the accidentally mixed commit, so this PR now exclusively contains the lock-free CAS fixes for gh-153713)

[bedevere-app]

Thanks for making the requested changes!

@ZeroIntensity: please review the changes made to this pull request.

[ZeroIntensity]

This isn't lock-free. There's a mutex around the linked list. I think we can switch to a compare-exchange loop there.

[sobolevn]

Have we even discussed this? There might be different solutions, jumping to AI-generated code is not the best idea.

[ZeroIntensity]

Yeah, I know. I talked with Eric a bit about this at EP this morning. We'll probably come up with a better solution at the sprints.