Test regressions with numpy 2.4.0 #522

Activity

1. jakelishman commented on Jan 7, 2026

   

   jakelishman

   on Jan 7, 2026

   Last edited by jakelishman

   Contributor

   More actions

   I believe this is specific to Python 3.14 with the introduction of the new PyUnstable_Object_IsUniquelyReferenced, which Numpy uses as of Numpy 2.4. The previous implementation of refcheck in NumPy allowed two references to exist, I believe assuming that one was held in Python space in the Python interface passing it down to the C. The test writes the check as

   let array = PyArray::<f64, _>::zeros(py, 3, false);
   assert_eq!(array.shape(), [3]);
   
   let mut array = array.readwrite();
   assert_eq!(array.as_slice_mut().unwrap(), &[0.0; 3]);
   
   let mut array = array.resize(5).unwrap();
   assert_eq!(array.as_slice_mut().unwrap(), &[0.0; 5]);

   If I insert a let array2 = array.clone(); after the first creation, I can reproduce the same failure on other versions of Python or previous versions of Numpy.
2. Icxolu commented on Jan 7, 2026

   

   Icxolu

   on Jan 7, 2026

   Member

   More actions

   I think you are right. This looks like the numpy change: numpy/numpy@0957720

   I believe this just shows that the test is slightly wrong in that we should drop the bound of the initial array after we created the readwrite guard.

   let array = PyArray::<f64, _>::zeros(py, 3, false);
   assert_eq!(array.shape(), [3]);
   
   let mut array = {
       let guard = array.readwrite();
       drop(array);
       guard
   };
   assert_eq!(array.as_slice_mut().unwrap(), &[0.0; 3]);
   
   let mut array = array.resize(5).unwrap();
   assert_eq!(array.as_slice_mut().unwrap(), &[0.0; 5]);

   cc @ngoldbaum Since you authored that change, maybe you can confirm that this is correct?
3. jakelishman commented on Jan 7, 2026

   

   jakelishman

   on Jan 7, 2026

   Contributor

   More actions

   I'd considered making a patch to that effect, but I worry that the Numpy change implies a big caveat in the use of PyReadwriteArrayMethods::resize that will have knock-on effects for downstream users.

   Perhaps we could ease the user pain by introducing a PyArray::into_readwrite that avoids the Bound::clone?
4. ngoldbaum commented on Jan 7, 2026

   

   ngoldbaum

   on Jan 7, 2026

   Contributor

   More actions

   I agree that the original test was buggy. If you created that reference in Python then it wouldn't have been possible. Because you created it in an extension, the refcheck passed spuriously.

   Now, NumPy is using a more stringent check based on new facilities in the CPython C API for this. It's correctly catching that the resized array is not uniquely referenced in the test.

   👍React with 👍1Reacted by Icxolu
5. jakelishman commented on Jan 7, 2026

   

   jakelishman

   on Jan 7, 2026

   Contributor

   More actions

   I've opened #524 as an exploratory version of the fix, where the introduction of the try_into_readwrite method makes it rather simpler to write the test, as a proxy for the user experience of using resize with these changes.
6. Icxolu commented on Jan 7, 2026

   

   Icxolu

   on Jan 7, 2026

   Member

   More actions

   Perhaps we could ease the user pain by introducing a PyArray::into_readwrite that avoids the Bound::clone?

   I was also wondering that. However resize is kind of unique in that it requires a unique reference and a unique borrow is not sufficient. into_readwrite makes writing the test much nicer, but in the general case does not guarantee success. The initial array could already be cloned earlier in either Rust or Python increasing its refcount above 1 and failing to resize. Maybe it still worth adding, because the naive/easy way is always wrong?
7. jakelishman commented on Jan 7, 2026

   

   jakelishman

   on Jan 7, 2026

   Contributor

   More actions

   There's a potential advantage to into_* anyway because of the lack of refcount update, which may make it worth it on its own?

   👍React with 👍1Reacted by Icxolu
8. Icxolu

   closed this as completedin #524on Jan 7, 2026