[hobu]

Continuing #423, I've added index.Property.to_json and index.Property.from_json methods and beefed up the tests to account for those while dropping mention of pickle testing.

[adamjstewart]

Needs the size check to avoid OOM errors and DoS attacks.

[adamjstewart]

I think we actually want to keep these tests. I added these a while back to document that an Rtree can't be pickled, and therefore can't be used in a multiprocessing context on many platforms. This isn't related to pickling objects, it's pickling the entire index. This is still a desirable property to fix in the future.

[hobu]

I didn't understand why there's a test that's marked as an expected failure. I thought it was a test that had rotted or something.

Shouldn't we instead nuke the __getstate__ and __setstate__ machinery if we want to short circuit index pickling?

[adamjstewart]

For the full history, the following bug reports:

• Error in "IndexProperty_GetOverwrite": Property IndexIdentifier was empty #47
• Multithreading #53
• Index does not pickle fully #65
• Pickling and unpickling messes up Index #87
• Multithreading issue #105
• About RTree Data Structure Persistent Storage #284

document several complaints that pickling of an index (especially important in a multiprocessing context) is broken. I tried to hack getstate/setstate in #197 but you wanted a better long-term fix. So we merged only the xfail tests in #233 so it would be easier to test this in the future. #307 tries to document that this is broken.

I'm definitely happy with any real fix for pickling you're able to come up with.

[hobu]

I happily forgot all this stuff!

I think the challenge for pickling the index itself is in the context of multi-threaded operations, there's no mechanism in libspatialindex to serialize access to the thing.