Repository navigation
Make SCIP result deterministic - #5665
yangdanny97 wants to merge 4 commits into
Conversation
stroxler
left a comment
There was a problem hiding this comment.
LGTM, thanks!
Could you rebase? We had an outage in diff import handling, but that should be fixed now
Optional recommendations:
Note: this comment updates automatically after changes to the PR. Updates may take up to half an hour. |
|
This pull request has been imported. If you are a Meta employee, you can view this in D132773044. (Because this pull request was imported automatically, there will not be any future comments.) |
This comment has been minimized.
This comment has been minimized.
|
I think the test failures were an issue on main that were fixed in 6df5707, so another rebase should get everything green. |
|
rebased again |
|
According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅ |
I realized the ordering of the SCIP documents is not deterministic when we dump the data. This PR adds a sorting pass at the end and also slightly tweaks the docs.
Storing these docs in memory increases peak memory by 150-200mb on pytorch
Perf vs scip-python on my macbook air
Pre-commit: 0.5 s and 157 MB versus 8.3 s and 457 MB.
Rich: 0.8 s and 190 MB versus about 29 s and 700 MB.
Django: 6 s and 748 MB versus 140 s and 4.4 GB. scip-python crashes on Django unless you raise Node’s memory limit.