[Fix] Pin ipython<9.16 to unbreak the docs build

[adityasingh2400]

The docs build currently fails on every branch, including master.

What breaks

make html dies while nbsphinx executes doc/tutorials/spade.ipynb, on the first cell at import viziphant:

ImportError: cannot import name 'backend2gui' from 'IPython.core.pylabtools'

Why

requirements-docs.txt asks for jupyter>=1.0.0, which pulls IPython unpinned. Installing the current docs requirements on Python 3.12 resolves to ipython 9.16.1 and matplotlib 3.8.4.

IPython 9.16 removed the public backend2gui alias from IPython.core.pylabtools. matplotlib below 3.9 imports that alias in pyplot.install_repl_displayhook(), which runs whenever pyplot is used inside a live IPython shell. import viziphant reaches it via holoviews calling plt.switch_backend("agg"). The resolved environment therefore cannot work.

Why pin IPython rather than raise matplotlib

Raising matplotlib does not fix it. viziphant 0.4.0, the latest release from November 2023 and a requirements-tutorials.txt dependency, calls matplotlib.cm.get_cmap, which matplotlib removed in 3.9. With matplotlib 3.11.1 the same notebook gets past the import and then fails at viziphant.patterns.plot_patterns with AttributeError: module 'matplotlib.cm' has no attribute 'get_cmap'.

So the matplotlib<3.9 cap is load bearing, and ipython<9.16 restores the last working combination. The pin goes in requirements-docs.txt because that is the only requirements file that brings in a Jupyter or IPython runtime. I confirmed empirically that a tutorials only install pulls neither.

This also records why the matplotlib cap exists. It was added in #634 without a note, which makes it easy to remove by mistake. The commit is 6ce65584, titled "[Fix] Tests for Neo 0.13.1, add same object to list", so the requirements change was incidental to that PR's stated purpose.

The durable fix is a viziphant release that stops using cm.get_cmap. Both pins can come off then, and environment-docs.yml could drop its python=3.12, whose stated reason is this same cap.

Verified on Python 3.12, matching the docs CI job:

• Reproduced. The unmodified requirements resolve to ipython 9.16.1 and matplotlib 3.8.4, and spade.ipynb fails with the error above on its first cell.
• With this change the same install resolves to ipython 9.15.0 and matplotlib 3.8.4, and spade.ipynb executes cleanly at 6 of 6 cells with no error outputs. statistics.ipynb, unitary_event_analysis.ipynb and granger_causality.ipynb also pass.
• Confirmed cm.get_cmap present in matplotlib 3.8.4 and absent in 3.9.0, and backend2gui present in IPython 9.15.0 and absent in 9.16.1.
• setuptools still parses the files, and ipython<9.16 appears in the built wheel's docs extra, so CI's pip install -e .[extras,tutorials,docs] picks it up.

Not verified

• I did not run a full make html, only notebook execution.
• gpfa.ipynb and asset.ipynb did not finish in my local environment and parallel.ipynb needs mpi4py, so I make no claim about those three.
• Tested on macOS arm64, not ubuntu-latest.

Disclosure: this change was prepared with AI assistance. I have reviewed and tested it.

[coveralls]

coverage: 45.709% (-42.9%) from 88.605% — adityasingh2400:fix-docs-matplotlib-ipython-pin into NeuralEnsemble:master

[adityasingh2400]

CI has now closed the one gap I flagged as not verified in the description.

I said there that I had only run notebook execution locally, not a full make html. The docs (3.12, ubuntu-latest) job on this branch did run the full build and it succeeded, with 19 warnings and no errors, and docs/readthedocs.org:elephant is green too. The install resolved to ipython 9.15.0, so the pin is being picked up the way the description predicted.

The remaining red on this PR is unrelated to the change. The pip, conda, and mpi jobs each end at 3 failed and 770 passed, and all three failures are elephant/test/test_datasets.py::TestDownloadDatasets hitting URLError: [SSL: CERTIFICATE_VERIFY_FAILED] while downloading test data. That matches what @CozySocksAlways described as runner network issues on my other PRs.

I want to be careful about one thing I could easily have claimed and cannot. My other open PRs do show docs failing while this one passes, but I checked their logs and their docs failure is the same SSL certificate error inside a notebook download rather than the IPython import error. So that contrast is not evidence for the pin. The evidence for the pin is this branch's own build succeeding on ipython 9.15.0, plus the reproduction in the description.

[adityasingh2400]

Worth flagging that this PR is what unblocks the other four open ones.

docs (3.12, ubuntu-latest) across all five right now:

#706  FAILURE
#707  FAILURE
#708  FAILURE
#709  SUCCESS   <- this PR
#710  FAILURE

All four failures are the same ImportError: cannot import name 'backend2gui' from 'IPython.core.pylabtools', which is exactly what the ipython<9.16 pin here addresses. This is the only branch carrying that pin and the only one whose docs job is green, and its docs/readthedocs.org check is green as well.

Separately, a re-run on #708 today cleared its pip and mpi jobs, which confirms those were the transient certificate failures from 05 August rather than anything in the branches.

[adityasingh2400]

Update that should make these cheap to clear. The data host looks recovered, so a re-run is likely all these five need now.

The red jobs on this PR are from 05 August and every failure in them is the corpus fetch, not the branch:

3 failed, 770 passed, 5 skipped
FAILED elephant/test/test_datasets.py::TestDownloadDatasets::test_valid_data
  URLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed

Since then the same fetch has started working. Your own Create caches for elephant_data job ran green on master at 08 August 23:41, and the endpoints answer now:

gin.g-node.org/NeuralEnsemble/elephant-data      HTTP 200
datasets.python-elephant.org                     HTTP 200

I cannot trigger a re-run myself. If you re-run the failed jobs on this one and on #706, #707, #708 and #710, I would expect all five to go green without any push from me.

The docs picture is unchanged from my earlier note, this branch is still the only one carrying the ipython<9.16 pin and the only one whose docs job passes.

[adityasingh2400]

Correcting my note above. A re-run happened on #706 this morning and it did not clear everything, so "a re-run is likely all these five need" was wrong.

Attempt 2 on 8075b6a:

conda   success   <- was failing
pip     success   <- was failing
mpi     failure
docs    failure

Two separate things are left, and neither is the branch.

mpi still dies on the corpus fetch, but the symptom changed from a connection reset to a refusal:

fatal: unable to access 'https://gin.g-node.org/NeuralEnsemble/elephant-data.git/': The requested URL returned error: 403
urllib.error.HTTPError: HTTP Error 403: Forbidden

I ran git ls-remote against that same URL from my own machine a few minutes ago and it succeeds, returning HEAD 666b920d. So the host is up and is refusing the runner specifically. My earlier reachability check was misleading because I tested from the wrong place.

docs has nothing to do with the data host:

ImportError: cannot import name 'backend2gui' from 'IPython.core.pylabtools'

That is notebook execution meeting a newer IPython that dropped backend2gui, the same one I flagged on #708. No number of re-runs clears it, it needs an IPython pin or nbsphinx_allow_errors.

[adityasingh2400]

Update on the data host, since I said earlier that a rerun would not help and that is no longer true.

All three red jobs here are TestDownloadDatasets reaching gin. conda gets HTTP Error 403, while mpi and pip get a socket level TimeoutError: [Errno 110] Connection timed out fetching dataset-1/dataset-1.h5. Nothing in the branch is involved.

This morning the host served GitHub runners again. NeuralEnsemble/python-neo#1894 ran at 08:39 UTC and pulled ephy_testing_data, a sibling dataset in the same gin org, with zero 403s and both jobs green. That is runner side evidence rather than me curling from my laptop, which is what made my earlier reachability check misleading.

So a rerun here is now worth a click. I cannot trigger one myself, rerun-failed-jobs needs admin and I only have read.

One thing worth not chasing. The warning in these logs

No corresponding version of 'elephant-data' found.
Elephant version: 1.2.2b1. Data URL:.../raw/v1.2.2b1/README.md, error: HTTP Error 404

is expected and harmless. elephant-data tags stop at v1.1.1, and download_datasets catches that HTTPError and falls back to raw/master by design, with a comment saying it is normal for development versions. I checked it because it looked like a deterministic failure, and it is not one.