Modernize + verify 5 tutorial notebooks (Longformer/IMDB, node2vec, bikeshare, jigsaw)

.gitignore

@@ -0,0 +1,20 @@
+.venv/
+.env
+__pycache__/
+*.pyc
+.ipynb_checkpoints/
+wandb/
+*.pt
+*.bin
+.DS_Store
+data/.hf_cache/
+data/capital_bikes/
+data/jigsaw/
+results/_nbruns/
+results/checkpoint-*/
+results/runs/
+results/logs/
+results/longformer_imdb/
+results/roberta_base_multilabel_jigsaw/
+results/longformer_base_multilabel_jigsaw/
+

.python-version

@@ -0,0 +1 @@
+3.11

README.md

@@ -1,3 +1,127 @@
-# Tutorials from my personal site [jesusleal.io](jesusleal.io)
+# Tutorials from my personal site [jesusleal.io](https://jesusleal.io)
 
-This repository contains the full versions of the tutorials published on [My personal website](https://jlealtru.github.io/).  I am interested in Deep Learning, particularly its applications in NLP and Graph Learning.
+This repository contains the full versions of the tutorials published on [my personal website](https://jlealtru.github.io/). Topics: Deep Learning, NLP, and Graph Learning.
+
+The notebooks were originally authored ~2020 against CUDA-only PyTorch. They have since been modernized: every library updated to a current pinned version, code made device-agnostic (CUDA → MPS → CPU), and the environment is now managed by [`uv`](https://docs.astral.sh/uv/).
+
+## Setup
+
+Prereqs: macOS, Linux, or Windows; [uv](https://docs.astral.sh/uv/getting-started/installation/) installed.
+
+```bash
+# 1. Create the venv and install pinned, hash-verified deps
+uv sync --frozen
+
+# 2. (ETM notebooks only) install the pinned spaCy models — en_core_web_lg + md
+uv sync --frozen --group etm
+
+# 3. Launch JupyterLab
+uv run jupyter lab
+```
+
+`uv sync --frozen` refuses any dependency whose hash doesn't match `uv.lock`, so installs are reproducible across machines.
+
+The hardware choice happens at runtime via `notebooks/_utils.py::pick_device()`:
+
+| Hardware | Result |
+|---|---|
+| NVIDIA GPU | `cuda` |
+| Apple Silicon (M1 / M2 / …) | `mps` |
+| Anything else | `cpu` |
+
+On Apple Silicon, `PYTORCH_ENABLE_MPS_FALLBACK=1` is set automatically so any op that lacks an MPS kernel falls back to CPU for that op only — the rest of training stays on the GPU.
+
+## Where caches and outputs live
+
+All artifacts produced or cached by the notebooks stay inside the repository — no surprise gigabytes under your home directory.
+
+| What | Where |
+|---|---|
+| HuggingFace models, datasets, hub cache | `data/.hf_cache/` (set via `HF_HOME` in `notebooks/_utils.py`) |
+| Trainer checkpoints, training logs | `results/` (TrainingArguments `output_dir='../results'`) |
+| nbconvert re-executed copies | `results/_nbruns/` |
+| wandb run dirs | `wandb/` (only if you opt in; default `report_to='none'`) |
+
+All of those paths are gitignored. To purge everything: `rm -rf data/.hf_cache results/_nbruns results/checkpoint-* results/runs results/logs wandb`.
+
+## Continuing a long-running training in a new session
+
+Long fine-tunes (Longformer, BigBird, multi-label) take hours on M1. With `save_strategy='epoch'` (set in all transformer notebooks), each completed epoch writes `results/checkpoint-<step>/`. If the training is interrupted:
+
+```bash
+# Quick status check
+find results -name "checkpoint-*" -type d        # what's been saved
+ps -A | grep ipykernel | grep -v grep            # is a kernel still alive?
+
+# Resume RoBERTa+IMDB from the latest checkpoint
+uv run python scripts/resume_roberta_imdb.py
+```
+
+`scripts/resume_roberta_imdb.py` auto-finds the highest-numbered `checkpoint-N/` under `results/` and continues from there with the same M1-tuned `TrainingArguments` used in the notebook. Equivalent resume scripts can be added for the other transformer notebooks the same way (1 file each, ~120 lines).
+
+## Data prerequisites
+
+Some notebooks rely on datasets that are **not bundled with this repository** (size, licensing, etc.). Helper scripts fetch them into `data/<name>/`:
+
+| Notebook(s) | Dataset | How to get it |
+|---|---|---|
+| `processing_capital_bikeshare_data.ipynb`<br>`node2vec with capitol bikeshare data.ipynb` | Capital Bikeshare trips 2019 + 2020 (24 monthly zips) | `bash scripts/fetch_bikeshare.sh` — public S3 bucket, no auth. Downloads to `data/capital_bikes/` (~140 MB zipped). |
+| `Multi_label_classification_longformer_tutorial.ipynb`<br>`Multi_label_classification_roberta.ipynb` | Jigsaw Toxic Comment Classification | `bash scripts/fetch_jigsaw.sh` — **needs** `~/.kaggle/kaggle.json` and acceptance of the [competition rules](https://www.kaggle.com/c/jigsaw-toxic-comment-classification-challenge/rules). Downloads to `data/jigsaw/`. |
+| `etm_preprocessed_data.ipynb`<br>`etm_spacy_pipeline.ipynb` | Pitchfork album reviews (`pitchfork.csv` + `stop.txt`) | `uv run python scripts/fetch_pitchfork.py` — pulls `reviews.csv` from the [HF `mattismegevand/pitchfork`](https://huggingface.co/datasets/mattismegevand/pitchfork) dataset, remaps it to the notebooks' schema, and generates the stopword file into `data/pitchfork/`. Also needs the spaCy models: `uv sync --frozen --group etm`. |
+
+Run the bikeshare processing notebook **before** the node2vec notebook — the latter consumes `data/capital_bikes/graph_data_full.csv` and `bike_locations.csv` produced by the former. (Station locations are pulled live from the Capital Bikeshare open-data layer, whose schema now exposes `NAME`/`LATITUDE`/`LONGITUDE`.)
+
+The IMDB-based notebooks (`RoBERTA with IMDB.ipynb`, `Longformer with IMDB.ipynb`, `BigBird text classification.ipynb`) auto-download IMDB through HuggingFace `datasets` — no manual setup needed.
+
+## Fast smoke test of the training notebooks
+
+A full run of the training notebooks takes hours-to-days on Apple Silicon (dense attention on MPS runs ~10× slower than the RTX 3090 these were built for; the ETM notebooks also tokenize ~26k reviews with spaCy). To verify that a notebook still **executes end-to-end** without paying for a full run, the training notebooks honour a `SMOKE_TEST` environment variable:
+
+```bash
+cd notebooks
+SMOKE_TEST=1 uv run jupyter nbconvert --to notebook --execute \
+  --ExecutePreprocessor.kernel_name=python3 \
+  --output-dir ../results/_nbruns --output <name>_smoke \
+  "Longformer with IMDB.ipynb"
+```
+
+With `SMOKE_TEST=1` the notebook sub-samples the data and cuts to a short single-pass run — a few minutes total. Unset (the default), every notebook runs at its original full-scale configuration. Notebooks with the toggle: `RoBERTA with IMDB`, `Longformer with IMDB`, `BigBird text classification`, `Multi_label_classification_roberta`, `Multi_label_classification_longformer_tutorial`, and the two ETM notebooks `etm_preprocessed_data` / `etm_spacy_pipeline` (there the toggle subsamples documents, relaxes the vocabulary pruning, and shortens training instead of `max_length`). The ETM notebooks also need `WANDB_MODE=disabled` in the environment unless you have run `wandb login` (it is set inside the notebook by default).
+
+## Streamlit app (`app.py`)
+
+```bash
+uv run streamlit run app.py
+```
+
+Requires a local [Ollama](https://ollama.com/) daemon with a Gemma-3 vision model pulled (out of scope for this repo).
+
+## Layout
+
+```
+.
+├── pyproject.toml          # exact-pinned deps
+├── uv.lock                 # hash-verified resolved graph
+├── .python-version         # 3.11
+├── app.py                  # Streamlit + Ollama OCR demo
+├── data/                   # external datasets land here (gitignored)
+├── results/                # training outputs / nbconvert reruns (gitignored)
+├── notebooks/
+│   ├── _utils.py           # pick_device(), set_seed()
+│   └── *.ipynb             # the tutorials
+└── scripts/                # one-off modernization patchers (run once)
+```
+
+## What changed during modernization
+
+- **Packaging**: introduced `pyproject.toml` + `uv.lock`; Python pinned to 3.11.
+- **Device**: every notebook routes through `pick_device()` — CUDA → MPS → CPU.
+- **Mixed precision**: `fp16=True` → `bf16=True` for MPS compatibility (CPU/CUDA ignore bf16 gracefully).
+- **gensim 4.x**: `Word2Vec(size=...)` → `vector_size=...`; `wv.vocab` → `wv.key_to_index`; `wv.index2word` → `wv.index_to_key`.
+- **pandas**: `display.max_colwidth=-1` → `None`.
+- **HuggingFace Trainer**: `evaluation_strategy=` → `eval_strategy=`; `gradient_checkpointing=False` removed from `from_pretrained()` (use `model.gradient_checkpointing_disable()` instead); `cache_dir='/media/...'` Linux paths removed; `report_to='none'` added (wandb opt-in).
+- **node2vec**: replaced unmaintained `stellargraph` with `pecanpy`, which has macOS arm64 wheels and a 1:1 mapping of biased-random-walk parameters. gensim `Word2Vec(iter=...)` → `epochs=...` (the 4.x rename).
+- **spaCy**: `spacy.prefer_gpu()` wrapped in try/except so it no-ops on hardware without CUDA.
+- **Kernelspec**: every notebook's dead `conda-env-torch-py` kernel replaced with the portable `python3` kernel so `jupyter`/`nbconvert` run against the `uv` venv.
+- **Capital Bikeshare**: added `scripts/fetch_bikeshare.sh` (public S3); handled the mid-2020 trip-CSV schema change and the station-locations layer's new `NAME`/`LATITUDE`/`LONGITUDE` columns (was `ADDRESS`/`ID`).
+- **Jigsaw notebooks**: fixed pre-existing `SyntaxError`s in the `from_pretrained(...)` calls (missing commas), repointed a dead `/media/...` checkpoint to `roberta-base`, and set DataLoader `num_workers=0` (macOS `spawn` can't pickle notebook-defined `Dataset` classes). Added `scripts/fetch_jigsaw.sh`.
+- **Smoke toggle**: `SMOKE_TEST` env var on the fine-tuning notebooks (see above).

notebooks/ processing_capital_bikeshare_data.ipynb

@@ -29,7 +29,7 @@
    ],
    "source": [
     "# modify display options to make sure we can see full texts fields\n",
-    "pd.set_option('display.max_colwidth', -1)\n",
+    "pd.set_option('display.max_colwidth', None)\n",
     "pd.set_option('display.max_columns', None)"
    ]
   },
@@ -1287,9 +1287,13 @@
     }
    ],
    "source": [
-    "# now we to also process a mapping file of the location of the files to get the latitute and longitude of the \n",
-    "# stations. That information is available here: \n",
+    "# now we also process a mapping file of the location of the stations to get their\n",
+    "# latitude and longitude. The Capital Bikeshare open-data \"station locations\" layer now\n",
+    "# uses NAME/LATITUDE/LONGITUDE (it formerly exposed ADDRESS/ID), so rename NAME -> ADDRESS\n",
+    "# to keep the station name as the join key used downstream (and in the node2vec notebook).\n",
+    "# read_csv follows the ArcGIS 301 redirect automatically.\n",
     "bike_locations = pd.read_csv('https://opendata.arcgis.com/datasets/a1f7acf65795451d89f0a38565a975b3_5.csv')\n",
+    "bike_locations = bike_locations.rename(columns={'NAME': 'ADDRESS'})\n",
     "print(len(bike_locations))\n",
     "bike_locations.head()"
    ]
@@ -1311,12 +1315,12 @@
     }
    ],
    "source": [
-    "# the data process reveals that there have been a few stations that have been removed or relocated. Out of the 667\n",
-    "# bikes in the system from 2010 to 2019 we have information of location for one hundred stations\n",
+    "# How many of the historical (2019) stations have no current location match?\n",
+    "# (the open-data layer reflects today's active stations, so relocated/retired ones won't match)\n",
     "list_stations_dataframe = pd.DataFrame(list_stations, columns = ['ADDRESS'])\n",
     "list_stations_dataframe['dummy']= 1\n",
     "list_stations_dataframe_2019 = list_stations_dataframe.merge(bike_locations, how='left', on='ADDRESS')\n",
-    "len(list_stations_dataframe_2019[list_stations_dataframe_2019['ID'].isnull()])"
+    "len(list_stations_dataframe_2019[list_stations_dataframe_2019['LATITUDE'].isnull()])"
    ]
   },
   {
@@ -1431,12 +1435,12 @@
     }
    ],
    "source": [
-    "bike_trips_data_graph_2019 = bike_trips_data_graph_2019.merge(bike_locations[['ADDRESS','ID']], \n",
+    "bike_trips_data_graph_2019 = bike_trips_data_graph_2019.merge(bike_locations[['ADDRESS','LATITUDE','LONGITUDE']],\n",
     "                                                              how='left',\n",
     "                                                              left_on='Start station',\n",
     "                                                             right_on='ADDRESS')\n",
     "\n",
-    "bike_trips_data_graph_2019 = bike_trips_data_graph_2019.merge(bike_locations[['ADDRESS','ID']], \n",
+    "bike_trips_data_graph_2019 = bike_trips_data_graph_2019.merge(bike_locations[['ADDRESS','LATITUDE','LONGITUDE']],\n",
     "                                                              how='left',\n",
     "                                                              left_on='End station',\n",
     "                                                             right_on='ADDRESS')\n",
@@ -1587,7 +1591,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3",
+   "display_name": "Python 3 (website_tutorials)",
    "language": "python",
    "name": "python3"
   },
@@ -1606,4 +1610,4 @@
  },
  "nbformat": 4,
  "nbformat_minor": 4
-}
+}