CS771 Mini-Project 2: Prototype-Based Continual Learning on CIFAR-10

Group 90

Full write-up: report.pdf.

---

Problem

We are given 20 CIFAR-10-style datasets D1 … D20 stored as .pth files. Only D1 is labelled; D2 … D20 are unlabelled. The datasets arrive as a stream and must be learned incrementally: a model trained on D_k may not revisit the raw data of D_1 … D_{k-1}.

• D1 … D10 share the same input distribution (Task 1).
• D11 … D20 come from a shifted distribution (Task 2).

For each step k we produce a model f_k and report its accuracy on the held-out eval splits D_1 … D_k, i.e. a lower-triangular accuracy matrix.

Approach

1. Feature extraction. A pre-trained ResNet-50 (ImageNet, classification head removed) maps every 32×32 image to a 2048-dim feature vector. Preprocessing: resize to 224×224 → tensor → ImageNet normalisation. Features for all 20 train + 20 eval splits are extracted once and cached to disk; every experiment then runs on the cached features.

2. Classifier: Learning with Prototypes (LwP). Each class k is represented by a single prototype p_k = mean of its feature vectors. Prediction is nearest prototype under Euclidean distance:

   y_hat = argmin_k || x - p_k ||
   

3. Incremental updates.

   • f1 is fitted directly on the labelled D1.
   • f_k (k ≥ 2) pseudo-labels D_k with the current prototypes, then updates them.

   Update strategies compared:

   #	Strategy	Rule	Best setting
   1	Equal-weight averaging	p_new = ½(p_old + mean(D_k))	baseline
   2	Confidence-threshold filtering	update only on samples whose normalised nearest-prototype distance ≤ threshold; threshold swept 0.1 → 1.0	threshold 0.7
   3	Weighted averaging	p_new = α·p_old + (1−α)·mean(D_k); α swept 0.2 → 0.8	α = 0.8

   Weighted averaging with α = 0.8 was the most reliable across both tasks. Confidence filtering reduced label noise but showed more variance across datasets and did not improve generalisation under the Task 2 distribution shift.

Repository layout

.
├── report.pdf                                  # project report
├── README.md
├── .gitignore
├── task_1_lwp_incremental_prototypes.ipynb     # Task 1: models f1 … f10 on D1 … D10
└── task_2_lwp_domain_shift_prototypes.ipynb    # Task 2: models f11 … f20 on D11 … D20

Directories created at runtime (git-ignored):

part_one_dataset/   part_two_dataset/     # raw .pth datasets (download separately)
features/part_one/  features/part_two/    # cached ResNet-50 feature .npz files
models/                                   # saved prototype .npz files (task1 / task2)

Expected dataset paths (as referenced by the notebooks):

part_one_dataset/train_data/{1..10}_train_data.tar.pth
part_one_dataset/eval_data/{1..10}_eval_data.tar.pth
part_two_dataset/train_data/{1..10}_train_data.tar.pth
part_two_dataset/eval_data/{1..10}_eval_data.tar.pth

How to run

1. Install dependencies

   pip install torch torchvision numpy pillow

2. Place the datasets under part_one_dataset/ and part_two_dataset/ as shown above.

3. Task 1. Open task_1_lwp_incremental_prototypes.ipynb, uncomment the build_feature_cache() call and run it once to populate features/part_one/, then run the remaining cells. This writes models/task1_prototypes.npz.

4. Task 2. Open task_2_lwp_domain_shift_prototypes.ipynb, uncomment build_feature_cache() for features/part_two/, then run the rest. It loads the f10 prototypes from models/task1_prototypes.npz and writes models/task2_prototypes.npz.

Feature extraction benefits from a GPU; everything else is CPU-only NumPy and runs in seconds because it operates on the cached features.

Results (from the report, weighted averaging, α = 0.8)

Task 1: accuracy (%) of f1 … f10 on D1 … D10

	D1	D2	D3	D4	D5	D6	D7	D8	D9	D10
f1	79.64
f2	79.24	80.24
f5	77.64	79.12	78.40	79.20	78.80
f10	75.72	77.60	77.12	77.36	77.04	76.36	76.20	76.44	74.32	78.00

Task 2: accuracy (%) of f11 … f20 on D11 … D20

	D11	D12	D13	D14	D15	D16	D17	D18	D19	D20
f11	60.36
f15	58.84	48.76	63.88	64.60	71.96
f20	56.64	47.16	62.04	62.40	69.84	59.04	59.48	58.80	56.60	65.80

(See report.pdf for the complete matrices.)

Links

• Extracted features and trained models (Google Drive): https://drive.google.com/drive/folders/1RobgJrxu0eVcw9vSvoDmmM75c19bzRMx?usp=sharing
• Task 2 walkthrough video (YouTube): https://youtu.be/kGKMdPuKPwM