Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
name: Bug report
description: Report a reproducible problem with the pipeline.
body:
- type: textarea
attributes:
label: What happened?
description: Include expected behavior and the exact command.
validations:
required: true
- type: textarea
attributes:
label: Environment and reproduction
description: Python version, OS, package versions, config, and a minimal example. Do not upload dataset images without permission.
validations:
required: true
- type: textarea
attributes:
label: Logs
render: shell
10 changes: 10 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
## What changes and why?

## Validation

Commands and results, including whether real data or synthetic fixtures were used.

## ML impact

Does this change data splits, preprocessing, class order, or the interpretation of metrics?
If reporting a result, link the run configuration and explain how the test split was protected.
38 changes: 38 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: CI
on:
push:
branches: [main, 'feat/**', 'fix/**']
pull_request:
permissions:
contents: read
jobs:
quality:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
with:
python-version: '3.11'
cache: pip
cache-dependency-path: uv.lock
- run: python -m pip install uv==0.13.6
- run: uv sync --frozen --extra dev
- run: uv run --no-sync ruff check .
- run: uv run --no-sync ruff format --check .
- run: uv run --no-sync pytest -m 'not ml' -q
- run: uv run --no-sync cat-vision --help
ml-smoke:
runs-on: ubuntu-latest
env:
CUDA_VISIBLE_DEVICES: '-1'
TF_CPP_MIN_LOG_LEVEL: '2'
steps:
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
with:
python-version: '3.11'
cache: pip
cache-dependency-path: uv.lock
- run: python -m pip install uv==0.13.6
- run: uv sync --frozen --extra ml --extra dev
- run: uv run --no-sync pytest -m ml -q
14 changes: 14 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
.venv/
__pycache__/
*.py[cod]
.pytest_cache/
.ruff_cache/
*.egg-info/
.DS_Store
.ipynb_checkpoints/
data/
runs/
*.keras
*.h5
.env
kaggle.json
19 changes: 19 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Changelog

## Unreleased — review corrections

- Require an explicit training config path so installed CLI runs do not assume the checkout is the current directory.
- Record oversized-image decompression errors as data rejections without disabling Pillow's safety limit.
- Snapshot Git provenance before creating run outputs, with regression coverage for clean and dirty source checkouts.

## 0.1.0 — reproducible project foundation

- Package the original cat-breed experiment as a CLI with prepare, train, evaluate and predict commands.
- Add deterministic stratified manifests, image validation, exact deduplication and dataset drift checks.
- Embed MobileNetV2 normalization in saved models and persist the label order.
- Select checkpoints using validation loss; report held-out metrics and per-image scores explicitly.
- Save configuration, manifest, environment and Git provenance with each run.
- Add tests, CI, pinned dependencies and a complete lockfile.
- Preserve the original notebook as historical work and document data, architecture, experiment protocol and model limitations.

No new real-data accuracy result or pretrained cat model is included in this release.
21 changes: 21 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Contributing

Cat Vision is a small hobby project with an emphasis on readable experiments. Focused fixes, clearer explanations, and reproducible ML improvements are welcome.

## Local setup

Use Python 3.11. Install `.[ml,dev]` in a virtual environment or use `uv sync --frozen --extra ml --extra dev`. Run `ruff check .`, `ruff format --check .`, and `pytest -q` before opening a pull request. CI runs the same checks using generated image fixtures; Kaggle access is never required.

## Changes to ML behavior

Explain the hypothesis and which data, preprocessing, label mapping, or metric semantics change. Add regression tests where contracts could break. Report actual commands and results. A unit test passing is evidence of software behavior, not model accuracy.

Use validation data for experiments, keep the final test set independent, and create a new run directory for each experiment. Commit new configs rather than editing old run artifacts. Update the data guide/model card if the assumptions change.

## Dependencies

Direct versions are pinned in `pyproject.toml`; regenerate `uv.lock` with `uv lock` after an intentional update and run both the lightweight and TensorFlow tests. Dependency updates should be separate from model-quality comparisons when possible. The historical notebook is excluded from formatting and linting so its saved experiment remains intact.

## Issues and pull requests

Include exact reproduction commands, Python/OS versions, relevant logs, and whether data is real or synthetic. Avoid credentials, personal images, datasets or generated model files in commits. Use a small synthetic fixture to reproduce a bug where possible.
8 changes: 8 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
.PHONY: install check test
install:
python -m pip install -e '.[ml,dev]'
check:
ruff check .
ruff format --check .
test:
pytest -q
144 changes: 141 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,141 @@
# Cat-Vision
Simple neural net for detecting cat breeds
As part of the kaggle competition: https://www.kaggle.com/datasets/shawngano/gano-cat-breed-image-collection
<p align="center">
<img src="assets/banner.svg" alt="Cat Vision — curiosity meets computer vision" width="100%">
</p>

<p align="center">
<a href="https://github.com/stefisha/Cat-Vision/actions/workflows/ci.yml"><img src="https://github.com/stefisha/Cat-Vision/actions/workflows/ci.yml/badge.svg" alt="CI"></a>
<img src="https://img.shields.io/badge/Python-3.11%20%7C%203.12-3776AB" alt="Python 3.11 and 3.12">
<img src="https://img.shields.io/badge/TensorFlow-2.21.0-FF6F00" alt="TensorFlow 2.21.0">
<a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-79d9d0" alt="Code license: MIT"></a>
</p>

I like cats. I also like understanding what a model has actually learned. **Cat Vision** brings those interests together: a small, reproducible project that classifies cat photos by breed using MobileNetV2 transfer learning.

It started as a Colab notebook. The current project adds the engineering needed to make an experiment inspectable: validated data, frozen splits, consistent preprocessing, traceable runs, held-out evaluation, and automated tests. The goal is a thoughtful portfolio project with a modest footprint.

> **Results status:** the original notebook recorded **88.16% training accuracy** across 15 breeds and 5,624 images. It had no held-out evaluation. The refactored pipeline has been tested on synthetic images; a real-data baseline has **not yet been measured**. No trained breed classifier or dataset is distributed here.

## What is here

- **Data integrity:** decode checks, byte-level deduplication, conflicting-label detection, SHA-256 image fingerprints, and a persisted stratified split.
- **Training:** a frozen ImageNet MobileNetV2 backbone, train-only horizontal flips, embedded pixel normalization, validation-loss early stopping, and best-checkpoint saving.
- **Evaluation:** held-out accuracy, macro F1, top-3 accuracy, per-breed metrics, confusion matrix, per-image scores, and a training-majority baseline.
- **Reproducibility:** one class mapping, seeded runs, configuration snapshots, dependency lockfile, package versions, and Git provenance.
- **Quality:** a packaged CLI, data contract tests, a no-download TensorFlow integration test, and GitHub Actions.

## Quick start

Use **Python 3.11** for the tested environment. Python 3.12 is allowed by the package; it is not in the CI matrix yet. The target platforms are Linux x86-64 and Apple Silicon macOS 12+. Intel macOS is not supported by this TensorFlow pin; use Linux or Colab there. CPU execution works; a GPU is optional. For accelerator setup, follow the [TensorFlow installation guide](https://www.tensorflow.org/install/pip).

```bash
git clone https://github.com/stefisha/Cat-Vision.git
cd Cat-Vision
python3.11 -m venv .venv
source .venv/bin/activate # Windows PowerShell: .venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install -e '.[ml,dev]'
cat-vision --help
```

For a fully resolved environment, use [uv](https://docs.astral.sh/uv/): `uv sync --frozen --extra ml --extra dev`, then run commands with `uv run --frozen --extra ml --extra dev`. The pip installation pins direct dependencies; `uv.lock` also fixes transitive dependencies. CI installs from the lockfile.

### 1. Bring your own cats

Download and extract the [Gano Cat Breed Image Collection](https://www.kaggle.com/datasets/shawngano/gano-cat-breed-image-collection). Review its current terms and image provenance before use or redistribution. Kaggle credentials and data stay out of Git.

Point `--data-dir` at the directory whose **immediate subdirectories are breed labels**:

```text
data/raw/Gano-Cat-Breeds-V1_1/
├── Abyssinian/
│ ├── cat-001.jpg
│ └── ...
├── Bengal/
└── ...
```

The pipeline discovers the labels; it does not hard-code 15 breeds. Each class needs at least three unique valid images. That minimum supports smoke testing, not a credible benchmark.

### 2. Validate and freeze the split

```bash
cat-vision prepare \
--data-dir data/raw/Gano-Cat-Breeds-V1_1 \
--manifest data/manifests/v1.json
```

This creates approximately **70% train / 15% validation / 15% test per class** using seed 42. Review the manifest's `rejected` and `duplicates` lists before training. Existing manifests are never silently overwritten. Training and evaluation verify file hashes, so modifying an image requires a new manifest and a new experiment.

Exact duplicates are removed before splitting. Near-duplicates and multiple photos of the same cat can still leak between splits; audit these before publishing performance claims. See the [data guide](docs/data.md).

### 3. Train a baseline

```bash
cat-vision train \
--data-dir data/raw/Gano-Cat-Breeds-V1_1 \
--manifest data/manifests/v1.json \
--config configs/baseline.json \
--run-dir runs/baseline-v1
```

`--config` is required; pass an absolute config path when running outside the checkout. The first baseline run downloads ImageNet weights. Validation loss selects `model.keras`; the test set is never supplied to `fit`. Run directories are immutable destinations: choose a new name for each experiment. See [training and experiments](docs/experiments.md) for the rationale and run artifacts.

### 4. Evaluate once the experiment is selected

```bash
cat-vision evaluate \
--data-dir data/raw/Gano-Cat-Breeds-V1_1 \
--run-dir runs/baseline-v1
```

Read `metrics.json` and `test_predictions.json` in the run directory. The confusion matrix uses the `classes` order, with **actual labels in rows and predictions in columns**. Compare macro F1 and per-breed recall with overall accuracy; inspect mistakes rather than selecting only attractive examples.

### 5. Try an image

```bash
cat-vision predict \
--run-dir runs/baseline-v1 \
--image /path/to/your/cat.jpg \
--top-k 3
```

The output is ranked breed scores. A softmax score is not a calibrated probability of ancestry. The model always chooses among its known labels—even for a mixed-breed cat, a dog, or an empty photo. See the [model card](docs/model-card.md).

## Project map

```text
src/cat_vision/ Data contracts, model, pipeline, CLI
configs/ Versioned experiment settings
notebooks/ Original exploratory experiment (historical)
tests/ Data tests and TensorFlow round-trip smoke test
docs/ Data guide, decisions, experiments, model card
assets/ Repository artwork
.github/ CI and contribution templates
uv.lock Resolved dependency versions
```

## Development

```bash
# With the activated environment from above:
ruff check .
ruff format --check .
pytest -q
```

The test suite creates tiny synthetic images and uses `weights=None`; it needs neither Kaggle data nor a pretrained-weight download. It verifies plumbing, not model quality. The lightweight subset can run without TensorFlow: install `'.[dev]'`, then `pytest -m 'not ml' -q`.

Read [CONTRIBUTING.md](CONTRIBUTING.md) for changes, [architecture](docs/architecture.md) for design decisions, and [verification](docs/verification.md) for the checks performed during this refactor.

## Next experiments

1. Audit near-duplicates and source groups, then publish a versioned real-data baseline.
2. Compare gentle fine-tuning against the frozen backbone using validation data only.
3. Examine calibration, class-specific errors, and photos outside the training distribution.
4. Add a small inference demo once a documented model is available.

## Credits and license

Dataset: [Gano Cat Breed Image Collection](https://www.kaggle.com/datasets/shawngano/gano-cat-breed-image-collection). Backbone: [MobileNetV2](https://arxiv.org/abs/1801.04381), via [Keras Applications](https://www.tensorflow.org/api_docs/python/tf/keras/applications/MobileNetV2). Original experiment: [historical notebook](notebooks/original_experiment.ipynb).

Code and repository artwork are under the [MIT license](LICENSE). Dataset images and third-party model weights retain their own terms. This is a hobby and learning project, not a tool for establishing a cat's ancestry or making care decisions.
12 changes: 12 additions & 0 deletions assets/banner.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
9 changes: 9 additions & 0 deletions configs/baseline.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"seed": 42,
"image_size": 224,
"batch_size": 32,
"epochs": 20,
"learning_rate": 0.001,
"patience": 4,
"weights": "imagenet"
}
50 changes: 50 additions & 0 deletions docs/architecture.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Architecture and decisions

## A small pipeline with explicit boundaries

```mermaid
flowchart LR
A[Breed folders] --> B[Decode + SHA-256 deduplication]
B --> C[Frozen stratified manifest]
C --> D[Training images]
C --> E[Validation images]
C --> F[Held-out test images]
D --> G[RGB resize + train-only flip]
G --> H[Embedded normalization + frozen MobileNetV2]
E --> I[Select checkpoint by validation loss]
H --> I
I --> J[Model + labels + config + provenance]
F --> K[Evaluate saved checkpoint]
J --> K
J --> L[Predict new photo]
```

`data.py` owns image decoding and manifest integrity. `model.py` owns the serialized network. `pipeline.py` connects those contracts to training and reporting. `cli.py` parses commands and reports actionable errors. Data preparation works without importing TensorFlow.

## Why these choices?

| Decision | Reason | Tradeoff |
| --- | --- | --- |
| Frozen MobileNetV2 | Small, understandable transfer-learning baseline; continues the original experiment | Fine-tuning may perform better, but must be measured |
| Global average pooling + dropout + softmax | Fewer head parameters than the original 256-unit dense head | Different model; historical accuracy is not directly comparable |
| Embedded `Rescaling(1/127.5, offset=-1)` | MobileNetV2 expects pixels scaled from 0–255 to -1–1; the saved model carries normalization | Callers must supply RGB pixels on the 0–255 scale |
| Shared Pillow RGB/EXIF/resize function | Same image semantics for train, test, and prediction, including GIF first frames | Python-backed decoding prioritizes shared semantics over maximum input throughput |
| Sorted labels in the manifest | Output index has one persisted meaning | Changing the label set requires a new run |
| SHA-256 manifest and per-image hashes | Detect accidental dataset drift and exact duplicates | Does not detect visually similar photos or identify the same animal |
| Per-class seeded split | Every supported class has examples in each partition | Source/animal groups need a future group-aware split |
| Validation-loss checkpoint | Select a model using validation evidence rather than training accuracy | Validation can still be overused during repeated experimentation |
| `.keras` run bundle | Straightforward round-trip with Keras 3 | Load only artifacts from a trusted source |
| Local artifacts and JSON config | Inspectable experiments without a service dependency | No experiment server or distributed execution |
| TensorFlow 2.21.0 + Python 3.11 baseline | Explicit, tested dependency baseline rather than an unbounded latest install | Periodic dependency updates need CI validation |

MobileNetV2 normalization follows the [official TensorFlow contract](https://www.tensorflow.org/api_docs/python/tf/keras/applications/MobileNetV2). EXIF correction and resizing happen before the network and are shared Python code; they are not part of the `.keras` graph. The graph embeds augmentation (active in training only) and normalization.

## Reproducibility boundaries

The manifest records relative paths, labels, content hashes, and split membership. A run copies it alongside configuration, class order, installed versions, Git revision and working-tree status. Completed runs fingerprint the configuration and checkpoint too; loading a run checks these fingerprints and the label order. Fixed seeds and TensorFlow deterministic operations reduce variability. They do not guarantee identical floating-point results across hardware, TensorFlow builds, or operating systems.

The canonical environment is in `uv.lock`; a run's provenance records what actually executed. The source Git revision and dirty flag are captured before creating the run directory, so the run’s own outputs cannot change the recorded starting state. A dirty Git tree is recorded explicitly because a revision alone cannot reconstruct uncommitted code. Commit code before collecting portfolio results.

## Scope

There is no API server, container stack, model registry, or production deployment here. The CLI and run bundles cover the current task. Add infrastructure when an actual deployment needs it, and keep ML evaluation evidence separate from application polish.
Loading
Loading