From 7b6efdfb8c2006dee011d29a7f78326eb0b758f7 Mon Sep 17 00:00:00 2001 From: Tyr Wiesner-Hanks Date: Fri, 31 Jul 2026 09:59:54 -0400 Subject: [PATCH 1/3] Initial README edits First-round edits to README, prior to attempting installation --- README.md | 98 ++++++++++++++++++++++++++++++++++--------------------- 1 file changed, 60 insertions(+), 38 deletions(-) diff --git a/README.md b/README.md index db33ef7..a1da84a 100644 --- a/README.md +++ b/README.md @@ -1,24 +1,39 @@ # MATS — Morphometric Analysis Toolbox Measure leaf **area, length, and width** in real-world units from a photo of -leaves laid on a printed calibration template. MATS finds four fiducial markers -with RF-DETR, corrects perspective, segments each leaf with a fast Otsu -threshold by default (or the heavier BiRefNet model for tougher backgrounds), -and writes a measurements CSV. +leaves laid on a printed calibration template. -Pipeline in one line: **detect markers → perspective-correct → segment leaf → -measure → CSV**. +MATS has four main steps: +1. Locate four fiducial markers using an RF-DETR detection model. +2. Applies a transform to undo any perspective distortion. +3. Segments each leaf, using either a fast Otsu threshold (default option) or a BiRefNet segmentation model for tougher backgrounds. +4. Writes out a CSV of the measurements. > Companion code for the manuscript (target journal: *Plant Phenomics*). > BiRefNet is optional and runs entirely from a locally installed checkpoint (see [Model weights](#model-weights)). > For USDA users the model weights are hosted on Agdatacommons and the pipeline is available on SciNET +## Table of Contents + +1. [Installation](#installation) +3. [Model weights](#model-weights) +4. 2. [Setup??](#) + +5. [Outputs](#outputs) +6. [Running on a compute cluster](#running-on-a-compute-cluster) +7. [How it works](#how-it-works) +8. [Troubleshooting](#troubleshooting) +9. [Citing](#citing) +10. [License](#license) + --- -## Install -MATS needs only Python ≥ 3.9 — QR codes are decoded with OpenCV, so the default -install pulls everything from wheels with **no system libraries and no conda + +## Installation + +MATS requires only Python ≥ 3.9; the recommended `pip` installation method +pulls all required packages from wheels with **no system libraries and no conda required**. **pip (recommended):** @@ -29,9 +44,9 @@ cd Morphometric-Analysis-Toolbox-for-Segmentation pip install -e ".[app]" # ".[app]" adds the Streamlit GUI ``` -**Enhanced QR reading (optional).** OpenCV decodes clean codes reliably; for -tougher photos (glare, skew, blur) you can add the `pyzbar` + `qreader` -fallbacks. `pyzbar` needs the system library `zbar`: +**Enhanced QR reading (optional).** OpenCV reads QR codes well when they are oriented correctly and clearly +visible. For images with any issues affecting the QR codes (glare, skew, blur) you can add the `pyzbar` + `qreader` fallbacks. +`pyzbar` requires the system library `zbar`: ```bash pip install -e ".[app,qr]" @@ -44,17 +59,40 @@ If a code can't be read, the pipeline continues — just pass the template size manually with `-t` (e.g. `-t 10.5x9.5in`), so enhanced QR is a convenience, not a requirement. -Then fetch the model weights once and confirm the environment: +Finally, fetch the model weights and confirm the environment: ```bash mats fetch-weights # fetches the ~134 MB RF-DETR checkpoint (mandatory, default) mats fetch-weights --only birefnet --source lfs # optional: explicitly fetch the ~2.65 GB BiRefNet checkpoint mats doctor # checks weights, GPU/CPU device, QR backends ``` +--- + +## Model weights + +The checkpoints for both the marker detection model and the leaf segmentation model are located in this repository: + +| Model | File | Size | +|---|---|---| +| RF-DETR marker detector | `rf_detr_marker.pth` | ~134 MB | +| BiRefNet leaf segmenter | `birefnet_leaf.pth` | ~2.65 GB | + +By default, only the RF-DETR model checkpoint will be downloaded. +The BiRefNet checkpoint is LFS-tracked but excluded from the default clone, so it is +downloaded only through an explicit action: + +- **Otsu (default)** — needs no BiRefNet checkpoint and never downloads one. +- **BiRefNet (optional)** — fetch explicitly with + `mats fetch-weights --only birefnet --source lfs`, or use the setup page. +- **Shared filesystem** — set `MATS_WEIGHTS_DIR` (e.g. a SCINet `/project` path) + to read weights in place with no per-user copy. + +Full details and checksums: [docs/weights.md](docs/weights.md). + --- -## Choose your path +## Running MATS - **I want to click buttons →** [Using the app](#using-the-app) - **I want to script it →** [Using the command line](#using-the-command-line) @@ -63,20 +101,22 @@ Both run the exact same pipeline and produce the same measurements. --- -## Using the app +### Using the app + +MATS comes with a point-and-click user interface. To open it, simply run: ```bash mats app ``` -This opens the Streamlit GUI in your browser. From there: +This will open the Streamlit app locally in your web browser. From there: 1. **Pick images** — a local folder, or drag-and-drop uploads. 2. **Set the scale** — enter the printed sheet's width, height, and unit (e.g. `10.5 x 9.5 in`), or tick **Variable dimensions, read QR code** to read it from each image's template QR code automatically. -3. **Choose segmentation** — Otsu threshold (fast, default) or BiRefNet (accurate when its optional local checkpoint is installed). -4. **Choose workers** — the app detects the CPUs assigned to it. One worker uses +3. **Choose segmentation** — Otsu threshold (fast, default) or BiRefNet (accurate, must have local model checkpoint installed). +4. **Choose workers** — the app detects the number of CPUs available to it. One worker uses CUDA/MPS when available; two or more workers use parallel CPU processing and disable CUDA/MPS for that run. A colored warning light shows CPU allocation; counts above 75% require a one-run break-glass acknowledgement. @@ -91,7 +131,7 @@ photograph it flat. See [docs/templates.md](docs/templates.md). --- -## Using the command line +### Using the command line ```bash mats run -i ./images -o ./out -r results.csv -t 10.5x9.5in @@ -150,29 +190,11 @@ A `leaf_morpho_failures.csv` records per-image warnings and failures. --- -## Model weights - -The checkpoints are tracked in this repository with Git LFS: - -| Model | File | Size | -|---|---|---| -| RF-DETR marker detector | `rf_detr_marker.pth` | ~134 MB | -| BiRefNet leaf segmenter | `birefnet_leaf.pth` | ~2.65 GB | - -RF-DETR is available in a normal checkout. BiRefNet is LFS-tracked but excluded -from the default clone, so it is downloaded only through an explicit action: - -- **Otsu (default)** — needs no BiRefNet checkpoint and never downloads one. -- **BiRefNet (optional)** — fetch explicitly with - `mats fetch-weights --only birefnet --source lfs`, or use the setup page. -- **Shared filesystem** — set `MATS_WEIGHTS_DIR` (e.g. a SCINet `/project` path) - to read weights in place with no per-user copy. -Full detail and checksums: [docs/weights.md](docs/weights.md). --- -## On a cluster (HPC / Open OnDemand) +## Running on a compute cluster An Open OnDemand Batch Connect app that serves the GUI on a compute node is in [deploy/ondemand/mats/](deploy/ondemand/mats/). See its README and From 0a8644e13b0b8183faaefd819090df3479c718a5 Mon Sep 17 00:00:00 2001 From: tyrwh Date: Wed, 5 Aug 2026 13:56:17 -0400 Subject: [PATCH 2/3] changing git-lfs calls and .lfsignore to get desired download behavior for larger birefnet weights --- .lfsconfig | 4 ++-- src/mats/weights.py | 51 +++++++++++++++++++++++++++++---------------- 2 files changed, 35 insertions(+), 20 deletions(-) diff --git a/.lfsconfig b/.lfsconfig index cafb65d..b6799f7 100644 --- a/.lfsconfig +++ b/.lfsconfig @@ -16,5 +16,5 @@ # # Note that a bare `git lfs pull` will NOT fetch it -- only -I/--include # overrides fetchexclude. That's deliberate: no accidental 2.65 GB pulls. -[lfs] - fetchexclude = weights/birefnet_leaf.pth +# [lfs] +# fetchexclude = weights/birefnet_leaf.pth diff --git a/src/mats/weights.py b/src/mats/weights.py index 4d47efe..ec40196 100644 --- a/src/mats/weights.py +++ b/src/mats/weights.py @@ -2,17 +2,15 @@ The checkpoints are large (RF-DETR ~134 MB, BiRefNet ~2.65 GB) and are delivered through two independent channels, plus a shared-filesystem escape -hatch -- :mod:`mats.paths` resolves whichever produced a real file: +hatch -- :mod:`mats.paths` resolves to whichever channel produces a real file: 1. **Hugging Face Hub** -- the default public host. Free, no account needed, but unreachable on some institutional networks (notably USDA's). -2. **Git LFS, in this repository** -- RF-DETR is fetched on every - ``git clone`` (mandatory for every run). BiRefNet is committed too, but - excluded from the default clone/fetch via ``.lfsconfig`` - (``lfs.fetchexclude``), so a plain clone stays small; it's pulled - explicitly through this module (or the BiRefNet setup page) when needed. - This channel exists because Hugging Face is not reachable from every - collaborator's network. +2. **Git LFS** -- By default, MATS will only pull the RF-DETR checkpoint file + when fetching weights via ``git-lfs pull``. To pull the larger BiRefNet file, + you can run ``mats fetch-weights --only birefnet --source lfs`` or just + ``git-lfs pull``. This channel exists because Hugging Face is not reachable + from every collaborator's network. 3. **A shared/mounted filesystem** (e.g. USDA SCINet ``/project``) -- point ``MATS_WEIGHTS_DIR`` at it and the weights are read in place, no download, for anyone who can mount it. @@ -207,7 +205,7 @@ def get_weight_status(name): checkout = _checkout_target(name) if checkout is not None and looks_like_lfs_pointer(checkout): - detail = "Excluded from `git clone` by design -- fetch it via Git LFS or Hugging Face." + detail = "Not yet fetched via Git LFS -- fetch it via the app, `mats fetch-weights --only birefnet` or Hugging Face." return WeightStatus(name, checkout, "missing", detail, 0, spec["size_bytes"], sources) target = _download_target(name) @@ -233,7 +231,7 @@ def _manual_instructions(): " (e.g. a shared SCINet /project path).\n" " - Or set RF_DETR_MARKER_CHECKPOINT / BIREFNET_CHECKPOINT to specific files.\n" " - Or, from a Git checkout with Git LFS installed:\n" - " git lfs pull --include=\"weights/birefnet_leaf.pth\"\n\n" + " git lfs pull\n\n" "See docs/weights.md.", file=sys.stderr, ) @@ -334,8 +332,13 @@ def _emit_lfs_progress(progress_path, progress_callback, fallback_total, last_do def _download_from_lfs(name, progress_callback=None): - """Fetch one checkpoint via `git lfs pull --include`, overriding this file's - .lfsconfig fetchexclude for just this invocation. + """Fetch one checkpoint via Git LFS. + + For BiRefNet, runs a plain ``git lfs pull`` (no flags) so the large + checkpoint is fetched without affecting other files. For all other + checkpoints, runs ``git lfs pull --exclude weights/birefnet_leaf.pth`` + so the 2.65 GB BiRefNet file is never pulled as a side-effect of an + unrelated weight update. Writes into the checkout's weights/ directory -- that's where Git LFS smudges content, and it's tier 3 of paths.py's resolution order, so the @@ -361,6 +364,17 @@ def _download_from_lfs(name, progress_callback=None): rel_path = f"weights/{spec['filename']}" print(f"Fetching {rel_path} via Git LFS -> {target}") + # For BiRefNet use a plain `git lfs pull` (no flags) -- without a + # fetchexclude in .lfsconfig a bare pull fetches all LFS files, which is + # what we want for this explicit opt-in download. + # For everything else, exclude the large BiRefNet checkpoint so it is + # never pulled as an unintended side-effect. + birefnet_rel = f"weights/{_MANIFEST['birefnet']['filename']}" + if name == "birefnet": + lfs_cmd = ["git", "lfs", "pull"] + else: + lfs_cmd = ["git", "lfs", "pull", "--exclude", birefnet_rel] + with tempfile.TemporaryDirectory() as tmp: progress_path = Path(tmp) / "progress" log_path = Path(tmp) / "output.log" @@ -370,7 +384,7 @@ def _download_from_lfs(name, progress_callback=None): # deadlocking the child if it writes enough to fill the OS pipe buffer. with open(log_path, "w") as log_file: proc = subprocess.Popen( - ["git", "lfs", "pull", "--include", rel_path], + lfs_cmd, cwd=_REPO_ROOT, env=env, stdout=log_file, stderr=subprocess.STDOUT, ) @@ -387,7 +401,8 @@ def _download_from_lfs(name, progress_callback=None): _emit(progress_callback, "verifying", 0, size) if not target.is_file() or looks_like_lfs_pointer(target): - print(f"error: {target} is still not a real file after `git lfs pull`.", file=sys.stderr) + print(f"error: {target} is still not a real file after `git-lfs pull`.", file=sys.stderr) + print(f"Please verify that you have Git LFS configured by running `git-lfs install`.") return False actual_size = target.stat().st_size @@ -516,12 +531,12 @@ def ensure_weight(name): f"{spec['filename']} not found and auto-fetch is disabled " f"({_AUTO_FETCH_DISABLED} is set). Pre-stage the weights, or run " f"`mats fetch-weights --only {name}` after unsetting {_AUTO_FETCH_DISABLED} " - f"(from a Git checkout, `git lfs pull --include=\"weights/{spec['filename']}\"` " - f"also works)." + f"(from a Git checkout, `git lfs pull` fetches all weights including birefnet, or " + f"`git lfs pull --exclude weights/{_MANIFEST['birefnet']['filename']}` fetches all others)." ) - # A checkout excludes BiRefNet from the default clone (.lfsconfig); a - # pointer stub here means "not yet pulled", not an error -- fetch it. + # A pointer stub for BiRefNet means the user hasn't run `git lfs pull` + # for it yet (it's large and opt-in) -- fetch it rather than failing. checkout = _checkout_target(name) if checkout is not None and looks_like_lfs_pointer(checkout) and _download_from_lfs(name): return checkout From 91a1ea3900a8929a40362bf190dd3ca89828ebc6 Mon Sep 17 00:00:00 2001 From: "A.J. Ackerman" Date: Mon, 21 Sep 2026 11:36:03 -0500 Subject: [PATCH 3/3] user agents/claude.md for chatbot interaction Ported from main (fbb3c09) onto dev at e7904eb. Agent instructions and docs - AGENTS.md: public agent guide, the single source of truth; CLAUDE.md imports it. docs/faq.md: human FAQ, linked from the README. - .gitignore: commit CLAUDE.md and AGENTS.md; ignore *.local.md. - README, FAQ, AGENTS: Git LFS is an install prerequisite (a clone made without it yields a 134-byte pointer stub, not the model), RF-DETR arrives with the clone, and `mats fetch-weights` is the repair path. Repair uses `git lfs pull --exclude=weights/birefnet_leaf.pth`, which stays RF-DETR-only whether or not .lfsconfig's fetchexclude is active. - README: reconciled with the restructure from PR #2; its headings and TOC are kept, and the hand-edited doctor/fetch paragraph is preserved verbatim. Also carried over from fbb3c09 (app work, unrelated to the docs) - src/mats/app/Home.py, branding.py, src/mats/core.py, pyproject.toml, tests/test_home_app.py Known and not addressed here - .lfsconfig has fetchexclude commented out on dev, so a clone now fetches BiRefNet (2.65 GB) as well as RF-DETR. - tests/test_weights.py::test_status_missing_for_excluded_checkout_pointer already fails on dev before this commit (message reworded in PR #2). Co-Authored-By: Claude Sonnet 5 --- .gitignore | 10 +- AGENTS.md | 225 +++++++++++++++++++++++++++++++++++++++ CHANGELOG.md | 7 ++ CLAUDE.md | 24 +++++ README.md | 99 +++++++++++++---- docs/faq.md | 221 ++++++++++++++++++++++++++++++++++++++ docs/weights.md | 5 + pyproject.toml | 2 +- src/mats/app/Home.py | 213 +++++++++++++++++++++++++++++------- src/mats/app/branding.py | 6 +- src/mats/core.py | 5 + tests/test_home_app.py | 124 +++++++++++++++++++++ 12 files changed, 881 insertions(+), 60 deletions(-) create mode 100644 AGENTS.md create mode 100644 CLAUDE.md create mode 100644 docs/faq.md diff --git a/.gitignore b/.gitignore index 124b515..f0dd5ca 100644 --- a/.gitignore +++ b/.gitignore @@ -39,7 +39,13 @@ env/ .vscode/ .idea/ -# AI-assistant working notes (local-only, not shipped) -CLAUDE.md +# AI-assistant instructions. +# Public and committed: AGENTS.md (the source of truth, read by Claude Code, +# Codex, Cursor, Copilot, Gemini CLI, ...) and CLAUDE.md (a thin file that +# imports it), so a fresh clone arrives with agent onboarding already in place. +# Private and never shipped: *.local.md working notes, which load alongside the +# public files for whoever created them, and the local tool directories. +CLAUDE.local.md +AGENTS.local.md .claude/ .agents/ diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..374b870 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,225 @@ +# MATS — instructions for AI coding agents + +Project instructions for any agent working in this repository (Claude Code, +Codex, Cursor, Copilot, Gemini CLI, …). Humans should start with +[README.md](README.md) and [docs/faq.md](docs/faq.md). + +**MATS (Morphometric Analysis Toolbox for Segmentation)** measures leaf area, +length, and width in real-world units from a photograph of leaves on a printed +calibration template: + +> detect the four fiducial markers (RF-DETR) → perspective-correct → segment the +> leaf (Otsu by default, BiRefNet optionally) → measure → CSV + +It ships as the installable package `mats-morpho` with one console script, +`mats`, and a Streamlit GUI. It is companion code for a scientific manuscript, +so **measurement correctness outranks convenience** in every trade-off. + +## Who you are helping + +Most people who clone this repo are **researchers measuring leaves**, not +contributors. Work out which one you have before answering: + +- **A user** — installing, running a batch, reading the CSV, or fixing bad + photos. They want working commands and a diagnosis, not a code tour. Jump to + [Getting a new user running](#getting-a-new-user-running) and + [Troubleshooting](#troubleshooting-playbook); point them at `docs/faq.md`. +- **A contributor** — changing the pipeline, the GUI, or packaging. See + [Working on the code](#working-on-the-code). + +When a user hits an environment problem, run `mats doctor` (or ask them to) +before theorising: it reports checkpoint resolution, the compute device, and +which QR decoders are usable. + +## Orientation + +| Where | What is in it | +|---|---| +| `README.md` | User-facing entry point: install, both run paths, outputs, troubleshooting | +| `docs/faq.md` | Human FAQ — installation, first run, QR, weights, units, GPU | +| `docs/cli.md` | Full `mats run` flag reference | +| `docs/gui.md` | The Streamlit workbench, page by page | +| `docs/weights.md` | Checkpoints, sha256s, Git LFS, `MATS_WEIGHTS_DIR`, resolution order | +| `docs/templates.md` | Template Creator rules: margins, marker sizes, printing | +| `docs/hpc.md` | Batch jobs and Open OnDemand on a cluster | +| `CHANGELOG.md` | What changed and when | + +Package layout (`src/mats/`): + +| Module | Role | +|---|---| +| `core.py` | The pipeline: marker detection, homography, segmentation, measurement, `run_leaf_morpho_batch` | +| `cli.py` | The `mats` entry point: `run`, `app`, `fetch-weights`, `doctor` | +| `paths.py` | Checkpoint **resolver** (import-light: no torch at module load) | +| `weights.py` | Checkpoint **delivery**: fetch channels, manifest, `doctor()`, LFS-pointer detection | +| `scaling.py` | Pixels-per-unit maths and unit conversion | +| `dimensions.py` | Parses template dimension strings (`12x12in`, `30x30cm`) | +| `qr_runtime.py` | QR decoding with OpenCV plus the optional pyzbar / QReader fallbacks | +| `devices.py` | CUDA / MPS / CPU selection | +| `birefnet_runtime.py`, `models/birefnet/` | BiRefNet loading and the pinned bundled architecture | +| `template_layout.py`, `template_exports.py` | Template geometry rules and the PDF / IDML exports | +| `samples.py` | Resolver for the packaged sample photos (`SAMPLES_DIR`, `SAMPLE_SETS`) | +| `app/` | Streamlit GUI: `Home.py` plus numbered `pages/N_Name.py` | +| `deploy/ondemand/mats/` | Open OnDemand Batch Connect app (GUI on a compute node) | +| `tests/` | Dependency-light unit tests — no torch, no network | + +## Getting a new user running + +The install needs **Python ≥ 3.9 and Git LFS**. Nothing else — no conda, no +other system libraries (QR codes are decoded with OpenCV). + +> **Check Git LFS first, before anything else.** The RF-DETR checkpoint +> (~134 MB) is stored in Git LFS and is mandatory for every run. A `git clone` +> on a machine without `git-lfs` **appears to succeed** but writes a 134-byte +> pointer stub in place of the model. This is the single most likely reason a +> fresh checkout fails to detect markers, and it looks like a model problem +> rather than a setup problem. Verify with `ls -l weights/rf_detr_marker.pth` +> (~134 MB, not ~134 bytes) or `mats doctor`; repair with +> `git lfs install && git lfs pull --exclude="weights/birefnet_leaf.pth"` — the `--exclude` matters: a bare `git lfs pull` can also +> fetch the 2.65 GB BiRefNet checkpoint. + +```bash +git lfs install # one-time, per machine, BEFORE cloning +git clone https://github.com/Breeding-Insight/Morphometric-Analysis-Toolbox-for-Segmentation.git +cd Morphometric-Analysis-Toolbox-for-Segmentation +pip install -e ".[app]" # ".[app]" adds the Streamlit GUI +mats doctor # checkpoints, device, QR decoders +``` + +The clone delivers RF-DETR — there is **no separate download step** for it. +`mats fetch-weights` exists to repair a checkout made without Git LFS, not as +part of a normal install; suggesting it as a routine step misleads users. + +Then either path — both run the same code and produce the same numbers: + +```bash +mats app # the GUI +mats run -i ./images -o ./out -r results.csv --sheet-dimensions 12x12in +``` + +Things worth telling a first-time user, in this order: + +1. **They need a printed template.** Measurements come from four corner markers + of known spacing. The GUI's **Template Creator** page produces a print-ready + PDF; print at 100 % scale ("fit to page" silently breaks calibration), lay + leaves inside the box, photograph flat with all four markers in frame. +2. **`--sheet-dimensions` is the finished printed sheet size**, e.g. `12x12in` — + MATS derives the marker-centre calibration area from it. Alternatively the + template's QR code can carry it per image. +3. **Otsu is the default** and needs no extra download. BiRefNet is the accurate + option for cluttered backgrounds and costs a ~2.65 GB checkpoint, fetched only + on request. +4. **They can try it with no data of their own.** Three de-identified sample + photographs ship with the package (`mats.samples.SAMPLES_DIR`) and are walked + through on the GUI's **Help** page — including a real QR-read failure that + shows why entering the printed sheet size by hand is the most reliable route. + +## Answer from the code, not from memory + +Model behaviour and flags have changed across versions. Before you state a flag, +a default, or a column name, check the source — `src/mats/cli.py`, `docs/cli.md`, +or `mats run --help`. + +Currently true, and worth knowing because users ask: + +- Subcommands are exactly `run`, `app`, `fetch-weights`, `doctor`. `mats` with + bare arguments implies `run`. +- Long flags accept both spellings where noted in `cli.py` (`--input_dir` and + `--input-dir`). +- Defaults that surprise people: `--mask-method threshold`, `--threshold-level + auto`, `--csv-schema full`, `--results-unit cm`, `--output-mode masks`. +- `mats fetch-weights` with no flag fetches **RF-DETR only**; BiRefNet needs + `--only birefnet` or `--all`. `--source {auto,hf,lfs}` picks the channel — + `lfs` is the one to use on networks that block huggingface.co. After a + successful clone it is a no-op that prints "already present". + +The two checkpoints are deliberately asymmetric in code, and it is easy to get +backwards: + +| | RF-DETR | BiRefNet | +|---|---|---| +| Loader calls | `weights.ensure_weight("rf-detr")` | `weights.require_local_weight("birefnet")` | +| Missing at run time | Fetched once, announced on stdout | **Never fetched** — raises with instructions | +| In the clone | Yes (Git LFS) | Intended: no — kept out by `lfs.fetchexclude` in `.lfsconfig`. **Verify it is active** with `git lfs env \| grep FetchExclude`; if that is empty, a clone or bare `git lfs pull` fetches BiRefNet (2.65 GB) too | +| In the GUI | Blocking Preflight error, no auto-download | Blocking only when BiRefNet is the selected method | + +Do not "fix" BiRefNet by switching it to `ensure_weight`: never starting a +2.65 GB download because someone picked a dropdown option is the intended +behavior, and `tests/test_weights.py` asserts it. + +## Measurement semantics — do not paraphrase loosely + +Scaling is **anisotropic**: each axis is calibrated independently against the +template. `width_cm` is the x-extent ÷ `px_per_cm_width`, `length_cm` is the +y-extent ÷ `px_per_cm_height`, and area is divided by *both*. There is no single +averaged scale factor — do not describe one. + +`scale_aspect_ratio` (`px_per_cm_width / px_per_cm_height`) is a QC signal: it +should sit near 1.0, and a value far from it flags a calibration problem (skewed +print, lens distortion, a non-planar sheet). Older CSVs with `*_meanscale`, +`*_widthscale`, `*_heightscale` columns predate this; the conversion is in the +README's migration note. + +**Never invent a measurement, an accuracy figure, or a citation.** If a number +isn't in the output or the docs, say so and run the pipeline to get it. + +## Troubleshooting playbook + +`mats doctor` first — it reports most of these. Then, by symptom: + +| Symptom | First move | +|---|---| +| QR code not read / no scale | Pass the sheet size explicitly: `--sheet-dimensions 12x12in`. Optional fallbacks: `pip install -e ".[qr]"` (QReader works without conda; pyzbar also needs the native `zbar`: `apt install libzbar0` / `brew install zbar` / `conda install -c conda-forge zbar`) | +| No markers detected | All four markers in frame? Printed at 100 % scale, in the Template Creator's marker colour? See `docs/templates.md` | +| BiRefNet unavailable | `mats fetch-weights --only birefnet --source lfs`, or the GUI's **BiRefNet setup** page | +| A `.pth` "loads" as text / torch errors on the checkpoint | A Git-LFS pointer stub (≤1024 bytes starting `version https://git-lfs.github.com/spec/v1`), not weights. `git lfs install && git lfs pull --exclude="weights/birefnet_leaf.pth"`. `weights.looks_like_lfs_pointer()` exists so torch is never handed one | +| CUDA out of memory | Only possible with `--mask-method birefnet`: use smaller batches, or the default `threshold` | +| Slow on CPU | Use `threshold`, and raise `-w/--workers`. Multiple workers disable CUDA/MPS for that run by design | +| Blank page on Open OnDemand | Almost always the reverse-proxy `baseUrlPath`; see `deploy/ondemand/mats/README.md` | + +Weights resolve in this order (`paths.py`), first hit wins: + +1. `RF_DETR_MARKER_CHECKPOINT` / `BIREFNET_CHECKPOINT` — explicit file paths +2. `MATS_WEIGHTS_DIR` — a shared or mounted directory, read in place, no copy +3. `~/.cache/mats/weights` (or `$XDG_CACHE_HOME/mats/weights`) +4. `/weights/`, then `./weights/` — the Git-LFS checkout + +Canonical filenames are `rf_detr_marker.pth` and `birefnet_leaf.pth`. Setting +`MATS_NO_AUTO_FETCH=1` turns off downloading entirely so a misconfigured path +fails fast — use it on HPC login nodes and air-gapped systems. + +## Working on the code + +```bash +pip install -e ".[app,dev]" +pytest # fast, offline, no torch required +``` + +Rules that keep this repo working: + +- **Tests stay offline and torch-free.** CI (`.github/workflows/ci.yml`) installs + with `pip install --no-deps -e .` on Python 3.9 and 3.11, so a new + module-level import of torch, rfdetr, or streamlit in an imported path breaks + CI even when it works locally. +- **`paths.py` stays import-light** — standard library only. Models load lazily + inside `core.py`; `samples.py`, `dimensions.py`, and `scaling.py` follow the + same contract. +- **The CLI and the GUI share one execution path** (`run_leaf_morpho_batch`). + Never fork pipeline logic between them — divergence would mean the two + interfaces report different measurements. +- **Never commit checkpoints.** `*.pth`, `*.pt`, `*.pkl`, `*.onnx` are gitignored + except the two LFS-tracked files already in `weights/`. +- Package data in `pyproject.toml` uses per-segment, extension-specific globs; a + new asset type or nesting level needs a new glob line or it won't ship in the + wheel. +- Streamlit pages are numbered (`app/pages/N_Name.py`) — the number sets sidebar + order. Match the surrounding style of whatever file you are editing. + +## Guardrails + +- Confirm before anything that downloads gigabytes, starts a long GPU run, or + writes into a user's image directories. +- Treat a user's input images and their results CSV as precious: write outputs to + the designated output folder, never overwrite inputs. +- Don't fabricate measurements, model accuracy claims, or citations. The + manuscript is not in this repository. diff --git a/CHANGELOG.md b/CHANGELOG.md index ec26775..3cb78d2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,13 @@ All notable changes to MATs are documented here. This project adheres to ## [Unreleased] +### Changed +- Documentation now matches the shipped weights-delivery behavior: Git LFS is + listed as an install prerequisite (a clone without it yields a 134-byte + pointer stub, not the model), RF-DETR is documented as arriving *with* the + clone rather than needing `mats fetch-weights`, and `mats fetch-weights` is + described as the repair path it is. + ### Added - A **Robust QR setup** sidebar page that explains the optional pyzbar and QReader fallbacks, reports their usable status, and keeps Conda optional. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..f639348 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,24 @@ +# MATS — Claude Code + +Leaf morphometrics from photographs of a printed calibration template: detect +markers → perspective-correct → segment → measure → CSV. + +Project instructions: @AGENTS.md + + + +## Claude-specific notes + +- Run `pytest` before proposing code changes — the suite is offline and takes + seconds. Do not add a module-level torch/streamlit import to a path it covers. +- Diagnose environment problems with `mats doctor` before reading code. +- `README.md` and `docs/` are user-facing and public: no internal planning, + unpublished results, or private paths belong in them. +- Never commit checkpoints (`*.pth`, `*.pt`) or a user's images. diff --git a/README.md b/README.md index 0239c35..013c6a1 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,6 @@ MATS has four main steps: > Companion code for the manuscript (target journal: *Plant Phenomics*). > BiRefNet is optional and runs entirely from a locally installed checkpoint (see [Model weights](#model-weights)). -> For USDA users the model weights are hosted on Agdatacommons and the pipeline is available on SciNET ## Table of Contents @@ -32,11 +31,27 @@ MATS has four main steps: ## Installation -MATS requires only Python ≥ 3.9; the recommended `pip` installation method -pulls all required packages from wheels with **no system libraries and no conda -required**. +MATS requires Python ≥ 3.9 and **Git LFS**. Apart from Git LFS, the recommended +`pip` installation method pulls all required packages from wheels with **no other +system libraries and no conda required**. -**pip (recommended):** +### Step 1 — install Git LFS *before* cloning + +The RF-DETR marker checkpoint (~134 MB) is stored with [Git LFS](https://git-lfs.com). +**If you clone without it, you get a 134-byte placeholder file instead of the +model** — the clone appears to succeed, and MATS then can't detect markers. + +```bash +# macOS: brew install git-lfs +# Debian/Ubuntu: sudo apt install git-lfs +# Conda: conda install -c conda-forge git-lfs +# Windows: included with Git for Windows +# RHEL/Fedora: sudo dnf install git-lfs + +git lfs install # one-time setup, per machine +``` + +### Step 2 — clone and install ```bash git clone https://github.com/Breeding-Insight/Morphometric-Analysis-Toolbox-for-Segmentation.git @@ -44,6 +59,22 @@ cd Morphometric-Analysis-Toolbox-for-Segmentation pip install -e ".[app]" # ".[app]" adds the Streamlit GUI ``` +The clone brings the RF-DETR checkpoint with it. Confirm it is the real file and +not a placeholder — it should be ~134 MB, not ~134 bytes: + +```bash +ls -l weights/rf_detr_marker.pth +``` + +**Already cloned without Git LFS?** No need to start over — install Git LFS as +above, then repair the checkout in place. The `--exclude` keeps this to the +~134 MB RF-DETR file; a bare `git lfs pull` can also fetch the 2.65 GB BiRefNet +checkpoint: + +```bash +git lfs install && git lfs pull --exclude="weights/birefnet_leaf.pth" +``` + **Enhanced QR reading (optional).** OpenCV reads QR codes well when they are oriented correctly and clearly visible. For images with any issues affecting the QR codes (glare, skew, blur) you can add the `pyzbar` + `qreader` fallbacks. `pyzbar` requires the system library `zbar`: @@ -65,25 +96,32 @@ If a code can't be read, the pipeline continues — pass the finished sheet size with `--sheet-dimensions` (for example, `--sheet-dimensions 12x12in`), so enhanced QR is a convenience rather than a requirement. -Finally, fetch the model weights and confirm the environment: +Finally, confirm the environment: ```bash -mats fetch-weights # fetches the ~134 MB RF-DETR checkpoint (mandatory, default) -mats fetch-weights --only birefnet --source lfs # optional: explicitly fetch the ~2.65 GB BiRefNet checkpoint mats doctor # checks weights, GPU/CPU device, QR backends ``` + +Run `mats doctor` after installing, this will report MATS operable status. +Note: RF-DETR weights are REQUIRED for marker detection, BiRefNet is OPTIONAL. +```bash +mats fetch-weights # repairs a clone made without Git LFS +mats fetch-weights --only birefnet --source lfs # optional: the ~2.65 GB BiRefNet checkpoint +``` + --- ## Model weights -The checkpoints for both the marker detection model and the leaf segmentation model are located in this repository: +The checkpoints for both the marker detection model and the leaf segmentation model are located in this repository, tracked with Git LFS: | Model | File | Size | |---|---|---| | RF-DETR marker detector | `rf_detr_marker.pth` | ~134 MB | | BiRefNet leaf segmenter | `birefnet_leaf.pth` | ~2.65 GB | -By default, only the RF-DETR model checkpoint will be downloaded. +By default, only the RF-DETR model checkpoint will be downloaded: it arrives with every +`git clone` made with Git LFS installed, so a normal checkout is immediately runnable. The BiRefNet checkpoint is LFS-tracked but excluded from the default clone, so it is downloaded only through an explicit action: @@ -93,6 +131,11 @@ downloaded only through an explicit action: - **Shared filesystem** — set `MATS_WEIGHTS_DIR` (e.g. a SCINet `/project` path) to read weights in place with no per-user copy. +**Cloned without Git LFS?** Both files come through as ~134-byte pointer stubs +rather than models, which MATS detects and reports rather than handing to +PyTorch. Fix it with `git lfs install && git lfs pull --exclude="weights/birefnet_leaf.pth"`, +or `mats fetch-weights`. + Full details and checksums: [docs/weights.md](docs/weights.md). @@ -101,9 +144,16 @@ Full details and checksums: [docs/weights.md](docs/weights.md). MATS installs in a lightweight **operating configuration** for convenience. The standard app includes fast Otsu segmentation and OpenCV's built-in QR reader, but it does not automatically download the optional ~2.65 GB BiRefNet -checkpoint or install the pyzbar/QReader robust-QR fallbacks. The required -~134 MB RF-DETR marker checkpoint is also fetched explicitly with `mats -fetch-weights` so installations never hide a model download. +checkpoint or install the pyzbar/QReader robust-QR fallbacks. + +The required ~134 MB RF-DETR marker checkpoint is different: it is mandatory for +every run, so it ships **in the clone** via Git LFS and needs no separate +download step. If it is ever missing — a clone made without Git LFS, or an +install outside a Git checkout — MATS fetches it once on first use and prints +`Fetching weights/rf_detr_marker.pth via Git LFS ...` while it does. Set +`MATS_NO_AUTO_FETCH=1` to turn that off and require pre-staged weights instead +(recommended on HPC login nodes). The app never does this silently: a missing +RF-DETR checkpoint is a blocking Preflight error. This keeps the initial network and disk footprint predictable, avoids native `zbar` failures on managed machines, and works better on HPC systems and @@ -114,7 +164,7 @@ photographs require: |---|---|---| | Otsu leaf segmentation | Yes | Nothing | | Clear QR codes with OpenCV | Yes | Nothing | -| RF-DETR marker detection | Code included | `mats fetch-weights` | +| RF-DETR marker detection | Yes — checkpoint ships in the clone (Git LFS) | Nothing | | BiRefNet segmentation | No checkpoint | `mats fetch-weights --only birefnet --source lfs` | | Robust QR fallbacks | No | `pip install "mats-morpho[app,qr]"` | @@ -129,6 +179,7 @@ a run. - **I want to click buttons →** [Using the app](#using-the-app) - **I want to script it →** [Using the command line](#using-the-command-line) +- **I have a question →** [FAQ](docs/faq.md) Both run the exact same pipeline and produce the same measurements. @@ -236,10 +287,6 @@ A `leaf_morpho_failures.csv` records per-image warnings and failures. > `leaf_area_cm2_widthscale * (px_per_cm_width / px_per_cm_height)`, and the new > `width_cm`/`length_cm` equal the old `width_cm_widthscale`/`length_cm_heightscale`. ---- - - - --- ## Running on a compute cluster @@ -267,7 +314,8 @@ detail. ## Troubleshooting -Run `MATS doctor` first — it reports most of these. +Run `mats doctor` first — it reports most of these, and the [FAQ](docs/faq.md) +covers the common questions in more detail. - **QR code not read / measurements need a scale** — the default OpenCV decoder couldn't read the code. Pass the finished sheet size with @@ -277,7 +325,11 @@ Run `MATS doctor` first — it reports most of these. conda: `conda install -c conda-forge zbar`). - **CUDA out of memory** (only relevant with `--mask-method birefnet`) — process in smaller batches, or use `--mask-method threshold` (the default). -- **No markers detected** — check print quality and that the marker color +- **No markers detected / "RF-DETR checkpoint missing"** — first check that the + checkpoint is a real file and not a Git LFS placeholder: + `ls -l weights/rf_detr_marker.pth` should show ~134 MB, not ~134 bytes. If it's + a placeholder, run `git lfs install && git lfs pull --exclude="weights/birefnet_leaf.pth"`. Otherwise check print + quality and that the marker color matches the template (the Template Creator uses the trained color); make sure all four corners are in frame. - **Blank page on Open OnDemand** — almost always the reverse-proxy @@ -285,6 +337,13 @@ Run `MATS doctor` first — it reports most of these. --- +## Working with an AI assistant + +This repository ships agent instructions in [AGENTS.md](AGENTS.md) (with a +companion [CLAUDE.md](CLAUDE.md)), so a coding assistant you point at your clone +— Claude Code, Codex, Cursor, Copilot, Gemini CLI — already knows how MATS is +installed, run, and structured, and can help you troubleshoot a batch. + ## Citing If you use MATS, please cite the manuscript. diff --git a/docs/faq.md b/docs/faq.md new file mode 100644 index 0000000..545b86f --- /dev/null +++ b/docs/faq.md @@ -0,0 +1,221 @@ +# FAQ + +Short answers for people who just cloned MATS. For the full reference see +[README.md](../README.md), [cli.md](cli.md), [gui.md](gui.md), +[templates.md](templates.md), [weights.md](weights.md), and [hpc.md](hpc.md). + +--- + +## Installing + +**What do I need?** Python ≥ 3.9 and Git LFS. Apart from Git LFS the install +pulls everything from wheels, with no conda environment and no other system +libraries. + +```bash +git lfs install # FIRST -- see the next question +git clone https://github.com/Breeding-Insight/Morphometric-Analysis-Toolbox-for-Segmentation.git +cd Morphometric-Analysis-Toolbox-for-Segmentation +pip install -e ".[app]" # ".[app]" adds the Streamlit app; drop it for CLI only +mats doctor # confirms checkpoints, device, and QR decoders +``` + +**Why do I need Git LFS?** The RF-DETR marker checkpoint (~134 MB) is stored +with [Git LFS](https://git-lfs.com), and it is required for every run. If you +clone without Git LFS installed, **the clone still succeeds** — but you get a +134-byte placeholder instead of the model, and MATS can't detect markers. + +Check it: + +```bash +ls -l weights/rf_detr_marker.pth # ~134 MB = good; ~134 bytes = placeholder +``` + +Repair an existing clone without re-cloning: + +```bash +git lfs install && git lfs pull --exclude="weights/birefnet_leaf.pth" +``` + +The `--exclude` keeps the repair to the ~134 MB RF-DETR file; a bare +`git lfs pull` can also fetch the 2.65 GB BiRefNet checkpoint. + +**Do I need to download the model first?** No. The clone brings RF-DETR with it, +so there is no separate download step — run `mats doctor` and you're done. (If +it reports the checkpoint missing, `mats fetch-weights` repairs it.) The large +BiRefNet checkpoint is the opposite: it is never downloaded unless you ask. + +**Why doesn't the install just download everything?** So the first install stays +predictable on laptops, managed machines, and clusters. You get fast Otsu +segmentation and OpenCV's QR reader immediately; the ~2.65 GB BiRefNet +checkpoint and the extra QR decoders are added only if your photographs need +them. + +**Do I need conda?** No. `conda` works if you already use it +(`environment.yml` is provided for clusters), but nothing requires it. + +**Do I need `zbar`?** Only for the optional `pyzbar` QR fallback. OpenCV reads +clear QR codes with no extra setup, and you can always enter the sheet size by +hand instead. + +**`mats: command not found`** — the console script landed outside your `PATH`, +usually from installing into a different interpreter. Check with +`python -m pip show mats-morpho`, and use the same Python you installed with: +`python -m mats.cli ...` works as a fallback. + +--- + +## Getting started + +**Fastest path from clone to a measurement?** Print a template, photograph +leaves on it, then: + +```bash +mats app # point and click +mats run -i ./images -o ./out -r results.csv --sheet-dimensions 12x12in +``` + +Both routes run exactly the same pipeline and produce the same numbers. + +**I don't have photographs yet.** Three de-identified sample images ship with +the package. Open `mats app` → **Help** in the sidebar: it walks through an easy +flat capture, a hard hand-held field capture, and a real QR-read failure, and +explains the settings each one needs. + +**What do I print?** Open **Template Creator** in the app, enter your finished +sheet's width and height, and download the PDF (an editable IDML is also +offered). Rules worth knowing: dimensions move in 0.5-unit steps, the +width-to-length ratio can't exceed 1.5:1, and margins and marker size are +derived for you. See [templates.md](templates.md). + +**Print at 100 % scale.** "Fit to page" or "shrink to fit" rescales the markers +and silently corrupts every measurement from that sheet. + +**How should I photograph the sheet?** Flat, evenly lit, with all four corner +markers inside the frame and the leaves inside the printed box. A hand-held +photo at a slight angle is fine — perspective correction handles it — but a +curled or folded sheet is not. + +--- + +## Running + +**What size do I enter — the sheet or the box?** The **finished printed sheet** +(for example `12x12in`). MATS derives the marker-centre calibration area from +it. Older or custom templates with different margins can still supply that area +directly via `-t/--template_dimensions` in the CLI, or the compatibility control +in the app. + +**Can it read the size from the template instead?** Yes — the Template Creator +puts a QR code on the sheet. In the app, tick **Variable dimensions, read QR +code**; in the CLI, leave `--sheet-dimensions` off. Per-image QR codes are what +you want when a batch mixes several template sizes. + +**Otsu or BiRefNet?** + +| | Otsu threshold (default) | BiRefNet | +|---|---|---| +| Speed | Fast, CPU-friendly | Slow without a GPU | +| Extra download | None | ~2.65 GB checkpoint | +| Best for | Clean, high-contrast backgrounds (a leaf on plain white) | Cluttered or low-contrast backgrounds | + +Start with the default. Switch with `--mask-method birefnet` only if the masks +disappoint you. + +**How many workers?** `-w/--workers` applies to the threshold path. One worker +uses CUDA/MPS when available; two or more switch to parallel CPU processing and +disable CUDA/MPS for that run. The app shows the CPUs allocated to it and warns +above 75 % usage. + +**Does it need a GPU?** No. The default path is CPU-only. A GPU helps only with +BiRefNet. + +--- + +## Model weights + +**Where do they come from?** Both checkpoints are tracked in this repository with +Git LFS. A normal `git clone` brings RF-DETR (~134 MB, required for every run). +BiRefNet (~2.65 GB) is meant to stay out of the default clone, so it +arrives only when you ask: + +```bash +mats fetch-weights --only birefnet --source lfs +``` + +**My network blocks huggingface.co.** Use `--source lfs`, which fetches through +the Git remote instead. + +**A checkpoint seems corrupt / torch complains about the file.** You probably +have a Git-LFS pointer stub — a ~130-byte text file, not the model. Run +`git lfs install && git lfs pull --exclude="weights/birefnet_leaf.pth"`. `mats doctor` flags this case. + +**Shared filesystem (lab server, HPC).** Stage the checkpoints once and point +everyone at them; nobody needs a personal copy: + +```bash +export MATS_WEIGHTS_DIR=/project/your_project/mats_weights +export MATS_NO_AUTO_FETCH=1 # fail fast instead of downloading on a login node +``` + +Full detail, checksums, and the complete resolution order: [weights.md](weights.md). + +--- + +## Results + +**What comes out?** Per image, a perspective-corrected `{sample_id}_target_box.jpg` +and a `{sample_id}_mask.png`, plus one measurements CSV and a +`leaf_morpho_failures.csv` listing anything that warned or failed. + +**Which columns?** `--csv-schema full` (the default) gives `sample_id`, +`leaf_area_cm2`, `width_cm`, `length_cm`, `px_per_cm_width`, `px_per_cm_height`, +`scale_aspect_ratio`, `source`, plus a QR trace when sizes are read from codes. +`--csv-schema compact` gives just `sample_id`, area, width, and length. The GUI's +Help page has a glossary for every column. + +**Can I get millimetres or inches?** Yes — `--results-unit mm|cm|in` (or +**Result units** in the app). The unit-bearing column names change to match. It +changes the reported units only, never the calibration maths. + +**Why isn't `scale_aspect_ratio` exactly 1.0?** Small deviations are normal. +MATS calibrates each axis independently, so this column is your quality check: a +value far from 1.0 means the horizontal and vertical scales disagree, which +usually points to a skewed print, a non-flat sheet, or strong lens distortion. +Re-print or re-photograph before trusting those rows. + +**I have CSVs from an older version.** Columns ending `_meanscale`, +`_widthscale`, and `_heightscale` predate the current anisotropic output. The +README's migration note gives the exact conversion — old files stay usable. + +**Can I check the length/width axes visually?** Add `--save-axes` to write +overlay images alongside the masks. + +--- + +## Troubleshooting + +Run `mats doctor` first; it reports most of these. + +| Symptom | Fix | +|---|---| +| No markers detected, on a fresh clone | Check for a Git LFS placeholder first: `ls -l weights/rf_detr_marker.pth` should be ~134 MB. If it's ~134 bytes, run `git lfs install && git lfs pull --exclude="weights/birefnet_leaf.pth"` | +| "QR code not read" | Pass the size yourself: `--sheet-dimensions 12x12in`. To add decoders: `pip install -e ".[qr]"` (plus the native `zbar` for pyzbar) | +| No markers detected | Get all four markers in frame; print at 100 % scale in the template's marker colour | +| Masks include the background | Try `--threshold-level low/medium/high`, or `--mask-method birefnet` | +| CUDA out of memory | Only with BiRefNet — process fewer images at a time, or use the default `threshold` | +| Very slow run | Use `threshold` and raise `-w/--workers` | +| Blank page in Open OnDemand | Reverse-proxy `baseUrlPath` mismatch — see [deploy/ondemand/mats/README.md](../deploy/ondemand/mats/README.md) | + +--- + +## Clusters and support + +**HPC?** MATS runs as an ordinary batch job, and an Open OnDemand app serves the +GUI on a compute node. See [hpc.md](hpc.md). + +**Something else is wrong.** Open an issue at +[github.com/Breeding-Insight/Morphometric-Analysis-Toolbox-for-Segmentation/issues](https://github.com/Breeding-Insight/Morphometric-Analysis-Toolbox-for-Segmentation/issues) +and include the output of `mats doctor`, the command you ran, and the error. + +**Citing MATS.** See [CITATION.cff](../CITATION.cff). diff --git a/docs/weights.md b/docs/weights.md index a516ed9..6ac7f88 100644 --- a/docs/weights.md +++ b/docs/weights.md @@ -57,6 +57,11 @@ mats fetch-weights --force # re-download even if present mats doctor # show resolved paths, channels, and source ``` +After a clone made with Git LFS installed, bare `mats fetch-weights` is a no-op +that prints "already present" — RF-DETR arrived with the checkout. The command +exists to repair a checkout made *without* Git LFS, and to populate a shared +`MATS_WEIGHTS_DIR`. + BiRefNet is never downloaded automatically. If it is absent when selected, MATs reports the missing local checkpoint and leaves Otsu fully usable. diff --git a/pyproject.toml b/pyproject.toml index 661c2a4..607603b 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -23,7 +23,7 @@ classifiers = [ # its pyzbar backend additionally needs the system `zbar` library, while # QReader works without it. dependencies = [ - "rfdetr", + "rfdetr==1.5.2", "transformers", "torch", "torchvision", diff --git a/src/mats/app/Home.py b/src/mats/app/Home.py index 85f6842..6c508c4 100644 --- a/src/mats/app/Home.py +++ b/src/mats/app/Home.py @@ -5,6 +5,8 @@ from pathlib import Path import altair as alt +import cv2 +import numpy as np import pandas as pd import streamlit as st @@ -259,9 +261,64 @@ def save_uploaded_images(uploaded_files, destination): PREVIEW_AUTO_HIDE_THRESHOLD = 50 PREVIEW_IMAGE_WIDTH = 280 LARGE_BATCH_THRESHOLD = 200 +OVERLAY_TINT_COLOR = (255, 0, 255) # BGR magenta -- reads clearly against green foliage +OVERLAY_TINT_ALPHA = 0.4 -def gather_output_files(output_dir, results_path): +def build_overlay_image(target_box, binary_mask, color=OVERLAY_TINT_COLOR, alpha=OVERLAY_TINT_ALPHA): + """Blend a translucent tint + contour outline over the segmented region, for QC review.""" + mask_bool = binary_mask.astype(bool) + tint = np.full_like(target_box, color, dtype=target_box.dtype) + blended = cv2.addWeighted(target_box, 1.0 - alpha, tint, alpha, 0) + overlay = target_box.copy() + overlay[mask_bool] = blended[mask_bool] + contours, _ = cv2.findContours(binary_mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE) + thickness = max(2, min(6, min(target_box.shape[:2]) // 300)) + cv2.drawContours(overlay, contours, -1, color, thickness, cv2.LINE_AA) + return overlay + + +def build_cutout_image(target_box, binary_mask): + """Isolate the segmented leaf pixels; everything outside the mask is black.""" + return cv2.bitwise_and(target_box, target_box, mask=binary_mask) + + +def generate_export_overlays(output_dir, include_overlay, include_cutout): + """Materialize {sample}_overlay.jpg / {sample}_cutout.jpg for every mask+target-box pair. + + Always regenerates: output_dir can be reused across runs with a different mask + method or source images, so a stale derived file from a prior run must not be + served instead of one matching the current mask. + """ + if not include_overlay and not include_cutout: + return + output_dir = Path(output_dir) + for mask_path in sorted(output_dir.glob("*_mask.png")): + sample_id = mask_path.name[: -len("_mask.png")] + target_box_path = output_dir / f"{sample_id}_target_box.jpg" + if not target_box_path.is_file(): + continue + + binary_mask = cv2.imread(str(mask_path), cv2.IMREAD_GRAYSCALE) + target_box = cv2.imread(str(target_box_path), cv2.IMREAD_COLOR) + if binary_mask is None or target_box is None: + continue + if binary_mask.shape[:2] != target_box.shape[:2]: + continue + + if include_overlay: + cv2.imwrite( + str(output_dir / f"{sample_id}_overlay.jpg"), + build_overlay_image(target_box, binary_mask), + ) + if include_cutout: + cv2.imwrite( + str(output_dir / f"{sample_id}_cutout.jpg"), + build_cutout_image(target_box, binary_mask), + ) + + +def gather_output_files(output_dir, results_path, include_overlay=False, include_cutout=False): """Return the artifact files an export ZIP should contain.""" output_dir = Path(output_dir) files = [] @@ -273,6 +330,10 @@ def gather_output_files(output_dir, results_path): files.append(failures_path) files.extend(sorted(output_dir.glob("*_target_box.jpg"))) files.extend(sorted(output_dir.glob("*_mask.png"))) + if include_overlay: + files.extend(sorted(output_dir.glob("*_overlay.jpg"))) + if include_cutout: + files.extend(sorted(output_dir.glob("*_cutout.jpg"))) return files @@ -1598,7 +1659,7 @@ def render_results(lm): icon=":material/folder_off:", ) render_output_preview(st.session_state.get("viewer_pairs", [])) - render_zip_export(results_path, output_path) + render_export_section(results_path, output_path, unit_symbol) return total_rows = count_csv_rows(results_path) @@ -1805,17 +1866,10 @@ def render_results(lm): unit_symbol, area_symbol, ) - st.download_button( - f"Download results CSV ({unit_symbol})", - data=results_path.read_bytes(), - file_name=results_path.name, - mime="text/csv", - icon=":material/download:", - ) + render_export_section(results_path, output_path, unit_symbol) with st.expander("Browse Output Previews", icon=":material/photo_library:"): render_output_preview(st.session_state.get("viewer_pairs", [])) - render_zip_export(results_path, output_path) def render_specimen_inspector(sample_id, measurement, pairs, unit_symbol="cm", area_symbol="cm²"): @@ -1912,39 +1966,126 @@ def render_output_preview(pairs, selected_sample_id=None): render_output_pair(pair) -def render_zip_export(results_path, output_path): - files = gather_output_files(output_path, results_path) - if not files: - return +def _clear_export_zip_cache(): + """Invalidate a previously prepared ZIP when the export selection changes.""" + st.session_state.pop("export_zip_path", None) - count, total_bytes = estimate_zip_inputs(files) - st.markdown("**Export**") - st.caption(f"{count} file(s), ~{human_bytes(total_bytes)} uncompressed.") - if total_bytes > ZIP_SIZE_WARN_BYTES: - st.warning( - f"Outputs total ~{human_bytes(total_bytes)}. Building a ZIP this large can be " - f"slow and memory-heavy. Consider collecting files directly from " - f"`{display_path(output_path)}` " - "instead." +def render_export_section(results_path, output_path, unit_symbol): + """Render a clearly defined Export section: CSV-only vs. the full ZIP bundle.""" + output_path = Path(output_path) + st.subheader("Export", anchor=False) + st.caption("Download the outputs from this run. Pick exactly what you need.") + csv_column, zip_column = st.columns(2, vertical_alignment="top") + + with csv_column.container(border=True): + st.markdown("**Measurements only**") + st.caption("Just the results CSV — sample IDs, area, width, length. No images.") + if results_path.is_file(): + st.download_button( + f"Download results CSV ({unit_symbol})", + data=results_path.read_bytes(), + file_name=results_path.name, + mime="text/csv", + icon=":material/download:", + key="download_csv_only", + ) + else: + st.caption("Not available yet.") + + base_files = gather_output_files(output_path, results_path) + with zip_column.container(border=True): + st.markdown("**Full export (ZIP)**") + st.caption( + "Results CSV + failure log + segmentation masks + specimen photos, " + "bundled together." ) + if not base_files: + st.caption("No output files found yet.") + return - if st.button("Prepare ZIP for download"): - with st.spinner("Building ZIP..."): - dest = Path(tempfile.gettempdir()) / "leaf_morpho_outputs.zip" - write_output_zip(files, dest) - st.session_state["export_zip_path"] = str(dest) + pair_count = len(list(output_path.glob("*_mask.png"))) + include_overlay = st.checkbox( + "Include overlay images (mask highlighted on photo)", + key="export_include_overlay", + help=( + "One extra JPG per specimen: the photo with the detected leaf " + "region tinted and outlined, for visually checking segmentation " + "accuracy." + ), + on_change=_clear_export_zip_cache, + persist_state="page", + ) + include_cutout = st.checkbox( + "Include specimen cutouts (background removed)", + key="export_include_cutout", + help=( + "One extra JPG per specimen: just the leaf pixels, with the " + "background blacked out." + ), + on_change=_clear_export_zip_cache, + persist_state="page", + ) + enabled_extra_count = int(include_overlay) + int(include_cutout) + + count, total_bytes = estimate_zip_inputs(base_files) + if enabled_extra_count and pair_count: + target_box_paths = list(output_path.glob("*_target_box.jpg")) + avg_target_box_bytes = ( + sum(path.stat().st_size for path in target_box_paths) / len(target_box_paths) + if target_box_paths + else 0 + ) + count += pair_count * enabled_extra_count + total_bytes += int(pair_count * avg_target_box_bytes * enabled_extra_count) - zip_path = st.session_state.get("export_zip_path") - if zip_path and Path(zip_path).is_file(): - with open(zip_path, "rb") as zf: - st.download_button( - "Download ZIP (target boxes, masks, CSV)", - data=zf, - file_name="leaf_morpho_outputs.zip", - mime="application/zip", + st.caption(f"{count} file(s), ~{human_bytes(total_bytes)} uncompressed.") + if total_bytes > ZIP_SIZE_WARN_BYTES: + st.warning( + f"Outputs total ~{human_bytes(total_bytes)}. Building a ZIP this large can be " + f"slow and memory-heavy. Consider collecting files directly from " + f"`{display_path(output_path)}` " + "instead." ) + if st.button("Prepare ZIP for download", icon=":material/folder_zip:"): + large_batch = enabled_extra_count and pair_count > LARGE_BATCH_THRESHOLD + if large_batch: + with st.spinner("Generating overlay/cutout images..."): + generate_export_overlays(output_path, include_overlay, include_cutout) + with st.spinner("Building ZIP..."): + files = gather_output_files( + output_path, results_path, include_overlay, include_cutout + ) + dest = Path(tempfile.gettempdir()) / "leaf_morpho_outputs.zip" + write_output_zip(files, dest) + else: + with st.spinner("Building ZIP..."): + generate_export_overlays(output_path, include_overlay, include_cutout) + files = gather_output_files( + output_path, results_path, include_overlay, include_cutout + ) + dest = Path(tempfile.gettempdir()) / "leaf_morpho_outputs.zip" + write_output_zip(files, dest) + st.session_state["export_zip_path"] = str(dest) + + zip_path = st.session_state.get("export_zip_path") + if zip_path and Path(zip_path).is_file(): + contents = ["target boxes", "masks", "CSV"] + if include_overlay: + contents.append("overlays") + if include_cutout: + contents.append("cutouts") + with open(zip_path, "rb") as zf: + st.download_button( + f"Download ZIP ({', '.join(contents)})", + data=zf, + file_name="leaf_morpho_outputs.zip", + mime="application/zip", + icon=":material/download:", + key="download_zip_export", + ) + if __name__ == "__main__": main() diff --git a/src/mats/app/branding.py b/src/mats/app/branding.py index 69e75e9..7d30f86 100644 --- a/src/mats/app/branding.py +++ b/src/mats/app/branding.py @@ -11,7 +11,8 @@ # st.logo's biggest built-in size ("large") renders at 2rem; override it to # stay legible at header scale. _LOGO_HEIGHT = "8rem" -_SIDEBAR_LOGO_TOP_OFFSET = "0.5rem" +_SIDEBAR_LOGO_TOP_OFFSET = "2rem" +_SIDEBAR_NAV_TOP_GAP = "0.5rem" # The collapsed-state header icon (stHeaderLogo) lives in a fixed 3.75rem # header bar, so it gets its own modest size -- not the sidebar's 8rem -- plus @@ -35,6 +36,9 @@ [data-testid="stLogoSpacer"] {{ height: calc({_LOGO_HEIGHT} + {_SIDEBAR_LOGO_TOP_OFFSET}) !important; }} +[data-testid="stSidebarNav"] {{ + padding-top: {_SIDEBAR_NAV_TOP_GAP} !important; +}} [data-testid="stHeaderLogo"] {{ height: {_HEADER_LOGO_HEIGHT} !important; margin-top: 0.75rem !important; diff --git a/src/mats/core.py b/src/mats/core.py index fd92bf2..ad22ce8 100644 --- a/src/mats/core.py +++ b/src/mats/core.py @@ -72,6 +72,7 @@ from .devices import available_cpu_workers, birefnet_device_report, worker_risk_report RF_DETR_MARKER_RESOLUTION = 1120 +RF_DETR_MARKER_POSITIONAL_ENCODING_SIZE = 44 RF_DETR_MARKER_CONFIDENCE = 0.5 RF_DETR_MARKER_PAD_COLOR = (0, 0, 0) BIREFNET_IMAGE_SIZE = 2048 @@ -116,6 +117,10 @@ def get_marker_model(device_override=None): checkpoint = weights.ensure_weight("rf-detr") # resolves or auto-fetches once model = RFDETRLarge( resolution=RF_DETR_MARKER_RESOLUTION, + # Trial 00168 was trained at 1120 px with a learned 44 x 44 + # positional-embedding grid, interpolated during inference. + positional_encoding_size=RF_DETR_MARKER_POSITIONAL_ENCODING_SIZE, + num_classes=1, pretrain_weights=str(checkpoint), device=device, ) diff --git a/tests/test_home_app.py b/tests/test_home_app.py index 56fc079..c583e00 100644 --- a/tests/test_home_app.py +++ b/tests/test_home_app.py @@ -1,16 +1,22 @@ from pathlib import Path +import numpy as np import pytest pd = pytest.importorskip("pandas") pytest.importorskip("streamlit") +cv2 = pytest.importorskip("cv2") from streamlit.testing.v1 import AppTest from mats.app.Home import ( PENDING_WORKSPACE_TAB_KEY, WORKSPACE_TAB_KEY, _WORKBENCH_STYLES, _resolve_sheet_layout, + build_cutout_image, + build_overlay_image, collect_output_pairs, + gather_output_files, + generate_export_overlays, merge_viewer_pairs, normalize_measurements, summarize_measurements, @@ -83,6 +89,92 @@ def test_merge_viewer_pairs_accumulates_current_session_without_duplicates(): assert pairs == [updated, second] +def _write_real_output_pair(output_dir, sample_id, size=10, fill=200): + """Write a real, decodable target-box JPG + binary mask PNG for image-helper tests.""" + target_box = np.full((size, size, 3), fill, dtype=np.uint8) + binary_mask = np.zeros((size, size), dtype=np.uint8) + binary_mask[2:-2, 2:-2] = 255 + target_box_path = output_dir / f"{sample_id}_target_box.jpg" + mask_path = output_dir / f"{sample_id}_mask.png" + cv2.imwrite(str(target_box_path), target_box) + cv2.imwrite(str(mask_path), binary_mask) + return target_box_path, mask_path + + +def test_build_overlay_image_tints_only_the_masked_region(): + target_box = np.zeros((10, 10, 3), dtype=np.uint8) + binary_mask = np.zeros((10, 10), dtype=np.uint8) + binary_mask[2:8, 2:8] = 255 + + overlay = build_overlay_image(target_box, binary_mask, color=(0, 255, 0), alpha=1.0) + + assert overlay.shape == target_box.shape + assert tuple(overlay[0, 0]) == (0, 0, 0) + assert tuple(overlay[5, 5]) == (0, 255, 0) + + +def test_build_cutout_image_blacks_out_everything_outside_the_mask(): + target_box = np.full((10, 10, 3), 200, dtype=np.uint8) + binary_mask = np.zeros((10, 10), dtype=np.uint8) + binary_mask[2:8, 2:8] = 255 + + cutout = build_cutout_image(target_box, binary_mask) + + assert tuple(cutout[0, 0]) == (0, 0, 0) + assert tuple(cutout[5, 5]) == (200, 200, 200) + + +def test_generate_export_overlays_writes_only_the_requested_kinds(tmp_path): + _write_real_output_pair(tmp_path, "leaf_1") + + generate_export_overlays(tmp_path, include_overlay=True, include_cutout=False) + + assert (tmp_path / "leaf_1_overlay.jpg").is_file() + assert not (tmp_path / "leaf_1_cutout.jpg").is_file() + + +def test_generate_export_overlays_skips_masks_without_a_target_box(tmp_path): + (tmp_path / "orphan_mask.png").write_bytes(b"not a real png but presence is what matters") + + generate_export_overlays(tmp_path, include_overlay=True, include_cutout=True) + + assert not (tmp_path / "orphan_overlay.jpg").is_file() + assert not (tmp_path / "orphan_cutout.jpg").is_file() + + +def test_generate_export_overlays_regenerates_stale_files(tmp_path): + # JPEG re-encoding is lossy, so compare with tolerance rather than exact equality. + _write_real_output_pair(tmp_path, "leaf_1", fill=200) + generate_export_overlays(tmp_path, include_overlay=False, include_cutout=True) + first_cutout = cv2.imread(str(tmp_path / "leaf_1_cutout.jpg")) + assert abs(int(first_cutout[5, 5][0]) - 200) <= 5 + + _write_real_output_pair(tmp_path, "leaf_1", fill=50) + generate_export_overlays(tmp_path, include_overlay=False, include_cutout=True) + second_cutout = cv2.imread(str(tmp_path / "leaf_1_cutout.jpg")) + + assert abs(int(second_cutout[5, 5][0]) - 50) <= 5 + + +def test_gather_output_files_includes_overlay_and_cutout_only_when_requested(tmp_path): + _write_real_output_pair(tmp_path, "leaf_1") + generate_export_overlays(tmp_path, include_overlay=True, include_cutout=True) + results_path = tmp_path / "leaf_morpho_results.csv" + results_path.write_text("sample_id\nleaf_1\n") + + plain = gather_output_files(tmp_path, results_path) + with_extras = gather_output_files( + tmp_path, results_path, include_overlay=True, include_cutout=True + ) + + plain_names = {path.name for path in plain} + extra_names = {path.name for path in with_extras} + assert "leaf_1_overlay.jpg" not in plain_names + assert "leaf_1_cutout.jpg" not in plain_names + assert "leaf_1_overlay.jpg" in extra_names + assert "leaf_1_cutout.jpg" in extra_names + + def test_home_page_renders_analyze_view_without_worker_control(): app = AppTest.from_file(str(HOME_PAGE)).run(timeout=30) @@ -279,6 +371,38 @@ def test_results_tab_uses_the_completed_run_unit(tmp_path): assert app.download_button[0].label == "Download results CSV (in)" +def test_results_tab_has_a_clearly_defined_export_section(tmp_path): + results_path = tmp_path / "leaf_morpho_results.csv" + results_path.write_text( + "sample_id,leaf_area_cm2,width_cm,length_cm\nleaf_1,12.5,2.5,7.0\n" + ) + app = AppTest.from_file(str(HOME_PAGE)) + app.session_state[WORKSPACE_TAB_KEY] = "Results" + app.session_state["last_run"] = { + "succeeded": 1, + "failed": 0, + "total": 1, + "workers": 1, + "worker_reason": "test", + "execution_device": "cpu", + "failure_rows": [], + "failure_overflow": 0, + "results_path": str(results_path), + "output_path": str(tmp_path), + "mask_method": "threshold", + } + app.run(timeout=30) + + assert not app.exception + assert any(item.value == "Export" for item in app.subheader) + markdown_values = {item.value for item in app.markdown} + assert "**Measurements only**" in markdown_values + assert "**Full export (ZIP)**" in markdown_values + checkbox_labels = {item.label for item in app.checkbox} + assert "Include overlay images (mask highlighted on photo)" in checkbox_labels + assert "Include specimen cutouts (background removed)" in checkbox_labels + + def test_results_tab_shows_qr_trace_when_full_qr_columns_are_present(tmp_path): results_path = tmp_path / "leaf_morpho_results.csv" results_path.write_text(