[Project516]

What this changes

Checks whether Project516's rebuilt-fuel YOLO26n (v1) can replace or sit beside Team 2826's YOLO11n on the Jetson, and makes the export reproducible.

• tools/fuelmodel/export.py replaces the laptop-only ~/build/yolo/export_fuel.py. It downloads rebuilt-fuel-model-v1.pt at a pinned Hugging Face revision, checks its SHA-256, exports the raw head, and fails unless the ONNX output is [1, 5, 8400], the layout detector/TensorRtYoloJNI.cu reads.
• tools/fuelmodel/eval.py scores both models on the 53 rebuilt-fuel photos, in colour and in gray (what a mono Thriftiest Cam gives the model). Both models go through the same letterbox, NMS at IoU 0.45 and COCO-style AP (101-point interpolation with step precision, as pycocotools does). Team 2826's model is PhotonVision's int8 TFLite at a pinned commit. The photos, train.txt and the TFLite are all pinned, and the downloads are checked by SHA-256.
• docs/GAME-PIECE-MODELS.md gets the results and an updated "Which to use".
• tools/fuelmodel/requirements.txt and the README pin the export dependencies, including torch and torchvision, to the versions the Actions run used.

Rebased onto main at ab6e086 on 2026-10-12. The rebase had no conflicts, and main's new commits don't touch tools/fuelmodel or docs/GAME-PIECE-MODELS.md.

Results

Run on GitHub Actions, 2026-09-30, after the AP fix. "Held out" is the 11 photos v1 did not train on, rebuilt from v1's own split.

Model	Photos	Balls	Image	AP50	AP50-95	Recall @0.5	Recall @0.9
Project516 v1	11 held out	13	colour	1.000	0.932	1.000	0.769
Team 2826	11 held out	13	colour	1.000	0.772	1.000	0.615
Project516 v1	11 held out	13	gray	1.000	0.844	0.923	0.000
Team 2826	11 held out	13	gray	1.000	0.731	1.000	0.615
Team 2826	all 53	111	colour	0.817	0.537	0.784	0.586
Team 2826	all 53	111	gray	0.743	0.509	0.604	0.495

• v1 can be used. The export matches our TensorRT runner, and in colour it is at least as good as Team 2826's model, with tighter boxes.
• On our mono cameras, keep Team 2826's model. In gray, v1 still finds the balls but none pass PhotonVision's default 0.9 confidence.
• On a colour camera, try v1 at a 0.5 threshold and A/B it on Rewind recordings before switching.
• The sample is small (13 held-out balls), and these are phone photos, not our camera.

Checks

• GitHub Actions on a temporary branch, removed afterwards and not in this PR: the export produced project516_yolo26n_fuel_v1_raw.onnx with output [1, 5, 8400], onnxruntime ran it, and eval.py printed the table above. Last run: https://github.com/Project516/SpectrumJetson/actions/runs/36801619049
• An earlier run used Ultralytics' own validator for v1 and gave close numbers (held-out colour mAP50 0.995, AP50-95 0.928). That validator cannot feed PhotonVision's uint8 TFLite, so eval.py uses one evaluator for both models.
• The 2026-09-30 run used the pinned requirements and the step-precision AP. AP50 went from 0.990 to 1.000 on the held-out photos, and AP50-95 moved by up to 0.04. The recall columns did not change.
• scripts/ci/lint.sh (main's new lint job) is clean on the rebased branch.
• Not run: a TensorRT engine build or inference on the Jetson. 12-install-yolo-model.sh has built v1's raw ONNX before, per the existing notes in GAME-PIECE-MODELS.md.

Noticed, not fixed

• train.txt in rebuilt-fuel lists 54 photos, but the dataset has 53. v1's training script skipped the missing one, and eval.py does the same.
• The Team 2826 numbers come from the int8 TFLite. The Jetson runs an FP16 engine built from Team 2826's .pt, which may score a little higher.

[Project516]

@allengregoryiv @SpectrumFRC3847 ready for a review.

Rebased onto main at e9b82ed. No conflicts. I re-ran python3 -m py_compile on export.py and eval.py and git diff --check, both clean. I did not recompute the accuracy table, so those numbers still come from the 2026-09-25 Actions run linked in the body.

What I would like checked:

• Whether the conclusion is right. The result says keep Team 2826's model on our mono cameras, because in gray v1 finds the balls but none pass PhotonVision's default 0.9 confidence, and try v1 at 0.5 on a colour camera. If that threshold guidance is wrong, the model choice flips.
• Whether the evaluation is fair. Both models go through one letterbox, one NMS setting, and one COCO-style AP implementation, because Ultralytics' own validator cannot feed PhotonVision's uint8 TFLite. That is the reason for eval.py existing, but it also means the numbers depend on my evaluator being right.

Caveats worth weighing before anyone acts on this: 13 held-out balls, so one miss moves recall by 0.077. These are phone photos, not our camera. Team 2826's numbers come from the int8 TFLite while the Jetson runs an FP16 engine, so they may be understated.

The main practical value here is that tools/fuelmodel/export.py replaces a laptop-only script in ~/build with a pinned, checksummed, reproducible export that fails loudly if the ONNX output is not the [1, 5, 8400] layout our TensorRT runner reads. That part is worth landing even if the model comparison gets re-run.

[Copilot]

Copilot review overview

🟡 Changes recommended

Evaluation metrics and preprocessing do not currently match the stated reproducibility requirements, with additional cache and dependency-pinning issues.

Review effort: Lite
Findings: 1 · 1

Open (2)

• AP interpolation inflates reported COCO-style metrics · New
• Unpinned export dependencies undermine reproducibility · New

What changed in this PR

Adds reproducible export and evaluation tooling for Project516’s YOLO26n fuel model alongside Team 2826’s model.

Changes:

• Adds pinned model download, verification, and ONNX export validation.
• Adds colour/gray evaluation tooling.
• Updates documentation with results and usage guidance.

File	Summary
tools/​fuelmodel/​requirements.txt	Defines export and evaluation dependencies; exporter dependencies are not fully pinned.
tools/​fuelmodel/​README.md	Documents setup and usage.
tools/​fuelmodel/​export.py	Downloads, verifies, and exports the model.
tools/​fuelmodel/​eval.py	Evaluates both models; AP calculation, preprocessing padding, and cached split handling require changes.
README.md	Updates model attribution.
docs/​GAME-PIECE-MODELS.md	Records results and recommendations.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Project516]

@allengregoryiv ready for a review when you have a moment. Rebased onto main at ab6e086 today with no conflicts, and lint passes.