Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 3 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ The rest of this README explains what each step does and why.

We turned an NVIDIA Jetson Orin Nano Super into a vision coprocessor that finds AprilTags on its GPU, for team 8515's robot at the October 2026 off-season event.

The robot controller is a SystemCore running 2027 alpha-6 robot code. The Jetson runs PhotonVision, the same software many FRC teams use on an Orange Pi. Ours is a special version that sends the AprilTag math to the GPU using a detector written by FRC team 971. The robot code talks to it through PhotonLib over NetworkTables, like any other PhotonVision camera. The current compatibility limit and alpha-7 migration order are in [docs/WPILIB-2027-ALPHA-7.md](docs/WPILIB-2027-ALPHA-7.md).
The robot controller is a SystemCore running 2027 alpha-6 robot code. The Jetson runs PhotonVision, the same software many FRC teams use on an Orange Pi. Ours is a special version that sends the AprilTag math to the GPU using a detector written by FRC team 971. The robot code talks to it through PhotonLib over NetworkTables, like any other PhotonVision camera.

Everything we did is scripted in this repo, so another Jetson can be set up the same way. These notes explain what we did and why, including the mistakes, so you can understand the system and not just copy commands.

Expand Down Expand Up @@ -76,15 +76,13 @@ flowchart LR
| Game pieces | YOLO models on the GPU through TensorRT 10.3 (our backend), FUEL model by Team 2826 |
| Robot side | Stock PhotonLib v2027.0.0-alpha-2 in `2026-FM-SystemCore`, team 8515 |

**Why the current 2026 PhotonVision fork works with the current 2027 robot code:** the CUDA version of PhotonVision only exists for 2026. We checked the source to confirm the current pair speaks the same language:
**Why a 2026 PhotonVision works with 2027 robot code:** the CUDA version of PhotonVision only exists for 2026. We checked the source to confirm the two versions speak the same language:

- the messages have the same format (PhotonLib compares a hash of the message layout, and it matches exactly),
- the NetworkTables protocol is the same,
- time sync is the same.

These checks apply to the current WPILib alpha-6 and PhotonLib alpha-2 pair. They do not clear the alpha-7 migration.

**Current compatibility limit.** Keep the robot on WPILib 2027.0.0-alpha-6 and PhotonLib v2027.0.0-alpha-2 until the separate CUDA port and robot-network test pass. WPILib 2027.0.0-alpha-7 is the migration target. As of 2026-09-25, no PhotonLib alpha-7 tag was found, and the current alpha-2 client is built against WPILib alpha-6. See [docs/WPILIB-2027-ALPHA-7.md](docs/WPILIB-2027-ALPHA-7.md).
The one rule: **don't upgrade the robot's PhotonLib past alpha-6.** Newer versions changed the message format, and the robot code would crash when it reads from this Jetson.

## Step 1: Flash JetPack onto the SSD

Expand Down
34 changes: 10 additions & 24 deletions docs/TECHNICAL.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,19 +105,10 @@ Settings shared by the scripts live in [config.env](../config.env).
(WPILib 2027.0.0-alpha-6 on a **SystemCore**, vendordep `photonlib v2027.0.0-alpha-2`).
The field is the **2026 Rebuilt AndyMark** layout, on the Jetson and in robot code.

**Current compatibility limit.** The deployed robot project uses WPILib
2027.0.0-alpha-6 and PhotonLib v2027.0.0-alpha-2. Keep those pins while the CUDA
fork and robot-network test remain on the 2026-era native bridge. WPILib
2027.0.0-alpha-7 is the migration target, but as of 2026-09-25 no PhotonLib
alpha-7 tag was found. The release notes also say alpha-6 and earlier vendordeps
do not work with alpha-7. The full order is in
[WPILIB-2027-ALPHA-7.md](WPILIB-2027-ALPHA-7.md).

The current Jetson build runs the `FRC-Team-4143/photonvision` fork
(`d8c9e8e`, WPILib 2026.2.1) with 971's CUDA detector. The robot uses stock
PhotonLib alpha-2. The source comparisons below apply only to this current
2026-era fork, WPILib alpha-6 robot project, and PhotonLib alpha-2 client. They
do not clear the alpha-7 migration:
**Decision:** the Jetson runs the **unmodified 2026** `FRC-Team-4143/photonvision` fork
(`d8c9e8e`, WPILib 2026.2.1) with 971's CUDA detector. The robot uses **stock**
photonlib alpha-2. This works because everything on the wire is identical between the
fork's base and the alpha-6 era (checked in source on 2026-09-23, not yet on hardware):

- **Serde hashes match** (`PhotonPipelineResult` = `4b2ff16a964b5e2bf04be0c1454d91c4`,
and all sub-messages). PhotonLib only throws on a hash mismatch; a different
Expand All @@ -126,17 +117,12 @@ do not clear the alpha-7 migration:
try `10.TE.AM.2` first, so SystemCore is `10.85.15.2`.
- **Time sync:** the same UDP 5810 packet layout and microsecond timebase.

> **Do not move the robot's PhotonLib to an alpha-7-era client while this Jetson
> still emits the pre-alpha-7 result format.** PhotonVision PR [#2566](https://github.com/PhotonVision/photonvision/pull/2566),
> merged as `a6167b049d7950b0d06d3e7a3ad0b14ad2d396db`, renamed the timestamp
> fields and changed the serde hashes. That commit is later source evidence, not
> a tagged PhotonLib alpha-7 release.

Porting CUDA to PhotonVision source that uses WPILib 2027.0.0-alpha-7 is a
separate PR. It needs a rebase of the PhotonVision patches, updated JNI headers,
and a real ARM64 Jetson build. Alpha-7 native code needs C++23 and G++ 14, while
JetPack 6 ships GCC 11, so the detector toolchain is part of that port. It must
not be combined with the robot-only pin change.
> **Do not upgrade the robot's photonlib past the alpha-6 era.** PhotonVision `main`
> after alpha-7 (`a6167b0`, 2026-09-17) renamed the timestamp fields, which changed the
> hashes, and the robot would throw against this Jetson.

Porting CUDA to 2027 PhotonVision was rejected. 2027 allwpilib needs JDK 25, C++23
and GCC 13 (Ubuntu 24.04), while JetPack 6 has GCC 11.

| Piece | Version | Built on | Script |
|---|---|---|---|
Expand Down
20 changes: 2 additions & 18 deletions docs/UPSTREAM-PORT.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ build script and bench-tested on the Jetson. For the setup itself, see the [READ

- **`photonvision-00-upstream-v2026.3.4.patch`:** the full diff from the 4143 fork (`d8c9e8e`) to
a merge of upstream **v2026.3.4**, 41 upstream commits. It still speaks the exact same
NetworkTables format: all six serde hashes are unchanged (table below), which matches
PhotonLib v2027.0.0-alpha-2. The robot-network integration test is still pending.
NetworkTables format: all six serde hashes are unchanged (table below), so PhotonLib
v2027.0.0-alpha-2 on the robot still works.
- **Our 01–10** are regenerated on top of it. Only 05 and 07 changed in content: upstream #2407
changed how the UI checks for dark mode.
- **`photonvision-11-set-enabled.patch`:** the server side of PhotonLib's
Expand Down Expand Up @@ -37,8 +37,6 @@ build script and bench-tested on the Jetson. For the setup itself, see the [READ
| TargetCorner | 16f6ac0dedc8eaccb951f4895d9e18b6 | same |

`photon-serde/` and `photon-targeting/src/generated/` are byte-identical to d8c9e8e.
These are the pre-alpha-7 hashes for the current 2026 fork and alpha-2 client.
They do not describe the later PhotonVision source merged by PR #2566.

## Patch 00: upstream v2026.3.4

Expand Down Expand Up @@ -95,20 +93,6 @@ Plus two that upstream still has:
Skipped: the lifecycle refactor, the calibration rework (tangled with 2027 changes) and pipelines
we don't use.

## WPILib 2027 alpha-7 migration

The current fork and robot pins remain on the 2026-era configuration described
above. PhotonVision `v2027.0.0-alpha-2` is the current released tag and pins
WPILib 2027.0.0-alpha-6. The audited main source revision
[`1f419c9d`](https://github.com/PhotonVision/photonvision/commit/1f419c9de3a6a0787571648d18211537a83fdca9)
pins 2027.0.0-alpha-7, but it is source material, not a release. PhotonVision PR
[#2566](https://github.com/PhotonVision/photonvision/pull/2566) also changes the
timestamp field names and serde hashes. The CUDA port is a separate PR, and it
needs a real Jetson and robot-network test.

See [WPILIB-2027-ALPHA-7.md](WPILIB-2027-ALPHA-7.md) for the migration order and
compatibility limits.

## Patch 21: Thriftiest Cam support (#2478)

Upstream [#2478](https://github.com/PhotonVision/photonvision/pull/2478) (merged 2026-09-13, after
Expand Down
39 changes: 0 additions & 39 deletions docs/WPILIB-2027-ALPHA-7.md

This file was deleted.

8 changes: 4 additions & 4 deletions scripts/host/03-build-photonvision-fork.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@
# on the x86-64 host. No sudo: Node 22, pnpm 10 and Temurin 17 (pinned below) go in ~/build/tools,
# matching upstream CI for this commit.
#
# The current fork is a bench-tested 2026-era build. Its serde hashes and time-sync
# packets match the alpha-2 robot vendordep. The alpha-7 CUDA port must rebase
# the fork and requires Jetson and robot-network testing. See
# docs/WPILIB-2027-ALPHA-7.md.
# Why a 2026 build against 2027 alpha-6 robot code: the serde message hashes
# (PhotonPipelineResult 4b2ff16a...), NT4 protocol and time-sync packets are identical,
# so stock photonlib v2027.0.0-alpha-2 on the robot accepts it. Robot photonlib must NOT
# move past the alpha-6 era (main after alpha-7 changed the hashes).
set -euo pipefail

FORK_URL=https://github.com/FRC-Team-4143/photonvision.git
Expand Down
7 changes: 2 additions & 5 deletions scripts/jetson/03-photonvision.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,8 @@
# Install upstream PhotonVision (CPU AprilTag detection) as a boot-time systemd service.
# Run ON THE JETSON. Needs internet.
#
# Current released installer pin. This CPU-only install provides the boot-time
# service before the custom CUDA fork jar replaces it. The robot uses WPILib
# 2027.0.0-alpha-6 and PhotonLib v2027.0.0-alpha-2. The alpha-7 migration target
# is documented in docs/WPILIB-2027-ALPHA-7.md. Do not point this script at a
# development asset.
# The version must match the photonlib vendordep in the robot code
# (Spectrum3847/2026-FM-SystemCore: photonlib-v2027.0.0-alpha-2.json).
# Usage: scripts/jetson/03-photonvision.sh [version]
set -euo pipefail
# Writes on the read-only system (ro-root on) land in RAM and vanish at the next boot.
Expand Down
6 changes: 3 additions & 3 deletions scripts/jetson/04-build-allwpilib.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@
# Run ON THE JETSON, ideally inside tmux (this takes hours):
# tmux new -s build ~/SpectrumJetson/scripts/jetson/04-build-allwpilib.sh
#
# The tag must match the WPILib version of the PhotonVision build that loads
# lib971apriltag.so. The current FRC-Team-4143 fork uses 2026.2.1. The
# 2027 alpha-7 CUDA port is a separate change and needs a real Jetson build.
# The tag must match the wpilibVersion of the PhotonVision build that loads
# lib971apriltag.so. The FRC-Team-4143/photonvision fork (d8c9e8e) uses 2026.2.1.
# allwpilib main does NOT work: wpi/jni_util.h has moved there.
# Usage: 04-build-allwpilib.sh [tag]
set -euo pipefail
# Writes on the read-only system (ro-root on) land in RAM and vanish at the next boot.
Expand Down
Loading