Skip to content

Labelling page: train the learned check from the labels - #63

Merged
pzhu-flexiv merged 5 commits into
mainfrom
task/check_training_ui
Oct 23, 2026
Merged

pzhu-flexiv merged 5 commits into
mainfrom
task/check_training_ui

Conversation

@pzhu-flexiv

@pzhu-flexiv pzhu-flexiv commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Labels are given on the Labelling Frames page (#60), but the learned check could only be trained from the command line. This adds Train this check to the same page, so step 6 of Stage 1 (label, then train the "latched" check) needs no terminal.

What changes

  • Panel: under the label bar:
    • Inputs: cameras (from the loaded datasets, each with an optional crop x0,y0,x1,y1), state columns, and the folder to save to (default: the label's name).
    • Train runs on the labels of every loaded dataset.
    • While it runs: elapsed time and the last log line (epoch 12/30: loss …), with Stop.
    • When it ends: the held-out report, false passes first, then each held-out episode's result (false passes and misses highlighted), and how a plan names the check (checks/<name>). A failure shows the command's own error: line.
    • It refuses a bad crop before sending anything.
  • Job: it runs the same command the README gives (python -m flexivtrainer.evaluation.learned_check train …) in its own process: one at a time, so the server stays responsive and the GPU memory is freed when it ends.
    • POST /checks/train starts it. Datasets are held to the storage root (403/404). A bad request gets 400, and so does a name already taken. One already running gets 409.
    • GET /checks/train gives its state, log tail and result. POST /checks/train/stop stops it.
    • GET /checks lists the trained checks under <storage>/checks/ with what they look at and their held-out reports.
  • Label view: /datasets/check-labels also returns each dataset's cameras (depth maps left out) and state columns, for the form.
  • README: the learned-check section explains the panel.

Tests

  • Backend: tests/test_check_training.py covers:

    • the exact command built from a request, and 6 refusals;
    • a real training run in its own process (pixels encoder, 3 epochs, on a labelled LeRobot dataset), its held-out report, and GET /checks listing it;
    • a failing command reporting its error: line;
    • a running job refusing a second one, then stopping;
    • routes keeping datasets inside the storage root.
  • Panel logic: tests/label_frames_ui.test.cjs gains 3 tests:

    • the camera and state choices;
    • the request it sends;
    • Train enabled only with an input chosen;
    • a bad crop refused before sending;
    • the running, done and failed views.

    Each of 5 mutations is caught.

  • By hand: headless Firefox against a test server on a scratch copy of a recorded dataset (4 episodes labelled through the API). I chose wrist_1 with a crop and Fz, then pressed Train. It trained ResNet-18 on the CPU in its own process, then showed the held-out report and saved checks/seated_ui. The report itself means nothing, as the labels were arbitrary. No page errors.

Note for merging

This PR and #61/#62 each bump the static asset version (?v=) in index.html and three tests. Whichever merges second has that one conflict; I'll resolve it.

Review fixes (second commit)

An independent review reproduced two real races in the job. Both are fixed and tested.

  • Start while stopping: a new job could start while the last was stopping. The old job's thread then took over the new one: it marked it failed and left its process untracked, so a second GPU training could start. Now starting is refused while a job runs or stops, and each process's threads only touch their own job.
  • Stop racing the end: a stop that raced the end reported "stopped" for a check that had been saved. Now:
    • an exit with the result counts as done;
    • a check saved before the stop is kept, and the status says so (saved);
    • a check folder left half-written by a stop or a failure mid-save is removed, so it no longer blocks its name;
    • stop no longer signals a process that has already ended.
  • Page:
    • The Stop button was rebuilt every second and could lose a click; only the progress line changes now.
    • A job found running when the page opens now locks the form at once.
  • Smaller:
    • crops keep every digit on the way to the command;
    • names must match whole (a trailing newline was accepted);
    • GET /checks reads the folder training writes to;
    • the server stops a training when it stops;
    • the child process speaks UTF-8 on Windows;
    • only video cameras and listed state names are offered;
    • escapeHtml escapes quotes. This is the same change as Chains: judge each run, and count clean runs in a row #62, in identical text, so the two merge cleanly.

Labels are given on the Labelling Frames page (#60), but the check was
trained from the command line only.

- The page gains Train this check: cameras (each with an optional crop),
  state columns and a folder name. It trains on the labels of every
  loaded dataset, shows its progress, and then the held-out report,
  false passes first, per held-out episode.
- The job runs the same command the README gives, in its own process
  (one at a time): GET/POST /checks/train, POST /checks/train/stop.
  GET /checks lists the trained checks with their reports.
- The label view also gives each dataset's cameras and state columns.
Comment thread tests/test_check_training.py Fixed
- A new job could start while the last was stopping, and the old job's
  thread then took over the new one (marked it failed, left its process
  untracked). Starting is refused while a job runs or stops, and a
  process's threads only touch their own job.
- A stop that raced the end reported "stopped" for a check that had been
  saved: an exit with the result is done; a check saved before a stop is
  kept and said so; a check folder left half-written (stopped or failed
  mid-save) is removed, so it no longer blocks its name. Stop does not
  signal a process that has already ended.
- The page's Stop button was rebuilt every second and could lose a
  click; only the progress line changes now. A job found running on
  opening the page now locks the form at once.
- Crops keep every digit on the way to the command; names must match
  whole (a trailing newline was accepted); GET /checks reads the folder
  training writes to; the server stops a training when it stops; the
  child speaks UTF-8 on Windows; only video cameras and listed state
  names are offered; escapeHtml escapes quotes (the same fix as #62).
# Conflicts:
#	src/flexivtrainer/web/index.html
#	tests/test_api_app.py
#	tests/test_gripper_ui_contract.py
#	tests/test_rollout_evaluation_ui_contract.py
# Conflicts:
#	src/flexivtrainer/web/index.html
#	tests/test_api_app.py
#	tests/test_gripper_ui_contract.py
#	tests/test_rollout_evaluation_ui_contract.py
@pzhu-flexiv pzhu-flexiv self-assigned this Oct 23, 2026
@pzhu-flexiv pzhu-flexiv added the enhancement New feature or request label Oct 23, 2026
@pzhu-flexiv
pzhu-flexiv merged commit c7c5105 into main Oct 23, 2026
10 checks passed
@pzhu-flexiv
pzhu-flexiv deleted the task/check_training_ui branch October 23, 2026 20:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant