Repository navigation
Port the translators to the moveapps R package - #14
Merged
Merged
Conversation
The translators are pinned to the old R co-pilot: their R side sources the SDK's src/ files, which the moveapps package era removed, so a build on the current co-pilot fails. The spec also covers what scoping turned up: an untested R side, an App Dockerfile nothing but the platform ever builds, and a conda that differs from every python App. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Running every input through the production images replaced three suspected defects with five confirmed ones: an empty collection fails the App, DST ambiguity fails it, sub-second times are truncated, single-fix tracks vanish, and logical columns come back as text. The suspected tz-aware index is dead code, since movingpandas drops the zone itself. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
No App version on the platform has ever pulled a base from outside the couchbits GitLab registry; kaniko holds no Docker Hub credentials and no mirror. The python co-pilot carries the same /opt/conda, untouched, in the registry the platform already reads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fourteen tasks: pin v2.2.0 from the production images, refactor and port against those pins, add the CI job, fix the five measured defects one commit each. Push, pull request and App versions wait for an explicit go. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The eight move2_loc inputs came from an older cut of the Template's data: same observations, but more event columns and another track id column than Movebank delivers today. Taken byte for byte from Template_R_Function_App e00af0d. The List file stays, it is the only input with list columns. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The v2.2.0 production images, pulled from the platform registry, produced every golden and every expectation here - so what later steps must reproduce is literally what runs in production. The scripts that did it are committed alongside and are reused to prove the refactoring, the port and each fix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The goldens from v2.2.0 now bind python: R's CSV pair goes in, and what python hands back must match production byte for byte. Adds the first tests of the meta.csv and link.csv python writes, and of reading meta.csv and building the collection. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both entry scripts were a single top-level tryCatch, so nothing on the R side could be tested. link.R holds the conversion as functions, the entry scripts keep only the SDK glue. Running this checkout's R files inside the v2.2.0 production images reproduces every golden and every round-trip expectation, so the move changes nothing. Includes the first R tests: unit tests per function and the contract goldens. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Since R co-pilot v4 the SDK is the moveapps package and the image has
no src/, so the old build failed at mv ../src. The entry scripts now
call the package; its readRdsInput needs no move1, so the patched copy
goes too. The lock moves to R 4.6.1.
conda comes from the python co-pilot's image instead of a Miniforge
installer fetched from "latest", so the python half resolves like every
python App. One Dockerfile builds both Apps: DIRECTION has no default,
so a forgotten edit fails the build instead of building the other App.
The start script is picked in RUN: BuildKit resolved a COPY of
${DIRECTION}.sh before the check ran and failed on "/.sh: not found".
Built on the real co-pilots, both Apps reproduce every v2.2.0 golden
and round-trip expectation.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The App Dockerfile was built by nothing but the platform, which is how the mv ../src break stayed invisible. CI now builds the real Dockerfile for both directions, on a public stand-in for the private co-pilot, and runs the R unit tests, the round trip and a coherence check of the version pins inside the images. A build without a valid DIRECTION must fail, at the check: the grep matches the message the check prints, not the command the log quotes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The README promises a NULL object, but the python half failed with "No objects to concatenate", and the R half would not have stored an output for zero rows either. Python now writes empty files, and R stores the NULL result: the empty file the next R App rejects with code 10. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
R wrote wall-clock times and python re-localised them, so two fixes an hour apart in the repeated autumn hour became the same local time and the App failed with AmbiguousTimeError. link.csv now carries UTC; meta.csv still names the zone and python converts. UTC data - all real Movebank data - stay byte-identical. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Python writes fractions of a second, R parsed them with %S and dropped them: 75 fixes of the stork sample came back up to 0.999 s early. The 1 ms left is R's own %OS3, which truncates on the way to python: 10 of the 75 are stored just below their millisecond. The time assertions of the R tests compare absolute differences now: expect_equal's tolerance is relative, and at 1.4e9 seconds it forgave 20 s - the old pin could not tell truncated times from exact ones. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A track with a single fix cannot become a trajectory, and movingpandas drops it without a word: the List sample goes in with 4 tracks and comes back with 1. The format cannot keep them, so the App log now names every dropped track and its number of fixes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Python writes booleans as True/False, which read.csv leaves as text, so every logical column came back as character: 91 in the samples, and 8 list columns of the List sample that R flattens to logical on the way out. The other class changes of the CSV bridge stay as they are and stay pinned. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
%OS3 truncates, so a time stored just below its millisecond lost it on the way to python: 10 of the 75 sub-second fixes of the stork sample came back 1 ms early, in v2.2.0 as well. Half a millisecond added before formatting makes the truncation round. No golden changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bake overrides a variable from the environment variable of the same name, and GitHub's runners export CONDA=/usr/share/miniconda: the App job passed that path as the conda image and failed on "invalid reference format". The variable is CONDA_IMAGE now; the Dockerfile's ARG CONDA stays. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first cold run took 8m36s without any cache. Twice that, rounded up, cuts off a hanging build within 20 minutes instead of 90. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The R half was locked, the python half resolved anew at every build: the platform built an App version from whatever conda-forge offered that day, never the environment CI had tested. The App and the python tests now install python/conda-linux-64.lock, an explicit package list python/lock.sh resolves from environment.yml - plain conda, no extra tool. environment.yml stays deliberately open; the lock is refreshed before a release. The weekly run tests environment.yml resolved fresh, as before, and warns when the lock is behind it. test/coherence.R fails while the lock was not made from the current environment.yml. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings v2.2.1 (#15, Trello #1419) onto v3: v3.0.0-RC1 locks movingpandas 0.23.0 and wrote the same pickles without locations. transform_to_pickle.py merged cleanly; the dropped-track warning still reads `data`, which keeps `coords_x`/`coords_y`. The test conflict was two additions side by side. `pytz` stays out - v3 removed it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
annescharf
approved these changes
Oct 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Trello 1402 — port both translator Apps to the current R co-pilot, which ships the SDK as the
moveappsR package.Design and measurements:
docs/superpowers/specs/2026-09-30-port-to-moveapps-package-design.md; task plan next to it.Why
Since R co-pilot v4 the SDK is a package and the image has no
src/, so the App Dockerfile broke atmv ../src.Nothing but the platform ever built that Dockerfile, which is how the break stayed invisible.
What
rds_2_csv.R/csv_2_rds.Rcallmoveapps(still no move1). The conversion moved, unchanged, intotestable functions in
r/link.R.r/src/is gone.r/renv.lockis re-snapshotted for R 4.6.1 / renv 1.2.4 /moveapps1.0.3 — seven direct packages, notestthat.FROMthe current R co-pilot; conda is copied from the python co-pilot's image(
/opt/condachecksummed identical tocondaforge/miniforge3:26.7.2-0) instead of a Miniforge installer fetchedfrom "latest".
ARG DIRECTIONhas no default: a forgotten edit fails the build instead of building the other App.python/conda-linux-64.lock(explicit, 161 packages, plainconda) instead of resolving
environment.ymlat build time, so an App version gets exactly what CI tested.python/lock.shresolves it fromenvironment.yml, which stays open; refresh it before a release. The weekly runtests a fresh resolution and warns when the lock is behind; CI fails while the lock was not made from the current
environment.yml.app: builds the real Dockerfile for both directions on a public stand-in for the private co-pilot(
docker buildx bake), runs coherence checks of the version pins, the R tests and a round tripR → python → R of 12 cases inside the images, and requires a build without a valid
DIRECTIONto fail at the check.proves itself by reproducing them.
promises.
(
AmbiguousTimeError).link.csvnow carries UTC instants; UTC data — all Movebank data — stay byte-identical.%Sand dropped fractions: up to 0.999 s off. Now%OS.%OS3truncates: a time stored just below its millisecond lost it on the way to python (10 of 75sub-second fixes in the stork sample, 1 ms early). Rounded now; no golden changes.
True/Falsecame back as character; logical columns now come back as logical.Not obvious from the diff
RUN, not byCOPY ${DIRECTION}.sh. BuildKit resolves a COPY source before thepreceding
RUNcheck executes, so a build withoutDIRECTIONfailed on"/.sh": not found— and a grep for thecheck's message still matched, because the log quotes the command. The CI grep is anchored to the message at the
line end; both positive builds give 0 matches, both negative ones 2.
~/.cache/condais created as root. The co-pilot'sinstall -d … $HOME/.cache/Rleaves~/.cacheowned byroot, and
conda env createfails on its notices cache. The stand-in reproduces the co-pilot, so CI sees the same.CRAN. The snapshot was taken with P3M binaries (minutes instead of hours, emulated);renv then recorded the P3M URL per package. Normalised to
CRAN; the co-pilot's repo override decides at restore.expect_equal's tolerance is relative: at 1.4e9 s itsdefault forgives ~20 s, so a pin of "v2.2.0 truncates sub-seconds" passed with and without the truncation.
Proofs (local, Apple Silicon, amd64 emulated)
(no diff) — CI and production agree.
output.rds(11 cases) is read by the v2.2.0 R → python App, exit 0 — R Apps that were notrebuilt keep working.
DIRECTION: missing and invalid are refused on the real co-pilot, with the message.environment.ymledit without anew lock each fail the bake suite; a stale lock shows up in the drift report.
After merge
Tag
v3.0.0- a major release: downstream Apps see logical columns as logical instead of character, timeswith their fractions of a second, and NULL instead of a failure for an empty TrajectoryCollection. The python lock
is from 2026-10-01; refresh it first if the weekly run reports it behind. Then one App version per translator. Its Dockerfile is this
Dockerfilewith exactly one line changed:ARG DIRECTION→ARG DIRECTION=r2pythonARG DIRECTION→ARG DIRECTION=python2rThe
appjob's first cold run took 8m36s; itstimeout-minutesis 20.