Conversation
Scene drew its own map: a three.js page that pulled the library from a CDN and needed WebGL, plus a canvas floor plan beside it. On a headless robot neither is available, so the 3D view rendered nothing there, and keeping both working was viewer maintenance that has nothing to do with a scene graph. The map pages are now rerun views. Scene logs what it knows and rerun draws it. What is logged is the annotated map, not raw sensor data: - one point cloud per object, coloured by instance, carrying its class label - the occupancy grid as a textured floor, one colour per cell meaning - one edge per relation, labelled in 3D and unlabelled from above - the robot's own footprint polygon and a heading arrow The 3D page is the landing page, because whether the perception is any good is the question this UI is opened to answer. The 2D page is the same annotations seen from above, over the grid itself: read that way, an object either sits in the room it belongs to or it does not, and a relation either crosses a wall or it does not. The box each object reports is kept but hidden. It is the perception layer's own extent estimate, it is often wrong, and a wireframe around every object buries the points that are the actual evidence. It is switched on from the entity tree when a detection needs checking. Every page now shares a sidebar. Before this the pages had no links between them and reaching the annotation view from the map meant editing the address bar; nothing on any page said the other views existed. Native installs keep exactly the pages they had. `web_viewer` (config) and SCENE_WEB_VIEWER select `auto`, `rerun` or `builtin`: `auto` uses rerun when it is importable, which is true in the x86 docker image and false natively. The built-in pages stay reachable at `?bare=1` either way, and setting `rerun` where it is not installed fails the start rather than serving an empty frame. The sink only reads. It takes a registry snapshot and publishes it; object persistence, annotations and the map lifecycle are untouched. Also adds testing/export_scene_pointcloud.py, which writes the same objects to a coloured PLY and an .rrd for offline comparison.
github-actions
Bot
added
type:feature
…dable Three random bytes per object spanned the whole RGB cube, which put neon magenta next to muddy brown and made the map read as noise rather than as a legend. Only the hue now comes from the object id; saturation is fixed and lightness takes one of two bands. Hue alone collides — thirty objects on a 360-degree wheel put some pair within a couple of degrees more often than not, and two near-identical clouds read as one object seen twice — so the second axis makes such a pair a light and a dark version of one hue. Forty objects come out as forty distinct colours. The map pages are light. A map is read next to a physical robot and a printed floor plan, and a dark page makes the occupancy grid look like a lit surface instead of a map: observed floor is now white, walls are dark slate, unobserved ground sits between them, and the view background is just off-white so the grid's extent is still visible against it. The viewer's own chrome stays dark and there is nothing to be done about it here: rerun 0.37's web backend ignores SetTheme, which the console reports as an unhandled viewport command. With the panels collapsed that leaves a thin strip at the top and the timeline at the bottom. Also adds a screenshot of the result, taken from a live Webots office session: eighteen objects with their own clouds and labels, the relation edges between them, and the occupancy grid drawn underneath as the floor.
The edges were drawn in saturated blue at the same weight as everything else, so a room with thirty relations read as a cat's cradle laid over the map and the point clouds — which are the evidence a reader is there to judge — competed with it. Relations are context. They are now thin, desaturated and translucent, and their predicate labels are off: the chips crowded the object labels they were drawn between, and the predicate is one click away in the entity tree when it is wanted. The screenshot is retaken from the resulting view and cropped to the map itself.
The viewer published a full copy of the map on every tick: the occupancy grid as a room-scale texture, every object's point cloud twice, every box, every edge, whether or not anything had moved. The browser keeps what it is sent, so a stationary robot still walked the viewer into its memory limit, and the page stopped responding once it got there. A 1 Hz feed of a 600x600 grid with twelve objects measured 0.27 MB per tick, about a gigabyte an hour. Each entity now carries a content digest and is logged only when that digest changes, which takes the same feed to 7 kB per tick. The occupancy grid is logged static in both history modes: SLAM rewrites it continuously and every superseded version was weight the browser then carried for the rest of the session. SCENE_RERUN_HISTORY=latest logs everything static for a forwarded or weak client, which bounds the store at one value per entity and gives up the timeline replay. SCENE_RERUN_PERIOD_S sets the publish period. The map is also drawn for a dark ground rather than a light one. The viewer's chrome is dark and cannot be changed, so a light view left the page half lit, and rerun draws a label in the colour of the entity that carries it: the muted colours that read well as point clouds were exactly the colours that made the text unreadable. Object colours are lifted above the background, the grid's brightness order is inverted so unknown ground recedes and walls stand out, and a box that is the only thing labelling an object is drawn solid enough for its label to read. The viewer link now pins theme=dark.
The map viewer is embedded in an iframe, but the iframe pointed straight at the ports rerun opened for itself: the viewer application on one, and each page's log stream on another. On the robot that is invisible. Off it, reading the map meant forwarding four ports, and forgetting one produced a blank frame with no error at all. Scene now proxies all of them under the port an operator has already reached it on, so one forward is enough. The stream is gRPC-web over HTTP/1.1, with status and trailers inside the body, so an ordinary streaming proxy carries it. Three things about rerun shape the routing and each of them fails silently when got wrong: the endpoint path in the source URL must be exactly /proxy, the requests themselves land on the gRPC service path at the origin root, and the viewer rewrites its own query once it has parsed it, so which of the two feeds a page wants travels in the path rather than the query. The proxy also does not trust proxy environment variables: the upstream is loopback, and HTTP_PROXY is set on both the robot image and a typical development machine. Starting the viewer no longer wedges the service. serve_grpc blocks forever on a port that is still held rather than failing, which is what a quick restart leaves behind, and the web UI then never came up at all: no map, no camera page, and nothing in the log naming the step that stopped. Bring-up runs on its own thread with a deadline and falls back to the built-in pages with the ports named. The shell is dark, on the annotation page's palette. It frames pages that are themselves dark, and a light sidebar put a seam down the middle of the window.
…service Three things the Webots run turned up, all of which look identical from the outside -- a map page that is simply empty. The object and relation list is back. It lived on the combined layout, which stopped being the landing page when the viewer became rerun's, and rerun draws the map but knows nothing about the registry behind it. The panel is shared now and sits over both map pages, fed by /api/state. Scene could freeze on startup with nothing in the log: no map, no camera page, no web UI at all. serve_grpc blocks rather than failing when its port is taken, and it blocks holding the interpreter lock, so the service stops entirely and a watchdog thread never gets scheduled to notice. The port check that was supposed to prevent the call reported a held port as free: it bound 0.0.0.0 with SO_REUSEADDR, which is the option that exists to allow precisely that. It now asks the direct question -- is anybody listening -- and asks again immediately before each bind, since importing rerun takes seconds and a previous run can finish exiting inside that window. An object flickered between its point cloud and a bare marker. Perception exports clouds intermittently, and an export that skipped an object was logged as an empty cloud, erasing geometry that was still valid. A cloud now survives a quiet export; only the registry dropping the object clears it. The viewer's 40 MB wasm bundle is served with no caching headers, so every visit downloaded it again -- three seconds over a forwarded connection, every time. It now carries an identity minted per process, which the browser revalidates in a round trip: 3.2s to 0.2s on a second visit, and never stale, since the bundle cannot change without this process restarting.
enkerewpo
marked this pull request as draft
September 13, 2026 07:19
rerun-sdk declares numpy>=2 and the perception stack requires numpy<2. uv's lockfile covers every workspace member's extras, so an extra that cannot co-resolve with its own package's dependencies makes the shared lock unsolvable, and that fails uv sync for members with nothing to do with Scene. A full rbnx boot died building memgraph, on a viewer for a map. rerun's bound is conservative rather than real. scene-viewer.txt already records 0.37.1 verified in this image against numpy 1.26.4, which is why the image installs it with --no-deps in the first place. Saying that once, as a workspace override, keeps the extra declared where the dependency belongs: uv sync --extra viewer now resolves and installs rerun against numpy 1.26. The alternative considered and rejected was deleting the extra. It unblocks the lock too, but by removing the declaration rather than the conflict, leaving the image's hand-pinned install as the only record that Scene has a viewer dependency at all.
# Conflicts: # uv.lock
One page called the same thing three names at once. The tab said "annotations", the heading said "Map & rooms", the button said "Annotate room", and the modal behind it was "Room" -- for a named polygon drawn on a map. The stale-map warning asked the reader to review "rooms", the page title announced "map & rooms", and none of it matched `list_regions`, which is what the same object is called over MCP. A region is a named polygon. Whether it is a room is what its name says -- a region named "room315" is a room, and a region named "loading bay" is not, and neither is a different kind of thing. Making "room" the type rather than the name is also what leaves the vocabulary stuck indoors, which the outdoor mapping work does not have the option of being. So the user-facing words are one word now, and the modal's element ids follow them rather than staying at room-modal-* underneath a dialog headed Region. Storage still stores `kind="room"` and MCP still exposes goal_room; those move next, together, because they cross a contract and a persisted file.
Every other test here reaches Scene through Python. The web tests build a page and assert on the HTML string, which catches a broken template and nothing at all of what the map pages actually get wrong: a viewer that mounts and then renders an empty canvas, a sidebar link pointing at a route that no longer exists, a WASM bundle that 404s, a page that still calls a region a room. None of those are visible without a browser, and this PR replaces both map pages with one that is mostly browser. So these run Chromium against a live Scene: the service, the simulator behind it and the models it loads. A viewer fed by a fake registry would only prove the template renders, which is the part already covered. Every test also asserts the console stayed quiet, because a viewer that fails to start logs and carries on showing an empty frame -- which looks exactly like a room with nothing in it. Where no Scene is listening the module skips. A machine without the stack up is not a broken build; equally, a test that cannot run must not be able to go green, which is why the skip is on reachability rather than on a try/except around the assertions.
The floating info panel is positioned at left:12px and the sidebar is 132px
wide, so the panel sat on top of the navigation with z-index 200 over it.
Every sidebar link underneath it was unclickable -- the browser delivered the
click to <h2>robot</h2> instead. It is draggable, so the page was navigable by
first moving the panel out of the way, which is not a thing to ask of someone
trying to reach the 2D map.
Moved to clear the sidebar rather than made narrower or given a lower z-index:
it is meant to float over the map, which is what the rest of the window is.
The browser tests found this on their first run, which is the argument for
having them. It is invisible to a test that asserts on rendered HTML, because
the markup was always correct -- the two elements simply overlapped.
The camera test was wrong rather than the camera page. Both map pages are a
sidebar shell around an iframe, so the panels live in the child document, and
page.inner_text("body") reads the shell and reports an empty camera page while
the camera is plainly streaming. It asserts inside the frame now, and waits
for a frame to land instead of for a fixed delay, which would have been a test
of the simulator's frame rate.
Suite is 12 green against a live stack: Webots, perception on the GPU, the
rerun viewer, all four pages and their bare forms.
The previous pass changed the markup and the script changed it back. The draw button's label is reassigned on every toggle, so it read "Mark region" until it was used once and "Annotate room" from then on, while the hint beside it told the reader to click a button by a name nothing on the page displayed -- an instruction that could not be followed rather than a word out of place. The row actions, the corner-count warning, the delete confirmation and the server's "already has saved room annotations" all said room too, so they say region now, along with the list row's style hook and the selector that reads it, renamed together so they cannot drift apart. Also two sentences that ran into each other in the map status line: "Enter a Map ID, then Save current.Map mode:". Found by screenshotting the pages rather than by testing them: the suite was green while the regions page was telling people to click a button that did not exist. Worth remembering that 12 passing browser tests did not catch a label, because nothing asserted on what the button says.
Scene called the same thing four names at once. The storage kind was "room",
the MCP contract was goal_room, the REST collection was /api/annotations, the
page was /user, and only list_regions used the word the UI now shows. A reader
had to know all four and which layer spoke which.
A region is a named polygon on the map. Whether it is a room is what its name
says: "room315" is a name, not a type. Making "room" the type is also what
pins the vocabulary indoors, which the outdoor mapping work cannot be.
BREAKING. The wire says region now:
robonix/system/scene/goal_room -> robonix/system/scene/goal_region
GoalRoom.srv / room_id -> GoalRegion.srv / region_id
GetRobotContext room_id/room_name-> region_id / region_name
Region.kind "room" -> "region"
GET|POST /api/annotations -> /api/regions
page /user -> /regions
Stored maps still load: a file written with kind "room" is normalised on read.
And references stay tolerant -- "room 315" still resolves, because people and
models say it. Both the stored name and the caller's reference are reduced the
same way, so a region named "region 315" asked for as "room 315" is the same
place either way round; matching only one side made the answer depend on which
word whoever drew it happened to use.
Renaming a contract turned out to cost more than the contract: every package
carries its own generated stubs, so all fifteen had to be regenerated before
anything would import. Worth knowing before the next one.
Two bugs fell out of doing this, neither caused by the rename:
* the rerun proxy's two-segment catch-all route was registered before the
API routes, so POST /api/regions was answered by the gRPC proxy with its
own 404 and no region could be created over REST at all. It is registered
last now.
* a resolver test inherited its annotation store from whichever test ran
before it, and the REST helper fed the query string to the router as part
of the path.
Suite: 276 passed. Browser suite: 12 passed against a live Webots stack.
…hem up by The map drew bare markers. The tick log said it plainly and nobody read it: "29 objects, 0 with points, 0 with colour", every tick, for objects the perception layer was exporting 256 points apiece for. The clouds were never missing. The 3D snapshot keys them by the perception layer's own uuid; the viewer draws an object under its registry id and asks for its cloud by that. Two key spaces that never meet, so every lookup missed and every object arrived stripped of the geometry this page exists to show. service.py already expected the entry to carry the registry id -- "the snapshot is keyed by the perception layer's own uuid and carries the registry id alongside it" -- and it did not. The mapping is the same _uuid_to_oid the exporter already reads to decide which objects are live, so it is put on the entry it was describing. before tick 120: 29 objects, 0 with points, 0 with colour after tick 60: 25 objects, 14 with points, 14 with colour Worth noting where this came from: scene-bench-eval already carries this fix. This branch is cut from dev and never had it, so the feature the branch exists to add has been broken on it from the start while working on the branch beside it.
The info panel floated, and an overlay is always over something: it had already been moved once because it swallowed the sidebar clicks underneath it, and the fix for that was a coordinate. So it docks. The dock is a flex sibling of <main> rather than a layer above it, which means the view resizes around it and there is no position left to get wrong. Blenders arrangement: a tab strip, one panel at a time, a drag handle on the inner edge, and collapse to a 34px rail that keeps the tabs -- so the strip is both the switch and the way back, and reopening does not need a separate button parked in a corner. Three tabs, because the page answers three questions: what the registry holds, how those things sit together, and where the robot thinks it is. They were one scrolling column before, which put the relation list below the fold exactly when the object list was long enough to be worth reading. Objects below 0.55 confidence are dimmed and flagged. Scene is not accurate enough to present every hit flatly, and a UI that does makes a second look seem like doubt about the whole list rather than the next step. The shell was monospace throughout. Monospace is for data whose columns line up -- ids, coordinates, stamps -- and it is wrong for navigation, headings and buttons, where it reads as a terminal. Two variables now: a system UI stack carrying PingFang SC and Microsoft YaHei, because the interface is going bilingual and a Latin-only stack leaves the browser to pick the Han face, and the monospace kept for the tables that earn it. The sidebar also gets an inline SVG per destination; four words in a column give the eye nothing to aim at. Seven tests cover it, the first geometric: the dock rectangle must not intersect the views or the sidebars. That is the property both moves of the old panel were about, and a test that only checked the tabs would have passed on the version that covered the nav. 19 passed against a live Webots stack.
The flow first, because the code is downstream of it. Every operation belongs
to one specific map, so which map is bound decides which operations exist:
1. Nothing bound. A live mapping session is always running, but an unnamed
session is not a map -- a region marked on it is lost with the session,
which is exactly how orphan annotations were produced. Only /maps can be
operated.
2. A map is bound, by saving the live session under a name or loading a
saved one. Now the objects belong to it, regions and POIs can be marked
on it, and its identity is on every page.
3. Switching maps returns to 2 with different contents. The previous map's
regions are not this one's, and drawing them here would misreport where
the robot is.
So /maps takes the map controls -- name and save, load, delete, pose estimate
-- which were crowded above the region drawing button, a layout that said
they were the same kind of thing. /regions keeps only marking, and with
nothing bound the control is absent rather than disabled, with a line saying
where to go: failing at save time is what made the orphans. Both pages are
one template in two modes; serving them from two copies is how one thing came
to have four names last time.
The 2D page stops being the 3D recording seen from above. It was rerun's
top-down view, which draws the point clouds, and from above a point cloud is
a smear over the floor plan it is drawn on. The built-in renderer draws the
occupancy grid as the floor and one dot per object, and nothing else.
Its labels no longer collide. Two objects a few centimetres apart -- a cup
and a monitor on one table -- printed their names over each other and neither
could be read. Placement is simulated annealing over eight candidate
positions per anchor, after Christensen, Marks and Shieber, "An empirical
study of algorithms for point-feature label placement" (ACM TOG 14(3), 1995),
which compared it against greedy, gradient descent and Hirsch's repulsion
method and found it best at equal time; it is also what d3-labeler
implements. Inline rather than vendored, because this runs on a robot with no
network and a library that cannot be fetched when it is needed is not a
dependency. A label that had to move away from its dot gets a leader line.
The solve is cached on the view and the dot positions: the map redraws five
times a second, and a label that crawls is harder to read than one that
overlaps.
Objects below 0.55 confidence are drawn as a hollow dashed ring with a marked
name, the same threshold the dock dims its rows at. The map is the view
someone navigates from, so it is where not overstating a guess matters most.
27 UI tests pass against a live Webots stack, including that no two label
boxes overlap by more than a tenth of a label's area.
It centred on the robot at a fixed 40px per metre, so a 3.3 x 5.2 m occupancy grid drew as a 132 x 206 px stamp in the middle of an empty field of grid lines -- the one thing the page is for, smallest on it. Fit to the occupancy extent with a 40px margin, capped at 120px/m; keep following the robot while there is no grid yet, which is the only view that shows anything during the first seconds of a session. Same arithmetic the regions page has used since it was written.
The gate was wrong. Marking regions and recognising objects work perfectly well on an unnamed live session; what such a session lacks is a name and persistence, not the ability to be worked on. The server already agreed: maps_save rebinds the annotation store with carry_current=True and snapshots the objects, so regions marked during the session are carried into the map when it is saved. Blocking the marking made the UI refuse something the backend supports, and then lose work it would not let you begin. So the controls stay. What replaces the block is the one fact a reader needs and cannot see from the map itself: this map is temporary, the marks live only as long as the session, and saving it under a name in /maps keeps them. That is a disclosure, and it belongs beside the work rather than in front of it. The binding pill follows: there is always a map, so "no map bound" was the wrong reading. It now says either the map's name and mode, or that this one is temporary and unsaved. The test that passed on the blocking behaviour said the wrong thing, so it is rewritten to assert what should hold instead: marking is offered, and when the map is temporary the page says so and says where to save it. 27 UI tests pass against a live Webots stack.
Bilingual meant i18n, and what was there was both languages printed at once. That is not a bilingual interface; it is one interface with everything said twice, and it gets worse with every string added. So: a keyed string table, one language on screen, chosen once and remembered. Keys rather than English source text, because matching on the English means a typo fix silently drops the translation and the table cannot be audited for gaps. The switch sits at the foot of the sidebar, away from the destinations -- it changes how the interface reads, not where you are in it. First visit follows navigator.language rather than assuming English. Two documents have to agree. The shell is a page and each view is an iframe with its own document, so the choice lives in localStorage -- same origin, so both read the same value -- and the shell posts it into the frame on change and on frame load, so the view follows immediately instead of on the next navigation. The robot marker is redrawn. It was a dark disc inside a 4px white ring with a 24px orange dart through it: a nose twice the length of the body, with a concave tail that made the whole thing read as an eye. Three fills, three strokes and two colours competing at the one place on the map a reader looks first. Now a disc for where it is and a translucent cone for where it faces, with a rim tick so heading survives on pale floor where the cone washes out. It is also the same blue the 2D page uses. One robot, one colour. Still English: the map form's own controls and status line, which live deep in the view template and are the next pass. 31 UI tests against a live Webots stack, four of them new: the switch changes the sidebar, the choice survives navigation, the view follows the shell, and no block of text is printed in both languages at once.
…k that hid them The viewer deadlocked the service. `RecordingStream.serve_grpc` is documented to return immediately and does in isolation; inside scene, with the asyncio loop, the rclpy executor and a CUDA perception tick contending, it stopped returning while holding the interpreter lock, so the 20s `join` meant to bound it could never be scheduled. Scene hung before binding its web port and the boot killed it at the driver timeout -- which read as "scene is slow to start" for a day. Ports, blueprint and SDK version were each ruled out by measurement. The servers now run as `rerun` CLI children: a child that wedges can be waited on with a real timeout and killed, and what stays in-process is `connect_grpc`, a client call against a port already proven to accept. A log view, reading scribe rather than collecting a second copy of the same records: rbnx already sets SCRIBE_LOG_DIR and pipes each package into `<tag>.log` there, so start.sh mounts it and the page tails it with a byte offset per file. Every service's log, interleaved, because a perception tick that went quiet is usually explained by something mapping said. The level chips are the filter and the count at once. Object corrections reach the panel. One entry point per correction -- `apply_label_correction`, `remove_object` -- and MCP, the web API and gRPC are adapters over them; the mechanism underneath is called from exactly one place each, so the protocols cannot drift on what a rename does. Persistence defaults to "if this map has a snapshot": a temporary session has none, and being unsaved is not a reason to refuse an edit on it. The panel itself: the header loses a unix timestamp that rewrote itself twice a second, rows lead with the name, and a row opens its properties below the list rather than instead of it. Rename edits in place and delete asks in the panel -- `prompt()` and `confirm()` are the browser's dialogs, in another typeface and palette, and they undid the point of one type scale. The poll no longer redraws a pane being typed in, and a selection that perception has re-registered says so instead of offering to delete a ghost. Also fixed, each its own small wrongness: Enter in the region-name dialog cancelled, because `method="dialog"` submits with the form's first submit button and Cancel was first; a link inside the framed page navigated the frame, so following it rendered a whole second shell inside the first; the log list rebuilt every row on every poll, which at two thousand rows is a DOM rebuild against a live stream; and a map could not be called 客厅 -- the id was sanitised with an ASCII allow-list, which replaced it character by character into `__`, colliding with every other two-character name. 45 UI tests against a live Webots stack.
61% of web.py was asset text: 156 KB of markup, stylesheets and scripts in ten triple-quoted constants, with the routing and the data shaping threaded between them. Editing a stylesheet meant scrolling through a Python file, no editor offered highlighting or a linter for any of it, and every patch had to match exact indentation inside a string literal -- which is how one of them silently wrote a literal backslash-n into the page. The module is 2216 lines now, down from 5771, and what is left is routing and data. This is a move and nothing else. The values were read from the imported module rather than scraped from the source, so a constant written as a non-raw string handed over what Python actually built rather than what the source spelled; each file's SHA-256 is recorded in web_assets/MANIFEST.json and checked back against the constant it came from. The constants matching is necessary and not sufficient -- they are assembled into pages, and an assembly mistake would pass a hash check on every input -- so the six pages both the old and new modules build were also compared byte for byte. They are identical. Read once at import: the files do not change while the service runs, and a per-request read would put a filesystem call in the path of every page. 45 UI tests still pass against a live Webots stack.
…d controls
Maps stops being the regions editor with its controls hidden by CSS. That
shortcut is why choosing a map looked like editing one and why two thirds of
the window showed a map you had not chosen yet. It is a library now: a grid of
cards, each with the stored occupancy thumbnail, and one action bar that adds
to it. A Details button opens a card in place -- not the card itself, because
a map id is there to be read and copied, and making the whole surface a
control takes that away.
The thumbnail is the grid's own occupancy image with the map's regions drawn
over it. A bare grid says where the walls are; what makes one saved map
recognisable is what someone marked on it. Opening a card lists all of both,
read out of band: the regions from that map's own annotation file and the
objects from its snapshot partition, without rebinding the live store, because
inspecting a map in the library must not move the session.
Two endpoints behind it, both read-only. `/api/maps/{id}/preview` serves the
image mapping has written beside every saved map since it was written and
which the web layer had no route for -- the list printed the word "preview"
and never showed one. `/api/maps/{id}/contents` answers "what is in this map"
without loading it, and carries the grid geometry so polygons in metres can be
drawn over an image in pixels.
A status strip on every page, in the shell rather than remembered per page:
what the robot is doing and which map it is doing it to. Those two facts
decide what every number below them means -- an object list means something
different while the map is still being built than while the robot localises
against a saved one -- and they were stated in one corner of one page. Mode
carries colour so it does not have to be read.
Controls are shared. Five files had defined their own button, which is why one
panel looked like a different product from the one beside it; `.btn`, `.field`
and `.tag` live in shell.css and the maps page and the dock use them. Motion
is shared for the same reason: one duration and one curve, on colour and
position only, never on layout, and off entirely under
`prefers-reduced-motion`.
The dock updates in place instead of rebuilding twice a second. Rewriting
`innerHTML` every poll destroyed the nodes under the reader, so a selection
collapsed the instant it was made and an object id could not be copied. Rows
are keyed by object id and a cell whose text has not changed is not written at
all -- assigning identical text to a Text node still collapses it.
`hidden` now actually hides: `[hidden] { display: none }` is a user-agent rule
and loses to any author `display`, so `.mg-more { display: grid }` left the
block laid out while the attribute said otherwise. It looked closed only
because its contents had been emptied, and the test that asked the browser
rather than the attribute was right to fail.
The viewer's data path is back in-process. Hosting rerun's servers as child
processes cured a boot deadlock -- `serve_grpc` holds the interpreter lock
under contention from the asyncio loop, the rclpy executor and a CUDA tick, so
the bounded `join` meant to catch it can never be scheduled -- but nothing
then reached the viewer, verified three ways through two paths. The deadlock
is diagnosed with a captured stack and left for a fix that does not cost the
viewer; NOTES-open-issues.md records that and the rest of what is still open.
51 UI tests against a live Webots stack.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Scene drew its own map. A three.js page pulled the library from a CDN and needed WebGL, and a canvas floor plan sat beside it. On a headless robot neither is available, so the 3D view rendered nothing there, and keeping both pages working was viewer maintenance that has nothing to do with a scene graph. The pages also had no links between them: reaching the annotation view from the map meant editing the address bar, and nothing on any page said the other views existed.
A live Webots office session. Each object is its own point cloud carrying one label; the occupancy grid is drawn underneath as the floor, white where the robot has observed it and dark at the walls; the orange polygon is the robot's own footprint with its heading. Relation edges are drawn thin and pale on purpose — they are context, and the clouds are the evidence. The box each object reports is logged but hidden, and switched on from the entity tree when a detection needs checking.
Change
The map pages are rerun views. Scene logs what it knows and rerun draws it. What is logged is the annotated map, not raw sensor data: one point cloud per object coloured by instance and carrying its class label, a second cloud in the colours the camera saw, the occupancy grid as a textured floor with one colour per cell meaning, one edge per relation, and the robot's own footprint polygon with a heading arrow.
The 3D page is the landing page, because whether the perception is any good is the question this UI is opened to answer. The 2D page is the same annotations seen from above, drawn in the grid's own pixel coordinates so they line up with the map rather than with a second drawing of it. Read that way, an object either sits in the room it belongs to or it does not, and a relation either crosses a wall or it does not. Every page now shares a sidebar.
Each object's reported box is logged but hidden by default. It is the perception layer's own extent estimate, it is often wrong, and a wireframe around every object buries the points that are the actual evidence. It is switched on from the entity tree when a detection needs checking.
Two recordings on two gRPC ports (9876 and 9877) are served by one web viewer (9090), because a blueprint belongs to an application and the viewer takes its data source from the query string.
Why rerun
rerun (rerun-io/rerun, Apache-2.0) is an open-source SDK and viewer for multimodal robotics data. It supplies four things this PR would otherwise have had to build and keep working:
The viewer, and the server that serves it. The page being replaced pulled three.js from a CDN and needed WebGL in the browser. A headless robot has neither, so that page rendered nothing exactly where a viewer is most useful. rerun ships the viewer as a WASM bundle, an HTTP server for it, and the wire protocol between them, so Scene logs entities and links to the viewer instead of maintaining one.
A data model that matches what Scene has. rerun is entity-and-component, not draw calls. Each object becomes three entity paths —
rgb_pcd/<id>for the colours the camera saw,sem_pcd/<id>for one colour per instance,bbox/<id>for the extent estimate — and the viewer's entity tree makes each one independently visible. That is why the debug boxes can be a layer the reader switches on rather than a UI feature we implement: the blueprint hides/map/objects/bboxby default and the tree switches it back on.A time axis for free.
set_timeputs every tick at a point on a timeline, so dragging it replays the map filling in as the robot drove. That is what separates a detection that persisted from one that appeared for a single frame, and it is not something a hand-rolled viewer gets without real work.A declarative layout. The blueprint is sent with the data, so a page opens on the right view with the panels collapsed, with no viewer-side configuration for an operator to get wrong.
It is the same shape of data the official examples show
rerun's own examples include several that are structurally what Scene produces, which is a good sign that the model fits rather than that we are bending it:
The three-layer per-object layout follows DualMap, which uses rerun the same way and is already a selectable Scene backend.
What it costs, stated honestly
SetTheme, so the light theme applies to the view contents and Scene's own shell, not to that strip.numpy>=2, which the perception stack cannot take. It is therefore an optional extra, not a dependency, and the x86 image installs it with--no-depsand pinned imports. Native installs keep the built-in pages and nobody porting Scene has to solve for it.Native deployments are unchanged
web_viewerin the Scene config, orSCENE_WEB_VIEWER, selectsauto,rerunorbuiltin.autouses rerun when it is importable, which is true in the x86 docker image and false natively, so a native install keeps exactly the pages it had. rerun is an optional extra rather than a dependency: a port to a new board has to be able to bring Scene up without solving for it, and rerun-sdk declaresnumpy>=2, which the perception stack cannot take. The x86 image installs it with--no-depsand pinned imports. Settingrerunwhere it cannot start fails the start with the reason rather than serving an empty frame. The built-in pages stay reachable at?bare=1either way.Boundary
The sink only reads. It takes a registry snapshot once a second and publishes it. Object persistence, annotations and the map lifecycle are untouched, and a viewer that cannot start does not stop the map service.
Validation
robonix_manifest.scene-eval.cg.nav.yaml), driven by the standard Explore skill. Verified in the live recording that every layer is logged:/map/floor,/map/objects/{rgb_pcd,sem_pcd,bbox}/<id>,/map/relations,/map/robot/{footprint,heading}, and the/map2dequivalents./,/2d,/cam,/user,/3dand each?bare=1form.pytest system/scene/tests/{test_rerun_sink,test_web_live_views,test_scene_graph,test_vlm_inference_cache,test_perception_vlm_geometry,test_image_relations,test_web_binding}.py: 95 passed.test_rerun_sink.pyis added to the CI Scene job.No Rust changes. No capability contract, generated code, or migration changes.
Also
testing/export_scene_pointcloud.pywrites the same objects to a coloured PLY and an.rrdfor offline comparison against ground truth.