Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 60 additions & 0 deletions claude-notes/plans/2026-06-24-fma-torso-bodyparts3d-splat.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# FMA torso gaussian splat — real BodyParts3D geometry, two cockpit pages

> 2026-06-24 · status: in progress
> The `/fma` heart slice (PR #51) renders the FMA partonomy with *synthesized*
> 8:8-HHTL layout because FMA itself has **zero geometry**. This adds a **torso**
> rendered from **real anatomical meshes** — BodyParts3D, which keys 3D meshes
> directly on FMA concept IDs — across two new cockpit pages.

## The convergence

- **FMA partonomy** (`part_of`) = mereotopological containment in BodyParts3D
(NAR 2009, Table 1: `A part_of B` ⟺ `A° ⊂ B°`, coveredBy). So the HHTL
`[container:identity]` cascade *is* the spatial nesting.
- **BodyParts3D** (DBCLS, CC-BY 4.0) realizes FMA concepts as OBJ meshes in one
shared whole-body frame. `concept id` column **is** the FMA id.
- Z-Anatomy / the Unity app are curated atlases on the same data; we use the raw
FMA-keyed OBJ (no Blender needed).

## Data (measured)

- Root `FMA7181 trunk` → **178 concepts, 577 OBJ meshes, ~694K verts**, all present.
Regions: thoracic segment (548 meshes), body wall (81), abdominal (24), perineum (4).
The heart (PR #51) nests inside (`trunk → … → content of middle mediastinum → heart`).
- Source (external, NOT committed): `dbarchive.biosciencedbc.jp/.../partof_BP3D_4.0_obj_99.zip`
+ `partof_inclusion_relation_list.txt` / `partof_element_parts.txt` / `partof_parts_list_e.txt`.
- License: **CC-BY 4.0** — attribution: "BodyParts3D, © The Database Center for
Life Science licensed under CC Attribution 4.0 International".

## Pipeline

```
BodyParts3D (FMA partof tree + FJ OBJ meshes)
→ tools/bake_torso_splat.py (BFS FMA7181 → concepts → FJ meshes → vertices,
recenter/normalize, region-colour, downsample)
→ cockpit/public/torso.splat (SPL1 binary: real positions + rgb + opacity)
→ /torso-live three.js orbit (reads SPL1; the "live orbit")
→ torso.ply (Inria) → ndarray splat3d_flex → frames → /torso (the "splat", CPU)
```

`splat3d` (ndarray, CPU-SIMD, no GPU) renders bake-side via its own 1.95 toolchain
(q2 stays clean of the ndarray dep). Both pages read ONE asset → identical geometry.

## Checklist

- [x] `tools/bake_torso_splat.py` — BodyParts3D → SPL1 gaussian asset + manifest
(231K gaussians, 577 meshes, 102 structures; muted pastel per-structure hues)
- [x] `cockpit/public/torso.splat` (3.7 MB) + `torso.manifest.json` (attribution/legend)
- [x] SPL1 TS decoder + `/torso-live` three.js orbit page + route
- [x] `/torso` splat3d CPU render: scratchpad `torso-render` driver reads SPL1 →
`Gaussian3D` → ndarray::hpc::splat3d turntable (no Inria .ply needed) →
20 JPEG frames in `cockpit/public/torso-frames/` → `/torso` viewer page + route
- [x] attribution surfaced in-UI; tsc clean
- [ ] PR

Notes:
- The CPU render runs under ndarray's own 1.95 toolchain (scratchpad project,
path-dep on ../ndarray) — q2's workspace stays free of the ndarray dep.
~6.6 s/frame on the scalar path (no AVX target-cpu in the scratchpad project);
correctness verified by viewing the rendered frames.
- Colours: golden-angle hue per structure at S=0.34 V=0.78 (muted, per request).
Binary file added cockpit/public/torso-frames/torso_000.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added cockpit/public/torso-frames/torso_001.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added cockpit/public/torso-frames/torso_002.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added cockpit/public/torso-frames/torso_003.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added cockpit/public/torso-frames/torso_004.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added cockpit/public/torso-frames/torso_005.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added cockpit/public/torso-frames/torso_006.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added cockpit/public/torso-frames/torso_007.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added cockpit/public/torso-frames/torso_008.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added cockpit/public/torso-frames/torso_009.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added cockpit/public/torso-frames/torso_010.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added cockpit/public/torso-frames/torso_011.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added cockpit/public/torso-frames/torso_012.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added cockpit/public/torso-frames/torso_013.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added cockpit/public/torso-frames/torso_014.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added cockpit/public/torso-frames/torso_015.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added cockpit/public/torso-frames/torso_016.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added cockpit/public/torso-frames/torso_017.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added cockpit/public/torso-frames/torso_018.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added cockpit/public/torso-frames/torso_019.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
42 changes: 42 additions & 0 deletions cockpit/public/torso.manifest.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
{
"source": "BodyParts3D 4.0 (DBCLS) part-of OBJ, decimated 99%",
"license": "CC-BY 4.0",
"attribution": "BodyParts3D, (c) The Database Center for Life Science licensed under CC Attribution 4.0 International",
"root_fma": "FMA7181",
"root_name": "trunk",
"concepts": 102,
"meshes": 577,
"vertices_total": 693959,
"gaussians": 231320,
"radius": 0.0045,
"bbox_min": [
-0.5499420475739409,
-0.2690260109315338,
-0.9999999999999999
],
"bbox_max": [
0.5499420475739409,
0.2690260109315338,
1.0000000000000002
],
"regions": [
{
"fma": "FMA259209",
"name": "thoracic segment of trunk"
},
{
"fma": "FMA259211",
"name": "abdominal segment of trunk"
},
{
"fma": "FMA10427",
"name": "body wall"
},
{
"fma": "FMA9579",
"name": "perineum"
}
],
"generated_by": "crates/osint-bake/tools/bake_torso_splat.py",
"format": "SPL1: hdr[magic4|count u32|radius f32|bbox_min 3f|bbox_max 3f]; body count*[pos 3f|rgb 3u8|opacity u8]; little-endian"
}
Binary file added cockpit/public/torso.splat
Binary file not shown.
110 changes: 110 additions & 0 deletions cockpit/src/TorsoRender.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
// FMA torso · splat3d CPU render (the "splat" page).
//
// A turntable of frames rendered bake-side by ndarray::hpc::splat3d — the
// CPU-SIMD EWA gaussian-splat renderer (no GPU) — over the same torso.splat
// asset the /torso-live page orbits live. The frames are baked from real
// BodyParts3D anatomy (FMA-keyed meshes); see crates/osint-bake/tools/
// bake_torso_splat.py + the scratchpad torso-render driver.
//
// This is the "splat" companion to /torso-live's "live orbit": splat3d owns
// these pixels (the spine renders), three.js owns the live one. Same geometry.
//
// Geometry: BodyParts3D, (c) The Database Center for Life Science, CC-BY 4.0.
import { useEffect, useRef, useState } from 'react';

const FRAME_COUNT = 20;
const frameSrc = (i: number) => `/torso-frames/torso_${String(i).padStart(3, '0')}.jpg`;

interface Manifest {
attribution: string;
gaussians: number;
meshes: number;
concepts: number;
}

export function TorsoRender() {
const [frame, setFrame] = useState(0);
const [loaded, setLoaded] = useState(0);
const [playing, setPlaying] = useState(true);
const [manifest, setManifest] = useState<Manifest | null>(null);
const drag = useRef<{ x: number; f: number } | null>(null);
const playRef = useRef(true);

useEffect(() => { playRef.current = playing; }, [playing]);

// preload all frames
useEffect(() => {
let n = 0;
for (let i = 0; i < FRAME_COUNT; i++) {
const im = new Image();
im.onload = () => { n += 1; setLoaded(n); };
im.src = frameSrc(i);
}
fetch('/torso.manifest.json')
.then((r) => (r.ok ? r.json() : null))
.then((m) => m && setManifest(m as Manifest))
.catch(() => {});
}, []);

// auto-rotate
useEffect(() => {
const id = window.setInterval(() => {
if (playRef.current) setFrame((f) => (f + 1) % FRAME_COUNT);
}, 95);
return () => window.clearInterval(id);
}, []);

const onDown = (e: React.PointerEvent) => {
setPlaying(false);
drag.current = { x: e.clientX, f: frame };
(e.target as Element).setPointerCapture?.(e.pointerId);
};
const onMove = (e: React.PointerEvent) => {
if (!drag.current) return;
const df = Math.round((e.clientX - drag.current.x) / 16);
setFrame((((drag.current.f - df) % FRAME_COUNT) + FRAME_COUNT) % FRAME_COUNT);
};
const onUp = () => { drag.current = null; };

const ready = loaded >= FRAME_COUNT;

return (
<div style={{ position: 'fixed', inset: 0, background: '#0a0e17', overflow: 'hidden', userSelect: 'none' }}>
<div
onPointerDown={onDown}
onPointerMove={onMove}
onPointerUp={onUp}
onPointerCancel={onUp}
onClick={() => drag.current === null && setPlaying((p) => !p)}
style={{ position: 'absolute', inset: 0, display: 'flex', alignItems: 'center', justifyContent: 'center', cursor: 'ew-resize', touchAction: 'none' }}
>
{/* eslint-disable-next-line jsx-a11y/alt-text */}
<img
src={frameSrc(frame)}
draggable={false}
style={{ maxHeight: '100%', maxWidth: '100%', opacity: ready ? 1 : 0.25, transition: 'opacity 0.3s', imageRendering: 'auto' }}
/>
</div>

<div style={{ position: 'absolute', top: 12, left: 16, color: '#cdd9e5', font: '13px ui-monospace, monospace', pointerEvents: 'none' }}>
<div style={{ fontSize: 15, color: '#fff' }}>FMA torso · splat3d CPU render</div>
<div style={{ opacity: 0.7 }}>
ndarray::hpc::splat3d (EWA, no GPU)
{manifest ? ` · ${manifest.gaussians.toLocaleString()} gaussians · ${manifest.concepts} structures` : ''}
{' · real BodyParts3D anatomy'}
</div>
<div style={{ opacity: 0.55, marginTop: 2 }}>
{ready ? `${playing ? 'auto-rotating' : 'paused'} · drag to scrub · click to ${playing ? 'pause' : 'play'}` : `loading frames ${loaded}/${FRAME_COUNT}…`}
</div>
</div>

<div style={{ position: 'absolute', top: 14, right: 16, font: '12px ui-monospace, monospace' }}>
<a href="/torso-live" style={{ color: '#7fa6c4', textDecoration: 'none' }}>live orbit →</a>
</div>

<div style={{ position: 'absolute', bottom: 10, left: 16, color: '#5a6b7e', font: '10px ui-monospace, monospace', maxWidth: '70%', pointerEvents: 'none' }}>
{manifest?.attribution ?? 'BodyParts3D, (c) The Database Center for Life Science, licensed under CC Attribution 4.0 International'}
</div>
</div>
);
}
Loading