Skip to content

Fix Gaussian reconstruction, compressed playback, and codec regressions - #54

Closed
cicm4 wants to merge 2 commits into
codex/library-api-and-bug-fixesfrom
Guassian_Python_API_Test
Closed

cicm4 wants to merge 2 commits into
codex/library-api-and-bug-fixesfrom
Guassian_Python_API_Test

Conversation

@cicm4

@cicm4 cicm4 commented Sep 21, 2026 •

Copy link
Copy Markdown
Member

What changed

Small QUEEN scenes and disabled gates could not save compressed frames; 3DGStream's defaults could crash or restart a temporal frame from its input cloud; short TSMC clips indexed nonexistent frames. These paths now complete actual training, encoding, decoding and playback.

This PR targets codex/library-api-and-bug-fixes, the existing API branch used for validation, and contains 23 repair, regression-test, dependency and documentation files.

  • Keep QUEEN computation indices valid while compacting their serialized representation, support empty residual streams, and handle disabled gates during save/load.
  • Validate 3DGStream NTC coverage, adapt completely disjoint scene bounds while retaining warmed identity weights, handle evaluation without test cameras, and load the previous saved temporal model by default.
  • Support TSMC clips with two through five frames while retaining the existing reference choice for longer clips.
  • Restore the CUDA rasterizer's 0.2-unit near guard and remove Vega's default six-unit scaling workaround; ORBIT now renders at its original camera distances.
  • Probe N4MC's required MPS operation before training: automatic selection falls back to CPU when unsupported, while explicit MPS requests receive an actionable error.
  • Make the ORBIT dataset root configurable, avoid inaccessible-path collection errors, and update compilation, native rebuild and vendor-provenance documentation.
  • Constrain the RGB-D runtime to Open3D 0.19.x and fail with an actionable error on incompatible manual installs. Open3D 0.20's legacy TSDF path returned empty meshes in real reconstruction tests.

Verification

Original repair commit (4cb577b): python -m pytest -q on macOS / Apple Silicon, Python 3.12.13 and Torch 2.7.0: 616 passed, 17 skipped, no failures. This checkout excludes four pre-existing local editor/viewer edits. The earlier 618-pass audit included those edits. Markdown-link and provenance checks and git diff --check also pass on the PR checkout.

CI compatibility update (62c772f): All 24 GitHub CI jobs pass, including Open3D on Python 3.10–3.12 and clean-wheel installation across Linux, macOS and Windows on Python 3.10–3.13. Local reconstruction/transport verification passed 31 tests (one external native-codec skip); seven unsupported-version/validation checks passed in an Open3D 0.20 environment. The five empty-mesh failures reproduced with 0.20 and passed with 0.19 while keeping NumPy 2.5.3 unchanged.

Native CUDA regressions: Linux / RTX 4090, Python 3.10.16, Torch 2.6.0+cu124, and the rebuilt bundled rasterizer:

Suite Result
Native Vega, including accessible ORBIT data and near-camera pixel/gradient checks 44 passed
Public Gaussian API with native runtime enabled 37 passed
QUEEN compressed round trips: 12, 900 and 32,769 splats; active, closed and disabled gates 9 passed
3DGStream evaluation, temporal loading and NTC bounds 4 passed

These suites overlap; their counts are not a combined unique-test total. Before the repairs, focused regressions reproduced failures in QUEEN (8/9), 3DGStream (4/4), TSMC reference selection (5/6), N4MC auto selection (1/1), and the near-camera rasterizer (2/3).

Actual workflows:

  • TSMC encoded two- and four-frame Rafa captures, decimated to 2,000 triangles. Encoder inputs were removed, artifacts relocated, and two fresh decoder processes produced identical geometry while preserving timestamps and metadata. Maximum sampled surface RMS was 0.583% / 0.566% of the input bounding-box diagonal.
  • QUEEN trained, compressed and played back three-frame, 900-splat synthetic scenes with gating enabled and disabled. All final rendered frames were distinct and visible. The synthetic camera-path focus bounds were corrected from the known cloud without changing trained or encoded models.
  • 3DGStream trained and rendered three synthetic frames using the corrected defaults, without an alternate cache, --eval or an explicit temporal-load override. Logs confirm loading iteration 30 followed by iteration 10.
  • Public Vega encoded six ORBIT basketball time steps containing 63–70k splats per frame. A relocated artifact was decoded in a fresh process and rendered from two calibrated views per frame, at original camera/world scale.
  • QUEEN and 3DGStream each reconstructed three ORBIT time steps from eight calibrated views. QUEEN native compressed playback also passed; residual-frame foreground MAE versus dense rendering was 0.000373 and 0.000497.
  • The native Vega HTTP/decode/CUDA-render/MJPEG path delivered 120 distinct, nonblank images to the receiving Mac at 15.10 measured FPS. Six scene frames repeat as the camera moves; this is not 120 reconstructed time steps. Gaussian chunks were decoded on the server; the Mac received rendered images.

Wheel and sdist builds, both distribution-boundary checks, an isolated installed-wheel I/O/CLI/mesh-TCP smoke run, and supported-path compilation passed during the fix audit. The expected publication block remains in place. Executed native source files were hash-checked against the 20 repair files in this branch. Logs, bitstreams and ten playback recordings are retained locally with the audit report; generated artifacts and datasets are not committed.

Safety checklist

  • Tests cover success, empty input, and relevant failure paths.
  • Optional dependencies remain optional and have actionable errors.
  • Wheel/sdist boundaries are unchanged or their exact checks were updated.
  • THIRD_PARTY.md was updated for copied code, data, models, papers, media, binaries, or submodules.
  • No credentials, private captures, generated outputs, or local datasets were added.
  • Documentation and status claims match verified behavior.

Remaining limitations

The rasterizer source fix requires rebuilding diff-gaussian-rasterization in each affected environment; the tested server environment was rebuilt. Separately cached native TSMC checkouts need the updated reference-center script when selected as a backend override.

Native N4MC MPS support remains unavailable in the tested Torch runtime; automatic execution now uses CPU. These are short functional runs, not production-quality or sustained-load benchmarks. Some streamed novel views show blur/ghosting with the short training budgets. Standalone SIBR residual playback and browser-side Gaussian decoding were not verified.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

@blacksmith-sh

This comment has been minimized.

@cicm4 cicm4 closed this Sep 23, 2026
@cicm4
cicm4 deleted the Guassian_Python_API_Test branch September 23, 2026 16:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant