Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -37,3 +37,6 @@

# Keep this small, reviewed, output-free walkthrough readable in code reviews.
/examples/imaging-pipeline/run_imaging_pipeline.ipynb -filter diff merge text eol=lf

# Original output-free Model Scene notebook.
/examples/model-scene/model-scene.ipynb -filter diff merge text eol=lf
99 changes: 99 additions & 0 deletions examples/model-scene/PUBLISHING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
<!-- SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. -->
<!-- SPDX-License-Identifier: Apache-2.0 -->

# Publishing Model Scene

The source notebook is output-free. The interactive HTML and numerical study
are generated artifacts kept outside Git. This example exports a local static
chapter inside the user guide; merging its source does not add it to the
automatically published versioned site.

## Build a reviewable artifact

1. Check out the intended committed source revision and keep it unchanged
through execution and rendering. Install that checkout's cuPhoton with
xImage and the optional example requirements, as described in README.md.
2. Execute the reference profile in a CUDA 13 environment, retaining the shared
GPU lock. Use a fresh destination outside the source checkout:

```sh
python examples/model-scene/run.py --profile reference --device cuda \
--gpu-lock /path/shared-by-all-gpu-users/gpu0.lock \
--output /path/new-scene-reference
```

3. Check `execution.json`, `protocol.json`, `execution.log`, and the study
diagnostics. Verify source hashes against this checkout, protocol hashes
against the executed protocol, and artifact hashes against the actual files.
Inspect the independent-truth endpoints and calibration controls; a successful
process exit alone does not establish scientific fidelity.
4. In the CPU documentation environment, render the completed notebook with
`preview.py --executed ... --output ... --source-ref COMMITTED_PUSHED_REF`.
The command verifies input hashes and invokes the existing postprocessor
and snapshot checks. Normal CPU guide chapters execute; the GPU notebook
retains its saved outputs. Build include fragments stay outside the
published guide resource tree.
5. Test the chapter under `file://` and static HTTP, including a versioned
`/cuPhoton/<version>/` prefix. Check desktop/mobile, light/dark, keyboard,
touch, terminology definitions, all selectors, linked pixel values and
display scales. Block scientific runtime network access and verify that
there are no WebSocket connections. Audit the HTML and public receipt for
private paths and restricted material. Confirm GitHub source links resolve.

## Package static files

The complete `_book` directory retains guide navigation and source downloads.
The chapter embeds its plot resources; a local static web server is sufficient.
After verification, package the rendered directory and executed notebook:

```sh
tar -czf /path/model-scene-guide.tar.gz \
-C /path/new-scene-preview/guide/_book .
cp /path/new-scene-preview/guide/_book/source/model-scene.executed.ipynb \
/path/model-scene.executed.ipynb
cp /path/new-scene-preview/guide/_book/source/model-scene-execution.json \
/path/model-scene-execution.json
sha256sum /path/model-scene-guide.tar.gz \
/path/model-scene.executed.ipynb /path/model-scene-execution.json \
> /path/model-scene-SHA256SUMS
```

Use the sanitized exported notebook/receipt for public distribution. Retain raw
native workflow records privately when they contain machine-local paths.
Full repeated-trial arrays can be archived separately; the HTML contains selected
experiments and the complete summary diagnostics. Measure the archive and page
sizes before distribution. Preserve numeric precision and avoid embedding
repeated-trial arrays merely to make them accessible from a selector.

Uploading assets is a separate future action. After approval, use an existing
GitHub Release whose tag contains the exact validated source:

```sh
gh release upload RELEASE_TAG --repo NVIDIA/cuPhoton \
/path/model-scene-guide.tar.gz /path/model-scene.executed.ipynb \
/path/model-scene-execution.json /path/model-scene-SHA256SUMS
```

This uploads assets; it neither creates a software release nor updates Pages.
Preserve existing assets. No Release upload is part of this example's local
preview or PR workflow.

## Versioned Pages integration still required

The current CPU builder globally requests execution. It needs a verified way to
stage the pre-executed GPU notebook with matching source/protocol/dependency
receipts while continuing to execute the ordinary CPU chapters. The local
preview demonstrates that separation without changing the central publisher.

The publisher also needs durable storage and restoration of large static
artifacts keyed to the exact source revision. A Release upload alone does not
make the publisher consume those assets. Main needs durable revision-keyed
artifacts too; expiring PR artifacts and a mutable latest-release fallback do
not satisfy source provenance. Keep rendered HTML out of `gh-pages` source
commits until that integration has an approved storage design.

Before enabling automatic publication, validate main, a release that contains
Model Scene, and an older release without it. Verify source links, navigation,
search, version switching, artifact integrity and preservation of earlier
snapshots. Future `nvidia.github.io` hosting must consume static artifacts only.
No public Python or Bokeh server is available or required.
129 changes: 129 additions & 0 deletions examples/model-scene/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
<!-- SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. -->
<!-- SPDX-License-Identifier: Apache-2.0 -->

# Model Scene

What can blurred exposures actually reveal? This notebook reconstructs a
static modeled scene with xImage, calibrates PSFs from separate modeled star
observations, and measures recovery against independent truth. Interactive
figures connect individual exposures with predictions and residuals.

Every input comes from the analytic generator in this directory. The example
uses no survey images, calibration products, catalogs or downloaded fixtures.
The physical renderer integrates continuous Gaussian mixtures over detector
pixels. A separate discrete-operator control quantifies representation error.
A 96×96 fitting region provides context around the central 48×48 science
window. A separate compact-context case demonstrates halo contamination.
The experiment uses a zero-dither grid and makes no alignment qualification.

## Execute

Use a fresh Linux Python 3.12+ environment with CUDA 13 and a compatible NVIDIA
GPU. From a checkout of the exact cuPhoton revision being evaluated:

```sh
python -m pip install -e '.[ximage]'
python -m pip install -r examples/model-scene/requirements.txt
python examples/model-scene/run.py --profile quick --device cuda \
--gpu-lock /path/shared-by-all-gpu-users/gpu0.lock \
--output /path/outside/git/new-scene-smoke
```

Pass the same advisory lock path to other cooperating GPU workloads. The runner
retains that lock through the study and the fresh notebook kernel. Inspect GPU
use before launching; a process that ignores the advisory lock is not blocked
by it. The destination must be absent, including absent as a symbolic link.
The runner never overwrites an earlier execution.

The quick profile exercises the bundle contracts, solver APIs, investigations
and plots with two trials and ten scene iterations. It substantiates no
scientific-performance claim. `--device cpu` provides an API smoke route.
The reference profile runs the frozen protocol, including at least 50 paired
science-noise realizations and 50 independent calibration draws:

```sh
python examples/model-scene/run.py --profile reference --device cuda \
--gpu-lock /path/shared-by-all-gpu-users/gpu0.lock \
--output /path/outside/git/new-scene-reference
```

`protocol.json` records the measurement definitions and seed ranges. A reference
run refuses an unfrozen protocol or a change to its scientific source hashes.
The reproducible development sweep uses separate seeds and a catalog that shifts the two isolated stars by two pixels:

```sh
python examples/model-scene/develop.py --device cuda \
--gpu-lock /path/shared-by-all-gpu-users/gpu0.lock \
--output /path/outside/git/new-scene-development
```

Development evidence determines the iteration budget before
test draws. The Huber threshold is declared in advance, not tuned by this sweep.
An invalidating scientific fix requires a new protocol revision
and fresh test seeds; retain the failed attempt and its receipt.

The source notebook has no outputs, execution counts, attachments or datasets.
`run.py` creates a source copy, native workflow artifacts, numerical study,
execution log, executed notebook and an execution receipt. It records source,
protocol and output hashes, exact source revision and package versions.
`CUPHOTON_MODEL_SCENE_OUTPUT` locates those results for notebook execution.
The notebook preserves warnings and recorded outputs. Work offline after
installing dependencies; no data download is part of generation or execution.

## Interpret the measurements

The primary endpoint is fractional error of radius-three-pixel aperture flux
at a common 2.4-pixel Gaussian resolution. Compare each fixed aperture with
the identical aperture on target truth. Apertures include neighboring light;
they do not measure deblended total source flux. Per-source summaries retain
brightness and crowding labels. This is inference about the fixed modeled
catalog and independent noise draws, rather than a population calibration.

The reference compares L2, Huber and a coadd on paired draws, then investigates
PSF-width errors, finite calibration, outliers, explicit masking, variance
rescaling, photon/read noise and a transient. Held-out frames test forward
predictions. Native coadd-relative metrics are secondary diagnostics. See the
notebook's derivations and caveats before interpreting a residual statistic
or a visually sharp feature.

## Inspect a static export

Use the CPU documentation environment and Quarto described in
[the documentation guide](../../docs/README.md) and
[developer tooling](../../dev/README.md):

```sh
python examples/model-scene/preview.py \
--executed /path/new-scene-reference/model-scene.executed.ipynb \
--output /path/outside/git/new-scene-preview \
--source-ref YOUR_PUSHED_SOURCE_REF \
--controller-python /path/to/developer-env/bin/python
python -m http.server 8798 --bind 0.0.0.0 \
--directory /path/new-scene-preview/guide/_book
```

Open `http://localhost:8798/model-scene.html`. The chapter embeds its JavaScript
and numeric data. Each experiment selector chooses saved results. It requires
no Python server callbacks, kernel, WebSocket, API or CDN. Individual standalone
chapter viewing also works with `file://`; guide navigation needs the rest of
the exported directory. Display-range changes retain the original pixel values.

Use `--controller-python` when the developer tooling dependencies (including
Beautiful Soup) live in a separate environment from the CPU guide renderer.

The preview verifies the execution receipt and source hashes, executes the
ordinary CPU guide chapters, preserves the completed GPU notebook outputs,
and invokes the current version postprocessor and snapshot checks. It does
not modify the central publisher. Source links must name a pushed ref.

Run the numerical tests with `python -m pytest examples/model-scene/tests`.
Workflow controls additionally require the xImage dependencies.

[Publishing instructions](PUBLISHING.md) describe packaging and the current
versioned-Pages integration gap. Opening or merging this source PR does not
publish the rendered chapter.

The frozen scientific source was replayed with the development command above.
Its twelve rows agree numerically with the selection sweep. The external
`development-v2-replay/development.json` receipt has SHA-256
`355ce588c2ccc039a1d1bbfe312576d9f4b53206c8e36e27c4c720e3a6304d6f`.
31 changes: 31 additions & 0 deletions examples/model-scene/assets/scene.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
/* SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. */
/* SPDX-License-Identifier: Apache-2.0 */
:root { --scene-accent:#247359; }
body.quarto-dark { color-scheme:dark; --scene-accent:#a6df36; }
body.quarto-light { color-scheme:light; }
main { line-height:1.65; min-width:0; }
.scene-term { color:var(--scene-accent); text-decoration:underline dotted;
cursor:help; border:0; background:none; font:inherit; padding:0; }
.scene-term:focus-visible { outline:2px solid currentColor; outline-offset:3px; }
#scene-definition { position:fixed; z-index:9999; max-width:min(27rem,90vw);
background:Canvas; color:CanvasText; border:1px solid currentColor;
padding:.8rem; border-radius:.4rem; box-shadow:0 4px 24px #0003; }
details.science { border-left:3px solid var(--scene-accent); padding:1rem;
margin:1.4rem 0; background:color-mix(in srgb, CanvasText 4%, Canvas); }
details.science summary { cursor:pointer; font-weight:600; }
.cell-output-display { max-width:100%; overflow-x:auto; }
@media(max-width:700px) { main { padding:.5rem; } }
.scene-pipeline { margin:1.5rem 0; padding:1rem; border:1px solid color-mix(in srgb, CanvasText 30%, Canvas); border-radius:.5rem; }
.scene-pipeline-preparation { display:flex; align-items:center; justify-content:center; gap:1rem; }
.scene-pipeline-node { display:flex; flex-direction:column; gap:.2rem; padding:.7rem 1rem; border:1px solid currentColor; border-radius:.35rem; }
.scene-pipeline-node strong { font-size:1.15em; }
.scene-pipeline-arrow { font-size:1.8em; }
.scene-pipeline-junction { text-align:center; padding:.8rem 0; border-bottom:1px solid currentColor; }
.scene-pipeline-branches { display:grid; grid-template-columns:1fr 1fr; gap:1rem; }
.scene-pipeline-branch { min-width:0; padding:1rem; border:1px solid color-mix(in srgb, CanvasText 30%, Canvas); border-radius:.4rem; }
.scene-pipeline-current { border:3px solid var(--scene-accent); background:color-mix(in srgb, var(--scene-accent) 8%, Canvas); }
.scene-pipeline-label { font-weight:700; margin-top:0; }
.scene-pipeline-current .scene-pipeline-label { color:var(--scene-accent); }
.scene-pipeline-connector { border-left:2px solid currentColor; margin-left:1rem; padding:.5rem .8rem; }
.scene-pipeline figcaption { margin-top:1rem; font-size:.9em; }
@media(max-width:700px) { .scene-pipeline-branches { grid-template-columns:1fr; } .scene-pipeline-preparation { flex-direction:column; } .scene-pipeline-arrow { transform:rotate(90deg); } }
90 changes: 90 additions & 0 deletions examples/model-scene/assets/scene.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0
(() => {
function start() {
function themePlots() {
const dark = document.body.classList.contains('quarto-dark');
const background = dark ? '#181b16' : '#ffffff';
const text = dark ? '#e6eddf' : '#172b45';
for (const doc of window.Bokeh?.documents ?? []) {
for (const model of doc.all_models.values()) {
if (model.type === 'Figure') {
model.background_fill_color = background;
model.border_fill_color = background;
}
if (model.type === 'Title') model.text_color = text;
if (model.type.endsWith('Axis')) {
model.axis_label_text_color = text;
model.major_label_text_color = text;
}
if (model.type === 'Legend') {
model.background_fill_color = background;
model.label_text_color = text;
}
if (model.type === 'ColorBar') {
model.background_fill_color = background;
model.major_label_text_color = text;
model.title_text_color = text;
}
}
}
}
new MutationObserver(themePlots).observe(document.body,
{attributes:true, attributeFilter:['class']});
// Bokeh embeds initialize after DOM parsing; the load event follows them.
addEventListener('load', themePlots);
themePlots();
document.querySelectorAll('.cell-output-display').forEach(region => {
if (region.querySelector('[data-root-id]')) {
region.tabIndex = 0;
region.setAttribute('aria-label', 'Interactive figure; arrow keys scroll this region');
}
});
const popup = document.createElement('div');
popup.id = 'scene-definition'; popup.role = 'tooltip'; popup.hidden = true;
document.body.appendChild(popup);
let active = null;
function close() {
popup.hidden = true;
if (active) active.removeAttribute('aria-describedby');
active = null;
}
function open(term) {
close(); active = term;
popup.textContent = term.dataset.definition;
term.setAttribute('aria-describedby', popup.id); popup.hidden = false;
const rect = term.getBoundingClientRect();
popup.style.left = Math.max(8, Math.min(rect.left,
innerWidth - popup.offsetWidth - 8)) + 'px';
const below = rect.bottom + 6;
const top = below + popup.offsetHeight <= innerHeight - 8
? below : rect.top - popup.offsetHeight - 6;
popup.style.top = Math.max(8, top) + 'px';
}
document.querySelectorAll('.scene-term').forEach(term => {
let pointerWasOpen;
term.addEventListener('pointerdown', () => { pointerWasOpen = active === term; });
term.addEventListener('mouseenter', () => open(term));
term.addEventListener('focus', () => open(term));
term.addEventListener('mouseleave', close);
term.addEventListener('blur', close);
term.addEventListener('click', event => {
event.stopPropagation();
if (pointerWasOpen ?? active === term) close(); else open(term);
pointerWasOpen = undefined;
});
});
document.addEventListener('keydown', e => { if(e.key === 'Escape') close(); });
document.addEventListener('click', close);
addEventListener('scroll', () => {
// Keyboard focus can scroll a term into view after its focus event.
if (active && document.activeElement === active) {
const rect = active.getBoundingClientRect();
if (rect.bottom > 0 && rect.top < innerHeight) open(active);
else close();
} else close();
}, {passive:true});
}
if (document.readyState === 'loading')
document.addEventListener('DOMContentLoaded', start); else start();
})();
Loading
Loading