Skip to content

python: expose rebuild_mesh/set_sun/set_materials/set_curvature; tiling script - #16

Open
thannart wants to merge 13 commits into
dkogan:masterfrom
thannart:tiled-panorama-rendering
Open

thannart wants to merge 13 commits into
dkogan:masterfrom
thannart:tiled-panorama-rendering

Conversation

@thannart

Copy link
Copy Markdown

Extends horizonator-pywrap.c so the Python bindings can drive everything render-panorama-tiles.py (new, below) needs:

  • The constructor gains restrict_mesh_azimuth/mesh_az_deg0/mesh_az_deg1, passed straight through to horizonator_init().
  • New rebuild_mesh(az_deg0, az_deg1) method, wrapping horizonator_rebuild_mesh() (see fix: real GL leak in horizonator_deinit(), plus horizonator_rebuild_mesh() #15): lets a caller change the meshed wedge on an already-constructed object, without recreating it.
  • New set_curvature()/set_sun()/set_materials() methods, wrapping the corresponding C API calls — previously only reachable from the standalone.c CLI, not from Python at all.

render-panorama-tiles.py (new, offline companion tool, not run by horizonator itself — same spirit as query-peaks-from-osm.py and build-landcover-tiles.py): renders a wide panorama as a sequence of narrow, memory-bounded tiles, then stitches them side by side. One horizonator object for the whole job (one DEM/land-cover load, one shader compile), rebuild_mesh() between tiles to change the wedge — the same "render a strip at a time" approach udeuschle.de's panoramas appear to use, but keeping everything in one process and one GL context (see #15 for why that matters: recreating the context, as a subprocess-per-tile approach would, leaks memory inside the Mesa/llvmpipe driver).

Verified: rendered a 3-tile (10deg each) --materials --shading --curvature panorama and a 4-tile --SRTM1 panorama, both stitching with no visible seam at the tile boundaries (checked pixel-for-pixel around one boundary) and no unusual memory growth.

While testing --SRTM1 against pre-baked SRTM3 land-cover tiles (a plausible real setup: a user switching resolutions across sessions), found landcover.c treated a wrong-sized tile file as a hard failure, aborting horizonator_init() entirely. Fixed to treat it the same as a missing tile instead (log a warning, fall back to the procedural classification for that area): a mismatched cache file is recoverable, and shouldn't be able to break a render this way.

Stacked on #15 (not yet merged) — this diff includes #15's commits until that lands.

🤖 Generated with Claude Code

thannart and others added 13 commits September 3, 2026 21:48
Lets you account for the fact that the viewer isn't standing at ground
level (e.g. an apartment floor), by reusing horizonator_move() to
reposition the camera at ground_elevation + viewer_height after the
usual auto-selected ground elevation is computed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…correction

The renderer worked entirely in the tangent plane at the viewer, ignoring
Earth's curvature. This is a good approximation at short range, but at
the 100-150km distances relevant for viewing the Alps from Lyon, a
distant peak's apparent elevation angle can be off by more than a
kilometer of apparent height.

Add the standard geodetic "curvature and refraction" correction:
  drop = (1-k) * distance^2 / (2*Rearth)
subtracted from each vertex's apparent height, where k is the
atmospheric refraction coefficient (k=0.13, the Gaussian refraction
coefficient used by udeuschle.de, is the default).

New horizonator_set_curvature() API, and --curvature / --refraction-k
flags on standalone. Off by default, so existing behavior is unchanged
unless --curvature is explicitly requested; this lets the flat-plane and
curved renders be compared directly.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…n use

The renderer always built a mesh covering the full 360-degree circle of
loaded DEM data, even for a render that only ever looks at a narrow
azimuth wedge (e.g. a fixed 110-degree panorama). Most of that geometry
was never visible and cost triangle-generation/vertex-shader time every
frame for nothing.

horizonator_init() takes 3 new parameters: restrict_mesh_azimuth,
mesh_az_deg0, mesh_az_deg1. When set, only cells within that azimuth
range (plus a 5-degree margin, plus a small radius always meshed near
the viewer where per-cell azimuth changes too fast to test reliably)
are triangulated; the rest of the loaded square is skipped in the index
buffer. Vertex generation, DEM sampling and the shaders are untouched.

standalone (a single fixed-wedge render per invocation) enables this by
default, since the meshed wedge always matches the rendered wedge
exactly -- verified pixel-identical output on 3 different az/height
combinations, 36-44% faster wall-clock render time. Pass
--no-restrict-mesh-azimuth to fall back to the old full-circle mesh.

The interactive horizonator tool and the Python bindings pass
restrict_mesh_azimuth=false, since they let the viewer pan around after
the data is loaded, potentially outside whatever wedge was meshed at
init time.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…s the render

horizonator_project() (used by annotator.c to predict where a named POI
should land on screen) computed the apparent elevation from a raw
lat/lon/ele difference, with no Earth-curvature/refraction correction.
When a render was made with horizonator_set_curvature() enabled, the
actual terrain in that render sits lower than this flat-plane
projection predicts -- by over a kilometer of apparent height at
100+km. annotator.c's occlusion/matching search (a small pixel-radius,
range-difference fuzz match) can't bridge that gap, so with
--curvature, no POIs were ever found: every one looked occluded.

horizonator_project() now takes curvature_enabled/refraction_k and
applies the same drop = (1-k)*d^2/(2*Rearth) subtracted from the
apparent height, matching vertex.glsl. annotate() takes and forwards
the same two values, which must match whatever was passed to
horizonator_set_curvature() for the render being annotated.
horizonator_unproject() needs no equivalent change: it derives lat/lon
from the already-rendered (already curvature-corrected, if enabled)
range image, not from an independently-computed real-world height.

Also fixes query-peaks-from-osm.py, which the Overpass API was
rejecting (HTTP 406) or timing out on: switch to https, add an
identifying User-Agent and an explicit query timeout.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…100km

MAX_MARKER_DIST was a #define fixed at 100000.0, independent of the
render's own --zfar. Any POI farther than 100km was silently never
labelled, even if it was actually rendered and clearly visible (e.g.
Mont Blanc, 158km from Lyon, with a --zfar of 150km+).

annotate() now takes max_marker_dist_m explicitly; standalone passes
zfar, since there's no point labelling something farther than what was
actually rendered.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Switches the default look from the original blue-background/red-distance
color-coding to a udeuschle.de-style drawn panorama: a white background,
near terrain in dark gray fading to light gray/white far away, and the
crest of each visible ridge/mountain-range layer outlined in a dark gray
line.

- horizonator-lib.c: glClearColor() to white instead of blue
- vertex.glsl: rgb is now a grayscale value (GRAY_NEAR..GRAY_FAR by
  distance) on all 3 channels, instead of red-only
- standalone.c: new draw_ridge_outlines(), a CPU-side post-process on the
  already-available range image. Darkens a pixel to --ridge-line-gray
  wherever it sits at a depth discontinuity of more than
  --ridge-line-threshold meters from the pixel above it (terrain against
  the sky, or one surface abruptly replaced by a much nearer/farther one
  behind a gap). Deliberately uses an absolute meters threshold, not a
  fraction of distance: at the bottom of the frame (grazing, near-horizontal
  sightlines) a fixed threshold naturally produces more, finer ridgelets
  than near the horizon, matching what a real drawn panorama looks like up
  close. --no-ridge-lines disables this.

Also rewrites standalone --help and the corresponding README section into
a single, organized reference for every rendering option and how it
affects the output (field of view, curvature/refraction, color/ridge
lines, peak labels, viewer position, performance, data sources).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The peak-visibility-matching logic (project each POI, then scan the
rendered range image to confirm it isn't occluded) was inlined in
annotate(), entangled with the PDF/SVG drawing code -- there was no way
to get just the list of visible peaks without generating a full
annotated render.

Extracts that logic into find_visible_pois(), a new function in
annotator.c/.h that annotate() now calls internally (no behavior
change, no more duplicated code). standalone gets a new
--list-visible-peaks OUT.txt option, independent of --image, that
writes the visible peaks (name, lat, lon, ele_m, range_m) as a plain
tab-separated file.

Also fixes a latent NULL-dereference: strlen(filename_image) was called
unconditionally, before checking filename_image for NULL. This was
never reached with a NULL filename_image before (every prior path
through this code already required --image), but --list-visible-peaks
without --image hits it directly.

cluster-visible-peaks.py is a companion script: reads that output,
drops peaks with no letters in their name (OSM-nameless fallback
entries, e.g. "1583.0"), and groups the rest into geographic clusters
("massifs" -- OSM doesn't reliably tag this for this area, so these are
unnamed, distance-ordered groups, not real massif names).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds directional slope shading: terrain is darkened/lightened based on
how its surface faces relative to a light direction, giving the relief
a 3D appearance beyond the existing pure distance-based grayscale.

Normals are estimated per-vertex, on the CPU, by finite differences
over the 4 DEM neighbors of each grid point (horizonator-lib.c, in the
existing vertex-buffer-filling loop -- no extra DEM reads, the samples
are already being fetched there). They're uploaded as a second vertex
attribute (a new float VBO; the existing position VBO stays untouched,
compact GLshorts) and interpolated by the rasterizer across each
triangle. This means shading is continuous across the edge between two
triangles sharing a vertex, unlike a flat per-triangle normal, which
would show a visible facet at every triangle boundary.

Lighting itself (dot(normal,sun_dir), with a 0.5 ambient floor so a
slope facing away from the sun dims rather than goes black) happens in
fragment.glsl, using the interpolated per-fragment normal;
geometry.glsl just passes normals through unchanged, one per vertex,
alongside the existing rgb/tex passthrough.

horizonator_set_sun(ctx, shading_enabled, sun_az_deg, sun_el_deg) is a
new public function, following the same on/off + uniforms pattern as
horizonator_set_curvature(): shading is off by default (unchanged
legacy rendering) via a shading_scale uniform (0.0/1.0), same trick as
curvature_scale. standalone gets matching --shading/--sun-azimuth/
--sun-elevation flags.

Memory cost: +12 bytes/vertex (a float3 normal) for the *entire* loaded
grid (Nvertices = (2*radius_cells)^2, independent of the azimuth-wedge
mesh restriction, which only shrinks the triangle/index buffer, not the
vertex buffer) -- e.g. +~250MB at a 150km zfar. This is a one-time
allocation per render (standalone is single-shot: one init, one
render, exit), not a repeated one, so it doesn't compound the way the
known horizonator_deinit() leak does across repeated init/deinit
cycles in one process. Tested at the 150km/4000x550 reference crop
with no crash and stable memory.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The distance-based color gradient (vertex.glsl) went from a dark gray
near the viewer to a light gray far away, converging on the white
background. Replaces GRAY_NEAR/GRAY_FAR with COLOR_NEAR/COLOR_FAR: near
terrain stays the same dark neutral gray, but far terrain now fades
towards a pale blue-gray instead of a neutral light gray, the way haze
and atmospheric scattering tint distant relief blue in a real photo.
Combines naturally with the slope-shading multiply in fragment.glsl
(unchanged there).

Updates the standalone --help COLOR section and the README to describe
the new blue-tinted far color instead of the old "light gray".

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds --materials: a first, purely procedural land-cover approximation
(no aerial imagery or real land-cover data) classifying each point by
elevation (snow line) and slope steepness (bare rock), with forest
below the tree line and alpine grass above it otherwise. Off by
default (materials_scale uniform, 0.0/1.0 pattern already used for
curvature and shading).

While testing --materials together with --shading, the near
(foreground) terrain came out much too dark: the legacy near-color
gray (a plain styling choice from the pre-materials grayscale look),
the material's own color, and the slope-shading dimming were all
multiplying together, crushing near, shaded forest to near black.

Restructured the near/far atmospheric blend to fix this: vertex.glsl
now only computes the scalar atmo_t (0 at znear_color, 1 at
zfar_color) and passes it through geometry.glsl to the fragment
shader, instead of computing a mixed color early. fragment.glsl picks
the near-color anchor itself: COLOR_NEAR_DEFAULT (the old gray) when
materials are off, or material_color() when they're on -- then blends
towards the far haze color COLOR_FAR with atmo_t, so a real land-cover
color is no longer also darkened by the legacy gray. The far-distance
haze blend is unchanged: it's genuinely distance-dependent atmospheric
scattering, so it applies the same regardless of materials.

Verified the materials=off/shading=off default path is unaffected: a
pixel diff against the pre-refactor render shows only 6 out of
6.6M bytes differ, by at most 1/255 -- floating-point reordering
noise, not a visual regression.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
--materials previously classified every point purely procedurally, from
fixed elevation/slope thresholds (a placeholder noted as such at the
time). This replaces that with real land-cover data where available,
falling back to the same procedural heuristic everywhere else -- never a
regression, only better where real data has been prepared.

New landcover.h/landcover.c module, deliberately mirroring dem.h/dem.c:
same 1-degree tile grid, same mmap-on-demand loading, same tolerance for
a missing tile file (reads back as LANDCOVER_UNKNOWN there, instead of a
warning-and-sea-level like dem.c: a missing area is the normal, expected
state until build-landcover-tiles.py has been run for it). The tile grid
geometry is taken directly from the already-initialized DEM context
rather than recomputed, so the two can never disagree.

Runtime wiring:
- horizonator_init() gains a dir_landcover parameter (default
  ~/.horizonator/landcover), threaded through every existing call site
  (standalone.c's two paths, horizonator.cc, horizonator-pywrap.c).
- A third per-vertex GL_UNSIGNED_BYTE attribute (location=2) carries the
  land-cover class, sampled alongside the existing normal computation in
  the VBO-fill loop.
- vertex.glsl/geometry.glsl pass it through to the fragment shader flat
  (unlike the smoothly-interpolated normal/elevation): it's a small
  integer code, and blending it across a triangle would produce
  meaningless values partway between two unrelated classes.
- fragment.glsl's material_color() now takes the class first, only
  falling back to the old procedural elevation/slope classification
  (renamed material_color_procedural(), logic unchanged) for
  LANDCOVER_UNKNOWN.
- standalone.c: new --dirlandcover flag, mirroring --dirdems.

build-landcover-tiles.py (new, offline companion tool, not run by
horizonator itself -- same spirit as query-peaks-from-osm.py): for every
1-degree tile a given viewer position/radius would touch, downloads the
covering ESA WorldCover 10m tile (public S3 bucket 'esa-worldcover', no
auth needed), resamples it (nearest-neighbor: never blend categorical
codes) onto that tile's exact DEM grid, and optionally overlays a
user-supplied CORINE Land Cover clip to flag permanent glaciers (CORINE
code 335) -- WorldCover alone can't reliably tell a glacier apart from
bare rock depending on the satellite pass. Requires rasterio
(python3-rasterio on Debian/Ubuntu).

Verified end-to-end: installed rasterio, ran build-landcover-tiles.py for
a 70km radius around the viewer (6 tiles, one shared WorldCover source
download), and rendered the reference crop with --materials --shading.
5.6% of output bytes changed vs. the all-fallback render (exactly the
area now covered by real data; the rest is untouched, confirming the
fallback path stays exact where no tile was prepared). Also confirmed
with an empty landcover directory (nothing downloaded) that the render is
visually identical to the pre-this-commit --materials output.

The CORINE glacier overlay itself wasn't exercised in this pass (no
CORINE clip was fetched for testing -- see build-landcover-tiles.py's
docstring for where to get one); it follows the exact same
reproject/override pattern already verified for WorldCover.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…esh()

The VAO/VBO/EBO/texture/shader-program/FBO handles horizonator_init()
creates were local variables, discarded as soon as it returned --
horizonator_deinit() had no way to free any of them, and relied entirely
on glutDestroyWindow() tearing down the whole GL context to implicitly
free everything created in it. Store the handles in the context struct
and explicitly glDelete* them (before destroying the window: that's what
makes the context, and so the handles, invalid).

Empirically this fixed only part of the memory growth from repeated
horizonator_init()/horizonator_deinit() cycles in one process (the
original motivation, from an earlier session, for adding a tile-by-tile
panorama render loop): looping just init+deinit with zero rendering still
grew RSS ~26-40MB per cycle. Comparing /proc/self/maps before/after
pointed at libLLVM/libgallium (llvmpipe's software-rendering shader
JIT) -- context creation itself leaks inside the Mesa driver, which this
project's code can't reach into and free.

The real fix for that is to not recreate the context at all: split mesh
construction (VAO/VBOs/EBO, previously inline in horizonator_init()) into
a new horizonator_rebuild_mesh(), callable again later on an
already-inited context to change the meshed azimuth wedge. A caller
wanting a wide panorama can now init once and loop
rebuild_mesh()+pan_zoom()+render_offscreen() per tile, on one context --
measured perfectly flat memory across 20 tiles in a row (vs. unbounded
growth from looping init/deinit). horizonator_init() itself now just
calls this function once, for its initial mesh.

While testing horizonator_rebuild_mesh() with a viewer position/radius
that happened to align exactly with a DEM tile's origin, found and fixed
a real latent bug in horizonator_dem_sample() (dem.c) and its
landcover.c twin: the "borrow the last row from the previous DEM tile"
adjustment can decrement a tile index to -1 when the queried cell is
exactly the first row/column of the whole loaded grid (no previous tile
exists to borrow from there), and the bounds check only caught the index
going too high, not negative -- an out-of-bounds array read, readily a
segfault. Both now also reject a negative index.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ng script

Extends horizonator-pywrap.c so the Python bindings can drive everything
render-panorama-tiles.py (new, below) needs:
- The constructor gains restrict_mesh_azimuth/mesh_az_deg0/mesh_az_deg1,
  passed straight through to horizonator_init().
- New rebuild_mesh(az_deg0, az_deg1) method, wrapping
  horizonator_rebuild_mesh() (see the previous commit): lets a caller
  change the meshed wedge on an already-constructed object, without
  recreating it.
- New set_curvature()/set_sun()/set_materials() methods, wrapping the
  corresponding C API calls -- previously only reachable from the
  standalone.c CLI, not from Python at all.

render-panorama-tiles.py (new, offline companion tool, not run by
horizonator itself -- same spirit as query-peaks-from-osm.py and
build-landcover-tiles.py): renders a wide panorama as a sequence of
narrow, memory-bounded tiles, then stitches them side by side. One
horizonator object for the whole job (one DEM/land-cover load, one
shader compile), rebuild_mesh() between tiles to change the wedge -- the
same "render a strip at a time" approach udeuschle.de's panoramas appear
to use, but keeping everything in one process and one GL context (see the
previous commit for why that matters: recreating the context, as a
subprocess-per-tile approach would, leaks memory inside the Mesa/llvmpipe
driver).

Verified: rendered a 3-tile (10deg each) --materials --shading --curvature
panorama and a 4-tile --SRTM1 panorama, both stitching with no visible
seam at the tile boundaries (checked pixel-for-pixel around one boundary)
and no unusual memory growth.

While testing --SRTM1 against pre-baked SRTM3 land-cover tiles (a
plausible real setup: a user switching resolutions across sessions),
found landcover.c treated a wrong-sized tile file as a hard failure,
aborting horizonator_init() entirely. Fixed to treat it the same as a
missing tile instead (log a warning, fall back to the procedural
classification for that area): a mismatched cache file is recoverable,
and shouldn't be able to break a render this way.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant