Skip to content

Level-load optimization, retail font parity, honest savegame compatibility, and soft particles by default - #147

Closed
themuffinator wants to merge 5 commits into
mainfrom
codex/idtech5-modernization-plan
Closed

Level-load optimization, retail font parity, honest savegame compatibility, and soft particles by default#147
themuffinator wants to merge 5 commits into
mainfrom
codex/idtech5-modernization-plan

Conversation

@themuffinator

Copy link
Copy Markdown
Owner

Five commits from one session on game/airdefense1: a load-time optimization round, two robustness fixes, a font-parity regression fix, a savegame-compatibility correction, and one defaults change.

What changed

Level-load recording was quadratic (1f756624)
The level-load cache matched every opened source against the whole recorded semantic-hint vector, building up to three temporary std::strings per comparison. game/airdefense1 records 4,428 hints, so recording scaled as sources x hints. Hints are now indexed by the three keys the match rules actually use, and a semantic name already resolved this generation is not resolved again.

With com_levelLoadModernization 1, warm medians of three runs: total load 11,893 -> 9,010 ms, level image loading 4,442 -> 2,209 ms, first visit 21,040 -> 18,044 ms. The shipping default path is unchanged at 11.3 s — this only speeds up the guarded cache experiment, which is still default off because it remains slower on a first visit.

Classic .proc parser hardening (same commit)
ParseModel did not range-check its file-provided vertex/index counts or the indexes it read, while ParseShadowModel and the binary render-world cache both did — and FinishSurfaces dereferences every index. Now bounded. Verified no stock surface is rejected across airdefense1, airdefense2, storage1, medlabs, mcc_landing, with .proc parse time unchanged. LoadLevelImages also bounds its fill loop against the array it pre-sized.

Retail font parity (2afc8ec7)
SetFontByScale chose its atlas from a viewport-enlarged scale so high-DPI displays stop magnifying the 12-point sheet. But the selected atlas is also what TextWidth, MaxCharHeight and the DrawText advances read, so above roughly 1.5x enlargement the hand-authored retail .fontdat atlases started reporting different metrics — the loading title measured 234 px against a retail 223 px, moving its right-aligned x by 11 px.

Scalable fonts rasterise all three slots from one face, so their normalised metrics agree and a larger slot only adds resolution; the enlargement now applies to those only. This took renderer_validation_matrix from 35/36 back to 36/36, and at the 2538x1312 window that exposed it: 15 metric mismatches before, zero after.

Savegame compatibility (1339033e, plus c4c45c7 in openQ4-game)
Loading every save on hand showed the gate admitting payloads it could not read; they failed a sync check roughly 550 KB in, after the running map had been unloaded — the outcome the policy exists to prevent.

Root cause: idPhysics_Player::swimSpeed entered the save on 2026-08-13 and idPlayer::nextLiquidSurfaceSoundTime on 2026-08-19, but one boolean gated both. Every save written between those builds read a field its file does not contain. Each field now carries the build that introduced it, which recovers builds 691 and 707.

What still cannot be read is no longer claimed. The v2 snapshot allowlist is removed rather than corrected — it asserted support never demonstrated against a real save, and all five tuples that could be tested were wrong. v2 payloads and v3 payloads below a verified build floor are refused during preflight, naming the build that wrote them, with the session left intact.

Soft particles on by default (6cf639e6)
Eligible effect sprites ended in a visible straight edge where they met a floor, wall, or crate. Measured at 2560x1334 through Vulkan with cooldowns: 429.1 Hz median with the fade on against 437.6 Hz off, ranges 313.9–457.1 and 381.2–443.4 — about 2%, inside the run-to-run spread.

For the reviewer

  • This needs the companion openQ4-game branch. 1339033e is one change split across repos: the engine refuses what it cannot read, openQ4-game (c4c45c7) does the per-field decoding. Reviewing the engine half alone will look incomplete. 2123e02 there also adds benchmarkViewSweep, the deterministic camera pan the frame-rate evidence below depends on.
  • Two behaviour changes worth deliberate attention. Saves from openQ4 0.10 and from 0.11-era v2 builds stop loading — deliberate, since none of them actually restored, but it is a withdrawn support claim, not a silent fix. And r_softParticles is a defaults change; it is archived, so existing configs keep their recorded value and only fresh ones pick it up. Both are in the v0.12.0 Upgrade Notes.
  • savegame_v3_contract now forbids reintroducing a tuple allowlist and pins the build floor plus the two per-field thresholds instead of the removed snapshot tables. If you want the allowlist back, that guard is the thing to argue with.

Validation

  • openq4_validate.py push passes.
  • renderer_validation_matrix 36/36 (was 35/36 before the font fix).
  • savegame_v3_contract and savegame_corruption_contract pass.
  • Refusal behaviour verified directly: anderson, airdefense2 and convoy2 refuse with zero map teardowns and zero errors; airdefense1, tram1b, process1_first still load.
  • Frame rate on the game/airdefense1 360-degree camera sweep, 1280x720 windowed, OpenGL, shipping defaults: 282.8–313.1 Hz across six passing captures, P95 6 ms / P99 7 ms, worst single frame 14 ms. docs/dev/airdefense1-optimization-evidence.md records the method, the load-time breakdown, and the ranked remaining work.

One measurement caveat recorded in the evidence: this host throttles hard under sustained load — back-to-back runs decay 303 -> 208 -> 183 Hz, and the CPU performance ratio swings 56–131% of nominal. All figures above were taken with cooldowns between runs.

themuffinator and others added 5 commits August 31, 2026 09:01
The level-load cache manager recorded every opened source by matching it
against the whole semantic-hint vector, and each comparison built up to three
temporary std::strings. game/airdefense1 records 4,428 hints, so recording a
level's media scaled as sources times hints. Hints are now indexed by the three
keys the match rules actually use - exact name, compiled-suffix name, and
extension-stripped stem - and lookup takes the highest matching hint index,
which is the hint the old reverse scan stopped on. A semantic name already
resolved in the current generation is also no longer resolved a second time,
because the repeat open could only yield the identity the first one learned.

On game/airdefense1 with com_levelLoadModernization 1, warm medians of three
runs: total load 11,893 -> 9,010 ms, level image loading 4,442 -> 2,209 ms, and
first-visit load 21,040 -> 18,044 ms. That path is now the fastest warm
configuration measured for this map, 20.1% below the shipping default, but it
is still slower on a first visit, so the default stays off. The classic default
path is unchanged at 11.3 s.

ParseModel now range-checks its file-provided vertex and index counts and every
index it reads. ParseShadowModel and the binary render-world cache already
applied those predicates; the classic text draw-surface path did not, and
FinishSurfaces dereferences every index while deriving tangents and silhouette
edges. Verified against game/airdefense1, game/airdefense2, game/storage1,
game/medlabs, and game/mcc_landing: no stock surface is rejected, and the
.proc parse time is unchanged.

LoadLevelImages also bounds its fill loop against the array it pre-sized from
CountPendingLevelLoads. The two share a predicate today so the bound cannot
trip; without it a future divergence would be a silent heap overflow rather
than a dropped image.

docs/dev/airdefense1-optimization-evidence.md records the camera-sweep capture,
the 360-degree frame-rate results, the load-time breakdown, and the ranked
remaining work.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
SetFontByScale chose its atlas from a viewport-enlarged scale so high-resolution
displays stop magnifying the 12-point sheet. Its comment said "text layout and
dimensions do not change", but the selected atlas is also what TextWidth,
MaxCharHeight and the DrawText advances read, so on a large window the retail
.fontdat atlases - drawn by hand at each size, and not agreeing to the pixel
once normalised - began reporting different metrics.

uiFontParitySelfTest caught it on a 2538x1312 window: the HUD radio marine cases
were a pixel out in line height, baseline, glyph y and overhang, the three radio
strings two pixels narrow, and the loading title measured 234 px against a
retail 223 px, moving its right-aligned x by 11 px. Below roughly 1.5x
enlargement nothing moved, which is why 1280x720 and 640x480 both passed.

A scalable font rasterises its small/medium/large slots from one face at 12/24/48
point, so their normalised metrics do agree and a larger slot only buys
resolution. The enlargement now applies to those fonts only, keyed on the "ttf/"
slot-name prefix that the scalable path assigns; the retail atlases select on the
authored scale and keep exact retail parity. Three self-test cases pin the
distinction, including a font that produced only some scalable slots.

renderer_validation_matrix is 36/36 again, from 35/36. Verified directly at the
2538x1312 window that failed: 15 metric mismatches before, zero after. The push
validation profile stays green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The note described the uiFontParitySelfTest failure as still needing an
owner. It has one now, so record the cause and the fix instead.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Loading every save on hand showed the compatibility gate admitting payloads it
could not read. They failed a sync check or a non-finite window variable roughly
550 KB in, which is after the running map has been unloaded - the outcome this
policy exists to prevent. Measured on the current build:

  v3 build >= 661        restore cleanly (691, 707, 762, 784, 794, 795)
  v3 build 1             the released v0.10 payload, desyncs mid-restore
  v2 builds 544/556/614  every tested snapshot desyncs mid-restore, 5 of 5
  unstamped legacy       already refused correctly

Builds 691 and 707 are recovered by the companion change that matches each
player liquid field to the build that added it. What remains cannot be read, so
it is no longer claimed: v3 payloads below a verified build floor and every v2
payload are refused during preflight, with a message naming the build that wrote
them, and the session stays intact.

The v2 allowlist is removed rather than corrected. It asserted support that had
never been demonstrated against a real save, and every tuple that could be
tested was wrong. savegame_v3_contract now forbids reintroducing a tuple
allowlist and pins the build floor and the two per-field thresholds instead of
the removed snapshot tables.

Verified: anderson, airdefense2, and convoy2 refuse with zero map teardowns and
zero errors; airdefense1, tram1b, and process1_first still load. push validation
and both savegame contracts pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Eligible effect sprites ended in a visible straight edge where they met a
floor, wall, or crate. The depth fade that removes it has been available as
r_softParticles since the soft-particle path landed, but off by default.

Measured on game/airdefense1, the BSE-smoke scene it matters most in, at
2560x1334 through the Vulkan backend with 45-second cooldowns between runs:
429.1 Hz median with the fade on against 437.6 Hz with it off, ranges 313.9-457.1
and 381.2-443.4. About 2%, inside the run-to-run spread on this host. A live
capture on the default OpenGL backend renders the plumes, fire, terrain, HUD and
view weapon correctly, with no broken effects or black quads.

Decals, beams, electricity, trails, model debris, custom GLSL stages,
post-process materials, and non-effect translucent materials are unaffected;
they were never part of this path. r_softParticleFadeDistance still controls the
fade width.

The cvar is archived, so an existing configuration keeps whatever value it
already recorded and only a fresh one picks the new default up. The release
notes say so.

push validation and renderer_validation_matrix 36/36 pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-01T18:35:11.025982Z 6cf639e PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@themuffinator
themuffinator deleted the codex/idtech5-modernization-plan branch September 1, 2026 18:35
@themuffinator
themuffinator restored the codex/idtech5-modernization-plan branch September 1, 2026 18:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant