Skip to content

Fix first-run setup loop on kits that pin a non-SCPH1001 BIOS - #27

Open
Alexbeav wants to merge 10 commits into
codex/wave3-release-cue-20260906from
fix/setup-host-bios-deadlock
Open

Fix first-run setup loop on kits that pin a non-SCPH1001 BIOS#27
Alexbeav wants to merge 10 commits into
codex/wave3-release-cue-20260906from
fix/setup-host-bios-deadlock

Conversation

@Alexbeav

@Alexbeav Alexbeav commented Sep 7, 2026

Copy link
Copy Markdown
Owner

Two users reported batch-3 titles (Wipeout XL SCUS-94351, Threads of Fate
SLUS-01019) looping forever in first-run setup. Same root cause, and it
affects all 26 wave-3 titles.

Root cause

bios_backends_missing() probed two hardcoded filenames:

psxrecomp/generated/OpenBIOS_dispatch.c
psxrecomp/generated/SCPH1001_dispatch.c

The CLI honours recompiler.bios_config and emits <stem>_*.c. Every wave-3
kit pins SCPH5552 (game.toml bios_config, CMakeLists.txt
PSXRECOMP_BIOS_STEMS, and the shipped README all agree), so those two files
never exist. The probe was permanently unsatisfiable: Generate succeeded every
time, the wizard reopened every time.

A second, independent bug made the Play path reject a correct BIOS on a setup
host: resolve_bios_for_runtime() validated an explicit --bios / bios.cfg
pick before the psx_bios_registry_count == 0 guard. With an empty registry
bios_backend_for_file() matches nothing, and a title with openbios = false
(109 of 156 ports) has no fallback. The launcher-side check already had this
guard in the right place.

Changes

  • runtime: check setup host before validating a chosen BIOS — pure move of
    the existing guard above the explicit-choice branch; guard body unchanged.
  • host: detect any recompiled BIOS stem — accept any stem with both
    <stem>_dispatch.c and <stem>_full.c, the same pairing runtime.cmake
    requires before linking a backend.
  • host: give the first-run loop breaker a cause-specific explanation — it
    appended "the project's boot-EXE names disagree" on every branch, which is
    wrong when only the BIOS backends are missing, and sent both reporters after
    a naming problem that did not exist. Also stops naming a specific BIOS image,
    since ports pin different ones.
  • runtime: point the recomp-ui preflight at submodule init — the error told
    the reader to git submodule add -b master https://github.com/mstan/recomp-ui.git,
    which on a released kit that already pins a fork commit either fails or wires
    up the wrong upstream at the wrong revision.

Verification

Structural: brace and paren balance unchanged, CRLF preserved, no stray NULs,
and the relocated BIOS guard is byte-identical to its previous form.

Functional: the new dispatch_has_full() / generated_has_bios_backend()
helpers were extracted and compiled standalone on both the Win32 and POSIX
branches with -Wall -Wextra, and exercised over three cases — both halves
present (found), dispatch only (missing), directory absent (missing). The
rewritten loop-breaker text was compiled and rendered for all three branch
combinations.

Not yet done: a real kit build. A framework-only checkout has no
recomp-ui, so this needs a full build against a kit before shipping.

Follow-ups, not in this PR

Other hardcoded-SCPH1001 assumptions remain: BIOS discovery filenames and
adoption paths in host/psxrecomp_codegen_host.c, and
runtime/src/main.cpp treats crc != 0x37157331u as "the validated dump" in
the launcher BIOS row, which warns misleadingly on a correct SCPH-5552.


Summary by cubic

Fixes the first-run setup deadlock on all 26 wave-3 kits, which pin SCPH5552 instead of SCPH1001, and adds headless setup verification. BIOS detection, discovery, seeding, and player-facing copy now follow the pinned stem via psx_bios_known_images.h instead of assuming SCPH1001, and the setup probe accepts any stem with both <stem>_dispatch.c and <stem>_full.c. A second bug made the Play path reject a correct BIOS on setup hosts, deadlocking titles with openbios = false (109 of 156 ports); the setup-host guard now runs before explicit BIOS validation.

Changes

  • The loop-breaker note now distinguishes "game dispatch missing" from "BIOS backends missing" and points at the port's README.
  • The recomp-ui preflight now leads with git clone --recurse-submodules / git submodule update --init and warns that GitHub source ZIPs never contain submodules.
  • The retcomm-rbengine preflight now recommends git submodule update --init --recursive and names the GitHub source-ZIP trap, since the old path only worked from inside psxrecomp/.
  • The known-image table covers SCPH1001 and SCPH5552; a stem absent from it degrades safely to asking the player.
  • The default BIOS path, size/CRC checks, and launcher copy all follow the pinned stem.
  • The codegen host gains --setup-selfcheck, which prints a JSON verdict of setup completeness and exits 0 (done) or 2 (wizard would reopen) so CI can assert it.
  • The nine output-capturing subprocess.run calls in psxrecomp_cli.py now decode as UTF-8 with errors="replace" instead of the ANSI codepage, which lost captured output on non-UTF-8 Windows locales.
  • Adds a regression test that compiles the codegen host, checks BIOS stem detection across four cases, and is registered in CMake so it actually runs.

Verification includes the regression test plus structural checks; no full kit build was run because a framework-only checkout lacks recomp-ui.

Written for commit 66f432e. Summary will update on new commits.

Review in cubic

resolve_bios_for_runtime() validated an explicit --bios / bios.cfg pick
before it noticed it was running on a first-run setup host, where zero
BIOS backends are linked. With an empty registry bios_backend_for_file()
matches nothing, so every image was rejected -- including the correct one
-- and a title with openbios = false had no fallback and exited. The
player could never supply the BIOS that Generate needs, so first-run
setup deadlocked. 109 of 156 ports set openbios = false.

The launcher-side check already had this guard in the right place; the
Play path did not. Pure move, guard body unchanged.
bios_backends_missing() probed two hardcoded filenames,
psxrecomp/generated/OpenBIOS_dispatch.c and SCPH1001_dispatch.c. A port
that pins a different image via PSXRECOMP_BIOS_STEMS / game.toml
recompiler.bios_config emits its backend under that stem instead, so the
probe could never be satisfied: every Generate succeeded, the wizard
reopened, and first-run setup looped forever. Every wave-3 kit pins
SCPH5552, so this affected all of them.

Accept any stem that has both <stem>_dispatch.c and <stem>_full.c, the
same pairing runtime.cmake requires before it will link a backend.
host_loop_breaker_note() appended "the project's boot-EXE names
disagree" whichever branch fired. That conclusion only holds when the
game dispatch is missing; for missing BIOS backends it is wrong, and it
sent two players (and the maintainer) after a naming problem that did
not exist.

Pick the explanation per branch, and do not name a specific BIOS image
in it -- ports pin different ones, so point at the README instead.
The missing-recomp-ui error told the reader to run "git submodule add -b
master https://github.com/mstan/recomp-ui.git". For anyone building a
released kit that already declares recomp-ui and pins a fork commit,
that either fails outright or wires up the wrong upstream at the wrong
revision.

Lead with clone --recurse-submodules / submodule update --init, name the
GitHub source-ZIP trap that produces this state, and keep submodule add
only for a repo that does not declare recomp-ui yet.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 3 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="host/psxrecomp_codegen_host.c">

<violation number="1" location="host/psxrecomp_codegen_host.c:1198">
P1: When `generated/` contains a paired game output or stale non-configured BIOS, this scan treats it as a BIOS backend. The setup host can skip BIOS generation and rebuild with no linked backend; restrict the scan to configured, descriptor-bearing BIOS stems.</violation>

<violation number="2" location="host/psxrecomp_codegen_host.c:3817">
P2: When both `game_missing` and `bios_missing` are true, this branch chooses only the boot-EXE mismatch message. Add a combined branch so the wizard also gives the required BIOS remediation instead of sending the player only to the port maintainer.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

if (!d)
return 0;
while (!found && (e = readdir(d)) != NULL) {
if (dispatch_has_full(dir, e->d_name))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: When generated/ contains a paired game output or stale non-configured BIOS, this scan treats it as a BIOS backend. The setup host can skip BIOS generation and rebuild with no linked backend; restrict the scan to configured, descriptor-bearing BIOS stems.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At host/psxrecomp_codegen_host.c, line 1198:

<comment>When `generated/` contains a paired game output or stale non-configured BIOS, this scan treats it as a BIOS backend. The setup host can skip BIOS generation and rebuild with no linked backend; restrict the scan to configured, descriptor-bearing BIOS stems.</comment>

<file context>
@@ -1140,15 +1140,74 @@ static int resolve_build_paths(void) {
+    if (!d)
+        return 0;
+    while (!found && (e = readdir(d)) != NULL) {
+        if (dispatch_has_full(dir, e->d_name))
+            found = 1;
+    }
</file context>

Comment on lines +3817 to 3826
if (game_missing)
cause = "please report this to the port maintainer: the project's "
"boot-EXE names disagree";
else
cause = "Generate produced the game code but no BIOS backend, which "
"normally means it had no retail BIOS to work from. Select "
"the PlayStation BIOS dump this port requires (named in the "
"README; it must be exactly 512 KB) in the launcher, then "
"run Generate again";
snprintf(g_loop_breaker_note, sizeof(g_loop_breaker_note),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: When both game_missing and bios_missing are true, this branch chooses only the boot-EXE mismatch message. Add a combined branch so the wizard also gives the required BIOS remediation instead of sending the player only to the port maintainer.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At host/psxrecomp_codegen_host.c, line 3817:

<comment>When both `game_missing` and `bios_missing` are true, this branch chooses only the boot-EXE mismatch message. Add a combined branch so the wizard also gives the required BIOS remediation instead of sending the player only to the port maintainer.</comment>

<file context>
@@ -3749,16 +3809,30 @@ static const char* host_loop_breaker_note(void) {
+     * BIOS backends do not: they mean Generate never emitted them, normally
+     * because no retail BIOS was available to emit them from. Blaming
+     * boot-EXE names for that sent players after the wrong thing. */
+    if (game_missing)
+        cause = "please report this to the port maintainer: the project's "
+                "boot-EXE names disagree";
</file context>
Suggested change
if (game_missing)
cause = "please report this to the port maintainer: the project's "
"boot-EXE names disagree";
else
cause = "Generate produced the game code but no BIOS backend, which "
"normally means it had no retail BIOS to work from. Select "
"the PlayStation BIOS dump this port requires (named in the "
"README; it must be exactly 512 KB) in the launcher, then "
"run Generate again";
snprintf(g_loop_breaker_note, sizeof(g_loop_breaker_note),
if (game_missing && bios_missing)
cause = "the generated game dispatch and BIOS backends are both "
"missing. Check the boot-EXE names, select the PlayStation "
"BIOS dump this port requires (named in the README; it must "
"be exactly 512 KB), then run Generate again";
else if (game_missing)
cause = "please report this to the port maintainer: the project's "
"boot-EXE names disagree";
else
cause = "Generate produced the game code but no BIOS backend, which "
"normally means it had no retail BIOS to work from. Select "
"the PlayStation BIOS dump this port requires (named in the "
"README; it must be exactly 512 KB) in the launcher, then "
"run Generate again";

test_cli_retail_bios_profile.py already covers recompiler.bios_config on
the psxrecomp_cli.py side. The C host half of the same feature had no
coverage, which is why bios_backends_missing() kept probing hardcoded
OpenBIOS/SCPH1001 filenames and shipped a first-run loop on all 26
wave-3 kits.

Compiles host/psxrecomp_codegen_host.c against a temporary project tree
and asserts sources_missing() over four cases: a pinned non-SCPH1001
stem, the bundled stem, a dispatch with no _full.c, and no backend at
all. Verified to fail on the pre-fix host and pass after.

Skips cleanly when recomp-ui or a C compiler is unavailable.
@Alexbeav

Alexbeav commented Sep 7, 2026

Copy link
Copy Markdown
Owner Author

Now verified end to end, not just structurally

Generated a real SCPH5552 backend from an actual BIOS dump
(psxrecomp-bios --config bios/SCPH5552.tomlSCPH5552_dispatch.c 627 KB +
SCPH5552_full.c 8.7 MB, descriptor present), assembled a project tree in the
shape the host expects, and compiled host/psxrecomp_codegen_host.c against
the pinned recomp-ui headers — the real translation unit, not an extract.

OLD (5952a72):  sources_missing = 1   -> setup wizard reopens, forever
NEW (this PR):  sources_missing = 0   -> setup completes

That is the reported bug reproduced against real generated artefacts, and the
fix closing it.

Added the missing regression test

runtime/tests/test_codegen_host_bios_stems.py compiles the host against a
temporary project tree and asserts sources_missing() over four cases: a
pinned non-SCPH1001 stem, the bundled stem, a dispatch with no _full.c, and
no backend at all. Confirmed it fails on the pre-fix host with

FAIL: a pinned non-SCPH1001 stem must satisfy the host
  artefacts=['SCPH5552_dispatch.c', 'SCPH5552_full.c'] expected=0 got='1'

and passes after. It skips cleanly without recomp-ui or a C compiler.

Why this shipped

test_cli_retail_bios_profile.py is a regression test for SCPH5552 on the
psxrecomp_cli.py side. When recompiler.bios_config was added, the CLI half
got the feature and a test; the C host half got neither. The two halves
disagreed about which files count as a BIOS backend, and nothing compared them.

Still outstanding: a full kit build/link and a launcher run.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 1 file (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="runtime/tests/test_codegen_host_bios_stems.py">

<violation number="1" location="runtime/tests/test_codegen_host_bios_stems.py:85">
P2: Compiling the host TU with strict `-std=c11` hides the POSIX functions it relies on (strcasecmp, opendir/readdir, realpath) behind glibc feature macros, so on glibc toolchains with GCC 14+ the probe fails to build and the test FAILs instead of running. Use `-std=gnu11` so the POSIX extensions the host needs stay declared.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

probe_c.write_text(PROBE, encoding="utf-8")
exe = tmp / ("probe.exe" if os.name == "nt" else "probe")
build = subprocess.run(
[cc, "-std=c11", "-o", str(exe), str(probe_c), str(HOST_C),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Compiling the host TU with strict -std=c11 hides the POSIX functions it relies on (strcasecmp, opendir/readdir, realpath) behind glibc feature macros, so on glibc toolchains with GCC 14+ the probe fails to build and the test FAILs instead of running. Use -std=gnu11 so the POSIX extensions the host needs stay declared.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At runtime/tests/test_codegen_host_bios_stems.py, line 85:

<comment>Compiling the host TU with strict `-std=c11` hides the POSIX functions it relies on (strcasecmp, opendir/readdir, realpath) behind glibc feature macros, so on glibc toolchains with GCC 14+ the probe fails to build and the test FAILs instead of running. Use `-std=gnu11` so the POSIX extensions the host needs stay declared.</comment>

<file context>
@@ -0,0 +1,126 @@
+        probe_c.write_text(PROBE, encoding="utf-8")
+        exe = tmp / ("probe.exe" if os.name == "nt" else "probe")
+        build = subprocess.run(
+            [cc, "-std=c11", "-o", str(exe), str(probe_c), str(HOST_C),
+             "-I", str(ROOT / "host"),
+             "-I", str(ui / "src"), "-I", str(ui / "src" / "common")],
</file context>
Suggested change
[cc, "-std=c11", "-o", str(exe), str(probe_c), str(HOST_C),
[cc, "-std=gnu11", "-o", str(exe), str(probe_c), str(HOST_C),

A built host asks its linked backends which image it accepts. A setup
host has none -- that is what makes it a setup host -- so first-run
discovery, bios.cfg seeding and every player-facing string hardcoded
SCPH-1001. On a kit pinning anything else (all 26 wave-3 kits pin
SCPH5552) that meant: a correct dump sitting beside the install was
never auto-discovered, seeding refused it, and the launcher told the
player their BIOS was wrong while quoting SCPH-1001's CRC at them.

Add psx_bios_known_images.h as the one place those identities live, and
have runtime.cmake pass the pinned stem in as PSX_EXPECTED_BIOS_STEM.
Both SCPH-1001 (0x37157331) and SCPH-5552 (0xD786F0B9) are verified
against the dumps and against the identity the recompiler emits into
generated/<stem>_dispatch.c.

Replaces the hardcodes in:
  - retail_bios_file_ok_c / discover_retail_bios_c (setup host)
  - retail_bios_file_ok / discover_retail_bios_near (runtime seeding)
  - the launcher BIOS row: size, CRC and "required" copy
  - the BIOS picker copy
  - PSXRT_DEFAULT_BIOS_PATH, which defaulted to bios/SCPH1001.BIN

A stem absent from the table degrades safely: nothing is auto-adopted,
the player is asked, and no message claims an identity it cannot back
up. SCPH-1001 builds are unaffected.
@Alexbeav

Alexbeav commented Sep 7, 2026

Copy link
Copy Markdown
Owner Author

Swept the remaining hardcoded SCPH-1001 assumptions

A built host asks its linked backends which image it accepts. A setup host has
none — so first-run discovery, bios.cfg seeding and every player-facing
string hardcoded SCPH-1001. On a kit pinning SCPH5552 that meant a correct dump
beside the install was never auto-discovered, seeding refused it, and the
launcher told the player their BIOS was wrong while quoting SCPH-1001's CRC.

New runtime/include/psx_bios_known_images.h holds those identities in one
place; runtime.cmake passes the pinned stem as PSX_EXPECTED_BIOS_STEM.
Both entries are verified twice — against the dumps, and against the identity
the recompiler emits into generated/<stem>_dispatch.c (SCPH5552_dispatch.c
records 524288u, 0xD786F0B9u, matching).

Replaced in: setup-host discovery/validation (retail_bios_file_ok_c,
discover_retail_bios_c), runtime seeding (retail_bios_file_ok,
discover_retail_bios_near), the launcher BIOS row (size, CRC, "required"
copy), the picker copy, and PSXRT_DEFAULT_BIOS_PATH.

Behaviour, from running the edited regions at three pinned stems:

pinned SCPH5552:  5552 dump -> "SCPH5552.BIN (CRC OK)."
                  1001 dump -> "this build expects SCPH-5552, CRC32 D786F0B9."
pinned SCPH1001:  unchanged from today
pinned SCPH101:   "this build pins SCPH101, whose identity is not
                   recorded here."  (nothing auto-adopted; player asked)

An unpinned or unknown stem degrades safely rather than asserting an identity
it cannot back up.

Verification

The header compiles clean under -Wall -Wextra as both C11 and C++20. The
codegen host compiles clean at both stems. The edited main.cpp regions were
extracted into a harness and compiled and run at all three stems (output
above). runtime/tests/test_codegen_host_bios_stems.py still passes.

main.cpp itself still has not been compiled whole — SDL is CMake-fetched, so
that needs the real configure. The full kit build remains the outstanding gate
for this PR.

Added the file without an add_test() entry, which is exactly the failure
mode runtime/check_test_registration.cmake exists to catch -- and it did,
at configure time:

    Test file(s) present on disk but registered nowhere:
        runtime/tests/test_codegen_host_bios_stems.py

An unregistered test cannot run and cannot fail. Register it alongside
the other runtime source-invariant guards; it needs neither a generated
BIOS nor a built runtime, and skips cleanly without recomp-ui or a C
compiler.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3 issues found across 5 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="host/psxrecomp_codegen_host.c">

<violation number="1" location="host/psxrecomp_codegen_host.c:1326">
P2: On case-sensitive filesystems, this lookup skips common mixed-case spellings such as `SCPH5552.bin` because the filename helper emits only all-uppercase or all-lowercase forms. Generate all independent base and extension case combinations so valid dumps are still auto-discovered.</violation>
</file>

<file name="runtime/runtime.cmake">

<violation number="1" location="runtime/runtime.cmake:1310">
P2: When a reused build changes `PSXRECOMP_BIOS_STEMS`, this definition can retain the old cached stem and make setup-host BIOS discovery reject the newly pinned dump. Derive the expected stem and default path from the current `PSXRECOMP_BIOS_STEM_PRIMARY` (or refresh the cache) instead of using the stale cache entry.</violation>
</file>

<file name="runtime/src/main.cpp">

<violation number="1" location="runtime/src/main.cpp:2732">
P2: In the default `OpenBIOS;SCPH1001` build, `psx_expected_bios()` is null because the primary stem is the bundled `OpenBIOS`. This makes `discover_retail_bios_near()` skip all filename probes, so a `scph1001` session cannot find a nearby retail dump unless another path hint is supplied. Base discovery on the nonbundled linked BIOS stem (or otherwise preserve the retail filename set) instead of the bundled primary stem.</violation>
</file>

Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic

"SCPH1001.bin", "scph1001.BIN",
};
char names[8][32];
int nnames = psx_known_bios_filenames(psx_expected_bios(), names, 8);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: On case-sensitive filesystems, this lookup skips common mixed-case spellings such as SCPH5552.bin because the filename helper emits only all-uppercase or all-lowercase forms. Generate all independent base and extension case combinations so valid dumps are still auto-discovered.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At host/psxrecomp_codegen_host.c, line 1326:

<comment>On case-sensitive filesystems, this lookup skips common mixed-case spellings such as `SCPH5552.bin` because the filename helper emits only all-uppercase or all-lowercase forms. Generate all independent base and extension case combinations so valid dumps are still auto-discovered.</comment>

<file context>
@@ -1307,22 +1315,22 @@ static int retail_bios_file_ok_c(const char* path) {
-        "SCPH1001.bin", "scph1001.BIN",
-    };
+    char names[8][32];
+    int nnames = psx_known_bios_filenames(psx_expected_bios(), names, 8);
     static const char* subs[] = {
         "bios", "", "system", "firmware", "psxrecomp/bios", "psxrecomp-v4/bios",
</file context>

Comment thread runtime/runtime.cmake
# The retail stem this build pins. A setup host has no linked
# backend to ask, so this is how it knows which image to look for
# and name (psx_bios_known_images.h) instead of assuming SCPH-1001.
PSX_EXPECTED_BIOS_STEM="${PSXRECOMP_BIOS_STEM}"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: When a reused build changes PSXRECOMP_BIOS_STEMS, this definition can retain the old cached stem and make setup-host BIOS discovery reject the newly pinned dump. Derive the expected stem and default path from the current PSXRECOMP_BIOS_STEM_PRIMARY (or refresh the cache) instead of using the stale cache entry.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At runtime/runtime.cmake, line 1310:

<comment>When a reused build changes `PSXRECOMP_BIOS_STEMS`, this definition can retain the old cached stem and make setup-host BIOS discovery reject the newly pinned dump. Derive the expected stem and default path from the current `PSXRECOMP_BIOS_STEM_PRIMARY` (or refresh the cache) instead of using the stale cache entry.</comment>

<file context>
@@ -1301,6 +1304,10 @@ function(psxrecomp_add_runtime_target target)
+        # The retail stem this build pins. A setup host has no linked
+        # backend to ask, so this is how it knows which image to look for
+        # and name (psx_bios_known_images.h) instead of assuming SCPH-1001.
+        PSX_EXPECTED_BIOS_STEM="${PSXRECOMP_BIOS_STEM}"
         # Where the shipped redistributable image lives, relative to the exe.
         # This is what a player gets when they choose no BIOS.
</file context>

Comment thread runtime/src/main.cpp
};
char name_buf[8][32];
const int name_count =
psx_known_bios_filenames(psx_expected_bios(), name_buf, 8);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: In the default OpenBIOS;SCPH1001 build, psx_expected_bios() is null because the primary stem is the bundled OpenBIOS. This makes discover_retail_bios_near() skip all filename probes, so a scph1001 session cannot find a nearby retail dump unless another path hint is supplied. Base discovery on the nonbundled linked BIOS stem (or otherwise preserve the retail filename set) instead of the bundled primary stem.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At runtime/src/main.cpp, line 2732:

<comment>In the default `OpenBIOS;SCPH1001` build, `psx_expected_bios()` is null because the primary stem is the bundled `OpenBIOS`. This makes `discover_retail_bios_near()` skip all filename probes, so a `scph1001` session cannot find a nearby retail dump unless another path hint is supplied. Base discovery on the nonbundled linked BIOS stem (or otherwise preserve the retail filename set) instead of the bundled primary stem.</comment>

<file context>
@@ -2708,24 +2710,27 @@ static bool retail_bios_file_ok(const std::filesystem::path& path) {
-    };
+    char name_buf[8][32];
+    const int name_count =
+        psx_known_bios_filenames(psx_expected_bios(), name_buf, 8);
+    if (name_count <= 0) return {};
     static const char* kSubdirs[] = {
</file context>

subprocess.run(..., text=True) decodes child output with the machine's
preferred encoding. On a Windows install whose ANSI codepage is not
UTF-8 the emitter's output kills the reader thread:

  File "encodings/cp1253.py", line 23, in decode
  UnicodeDecodeError: 'charmap' codec can't decode byte 0x9c

Hit on a Greek-locale machine during a real generate. The generate
still completed, but the captured stdout/stderr for that step is lost,
so any diagnostic the CLI meant to surface silently vanishes -- and the
callers that parse proc.stdout see nothing.

Pin all nine output-capturing calls to utf-8 with errors="replace".
Disc selection, BIOS selection and generation were already scriptable
through psxrecomp_cli.py (generate --disc --bios, rebuild, verify-disc).
The one part with no headless entry point was the verdict: does the
setup host consider the generated sources complete? That was reachable
only by clicking through the wizard, so nothing in CI could assert it --
and a BIOS stem mismatch in exactly that check shipped a first-run loop
on 26 titles.

--setup-selfcheck prints that verdict as JSON and exits: 0 when setup is
complete, 2 when the wizard would reopen, 1 when it cannot tell.

Hooked into psxrecomp_codegen_host_forward_if_built(), which every title
already calls with argc/argv at the top of main(), so existing titles
gain the flag with no per-title change. Placed ahead of the
PSX_HAS_GAME_DISPATCH early return so product builds answer too.
@Alexbeav

Alexbeav commented Sep 7, 2026

Copy link
Copy Markdown
Owner Author

Full kit build done — the outstanding gate is closed

Built Wipeout XL end to end from the staged kit with this branch's framework:
generate (headless, real disc + real SCPH-5552 dump) then rebuild.

[249/249] Linking CXX executable Wipeout_XL_Recompiled.exe
Rebuild complete

main.cpp now compiles and links whole, with every change in this PR. The
derived-stem plumbing is verifiable in the produced binary:

build.ninja:  PSX_EXPECTED_BIOS_STEM=\"SCPH5552\"
build.ninja:  PSX_DEFAULT_BIOS_PATH=\"bios/SCPH5552.BIN\"   (was SCPH1001.BIN)
exe strings:  "CRC32 %08X (this build expects %s, CRC32 %08X)."

Three more defects found while doing it

  1. My new test was registered nowhere. check_test_registration.cmake
    caught it at configure time — exactly the guard's purpose. Fixed in
    8594fb2ef; it is now in the runtime source-invariant list.
  2. subprocess.run(..., text=True) decodes with the ANSI codepage. On this
    Greek-locale machine a real generate threw
    UnicodeDecodeError: 'charmap' codec can't decode byte 0x9c in the reader
    thread. Generation still finished, but that step's captured output — and any
    diagnostic in it — is silently lost. All nine capture sites pinned to utf-8.
  3. Not a product bug, for the record: --gen-marker takes a basename, and
    the host correctly passes one. My first invocation passed a project-relative
    path and got a spurious "boot-exe name disagrees" error. Operator error.

Answering the automation question

Disc selection, BIOS selection and generation were already headless —
psxrecomp_cli.py generate --disc --bios, rebuild, verify-disc. The GUI
just shells out to them; that is how this whole build was driven.

What had no headless entry point was the verdict: does the setup host think
the generated sources are complete? That was reachable only by clicking through
the wizard — which is why a stem mismatch in that exact check shipped a
first-run loop on 26 titles with nothing to catch it.

--setup-selfcheck (commit 5ed1c948d) prints it and exits 0 / 2 / 1:

$ ./build-release/Wipeout_XL_Recompiled.exe --setup-selfcheck
{
  "display_name": "Wipeout XL",
  "project_root": "I:\...\build-kit",
  "expected_bios_stem": "SCPH5552",
  "expected_bios_id": "SCPH-5552",
  "expected_bios_crc32": "0xD786F0B9",
  "game_dispatch": "generated/SCUS_943.51_dispatch.c",
  "game_dispatch_present": true,
  "bios_backends_present": true,
  "sources_missing": false
}
exit=0

Negative test — hide SCPH5552_full.c and it flips:

"bios_backends_present": false
"sources_missing": true
exit=2

It hooks psxrecomp_codegen_host_forward_if_built(), which every title already
calls with argc/argv at the top of main(), so existing titles get the flag
with no per-title change, and it sits ahead of the PSX_HAS_GAME_DISPATCH
early return so product builds answer too.

A CI canary can now assert first-run completion in one line.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 3 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="recompiler/CMakeLists.txt">

<violation number="1" location="recompiler/CMakeLists.txt:630">
P3: This test is registered in the "runtime source-invariant guards" loop, which the enclosing comment guarantees only reads runtime source and "runs from a plain recompiler build" with no generated BIOS and no built runtime. `test_codegen_host_bios_stems.py` does not meet that contract: it returns 0 (reports PASS) as a SKIP whenever recomp-ui is absent or no C compiler is on PATH, and it compiles and executes `psxrecomp_codegen_host.c` rather than reading source. In a framework-only recompiler build recomp-ui is never present, so this test always skips and ctest reports it green with zero coverage, giving false confidence that the loop-breaker regression is guarded in the very tree the comment says it runs from. The host half only gets real coverage in full kit builds. Make the skip visible (e.g. return 77 so ctest treats it as SKIPPED) or relocate the registration so the loop's documented guarantee stays accurate.</violation>
</file>

<file name="host/psxrecomp_codegen_host.c">

<violation number="1" location="host/psxrecomp_codegen_host.c:4366">
P2: When a project path or display name contains a control character, `--setup-selfcheck` emits invalid JSON and headless CI parsers fail. Escape bytes below `0x20` (`\n`, `\r`, `\t`, `\b`, `\f`, or `\u00XX`) before writing them.</violation>
</file>

Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic

for (; s && *s; ++s) {
if (*s == '\\' || *s == '"')
putchar('\\');
putchar(*s);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: When a project path or display name contains a control character, --setup-selfcheck emits invalid JSON and headless CI parsers fail. Escape bytes below 0x20 (\n, \r, \t, \b, \f, or \u00XX) before writing them.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At host/psxrecomp_codegen_host.c, line 4366:

<comment>When a project path or display name contains a control character, `--setup-selfcheck` emits invalid JSON and headless CI parsers fail. Escape bytes below `0x20` (`\n`, `\r`, `\t`, `\b`, `\f`, or `\u00XX`) before writing them.</comment>

<file context>
@@ -4344,9 +4344,92 @@ static int host_paths_same_file(const char* a, const char* b) {
+    for (; s && *s; ++s) {
+        if (*s == '\\' || *s == '"')
+            putchar('\\');
+        putchar(*s);
+    }
+    putchar('"');
</file context>
Suggested change
putchar(*s);
if ((unsigned char)*s < 0x20) {
switch (*s) {
case '\b': fputs("\\b", stdout); break;
case '\f': fputs("\\f", stdout); break;
case '\n': fputs("\\n", stdout); break;
case '\r': fputs("\\r", stdout); break;
case '\t': fputs("\\t", stdout); break;
default: fprintf(stdout, "\\u%04X", (unsigned char)*s); break;
}
} else {
putchar(*s);
}

Comment thread recompiler/CMakeLists.txt
bios_selection_guards
capture_history
cli_retail_bios_profile
codegen_host_bios_stems

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: This test is registered in the "runtime source-invariant guards" loop, which the enclosing comment guarantees only reads runtime source and "runs from a plain recompiler build" with no generated BIOS and no built runtime. test_codegen_host_bios_stems.py does not meet that contract: it returns 0 (reports PASS) as a SKIP whenever recomp-ui is absent or no C compiler is on PATH, and it compiles and executes psxrecomp_codegen_host.c rather than reading source. In a framework-only recompiler build recomp-ui is never present, so this test always skips and ctest reports it green with zero coverage, giving false confidence that the loop-breaker regression is guarded in the very tree the comment says it runs from. The host half only gets real coverage in full kit builds. Make the skip visible (e.g. return 77 so ctest treats it as SKIPPED) or relocate the registration so the loop's documented guarantee stays accurate.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At recompiler/CMakeLists.txt, line 630:

<comment>This test is registered in the "runtime source-invariant guards" loop, which the enclosing comment guarantees only reads runtime source and "runs from a plain recompiler build" with no generated BIOS and no built runtime. `test_codegen_host_bios_stems.py` does not meet that contract: it returns 0 (reports PASS) as a SKIP whenever recomp-ui is absent or no C compiler is on PATH, and it compiles and executes `psxrecomp_codegen_host.c` rather than reading source. In a framework-only recompiler build recomp-ui is never present, so this test always skips and ctest reports it green with zero coverage, giving false confidence that the loop-breaker regression is guarded in the very tree the comment says it runs from. The host half only gets real coverage in full kit builds. Make the skip visible (e.g. return 77 so ctest treats it as SKIPPED) or relocate the registration so the loop's documented guarantee stays accurate.</comment>

<file context>
@@ -627,6 +627,7 @@ if(BUILD_TESTING)
                 bios_selection_guards
                 capture_history
                 cli_retail_bios_profile
+                codegen_host_bios_stems
                 default_renderer_guards
                 mod_owned_display_controls
</file context>

The message said:

    git submodule update --init lib/retcomm-rbengine

That only works from inside psxrecomp/. Someone standing at the game
repo root -- which is where they cloned, and where the error is read --
gets "no submodule mapping found", because from there the path is
psxrecomp/lib/retcomm-rbengine.

retcomm-rbengine is a submodule of psxrecomp, not of the game, so a
non-recursive init at the game root populates psxrecomp/ and leaves
lib/* empty, which is exactly the state that trips this check. Name
--recursive, and name the GitHub source-ZIP trap that also produces it.
@Alexbeav

Alexbeav commented Sep 8, 2026

Copy link
Copy Markdown
Owner Author

Correction: the published kits were never broken

I said earlier that "the staged kits ship submodules empty". That was wrong
about the published repos, and I should not have generalised from my own local
copy. Verified properly this time:

  • Alexbeav/wipeout-xl-recomp is public, carries .gitmodules, and pins
    psxrecomp 5952a72 + recomp-ui be8ac1d.

  • psxrecomp@5952a72 carries its own .gitmodules pinning
    lib/recomp-net 268e74f + lib/retcomm-rbengine ebd94a4.

  • All four commits still exist and are reachable upstream; both
    TechnicallyComputers repos are public.

  • A real git clone --recurse-submodules --shallow-submodules --depth 1 of the
    published repo checks out all four, and that pristine tree configures
    cleanly
    as a setup host:

    -- psxrecomp: rewind snap_ring (.../psxrecomp/lib/retcomm-rbengine)
    -- psxrecomp: setup host (no game C, no BIOS backends)
    -- Configuring done
    

So a user who clones recursively is fine. Nothing to fix in the repos.

What actually was broken: the staging checkouts

stage-wave3-titles-20260906.py:200 assembles each kit with

git clone --no-hardlinks <framework-public> <dest>/psxrecomp

— a plain clone, no recursion, and no submodule update afterwards. So every
staged working copy ended with psxrecomp/lib/* empty. The published repo is
unaffected because only the gitlink is committed, which is exactly why this
stayed invisible: it only bites whoever builds from the staging tree, which
is what a clean-room canary would do.

Fixed both ways:

  • Populated all 26 staged kits (git submodule update --init --recursive);
    0 uninitialised submodules remain, all at the recorded pins.
  • Patched the staging script to recurse after the detach checkout, so the next
    wave does not reproduce it. (That script is a dated one-off in _scratch, so
    this only helps if wave 4 is copied from it — worth promoting into the
    reusable pipeline tooling.)

threads-of-fate-recomp — one of the two reported titles — now configures
straight from the staging checkout, where it previously hard-failed on
PSX_REWIND=ON with no rbengine.

One real papercut fixed in this branch (66f432e3c)

The rbengine preflight told the reader to run

git submodule update --init lib/retcomm-rbengine

which only works from inside psxrecomp/. From the game repo root — where you
cloned, and where you read the error — that path does not exist, and a
non-recursive init at the root populates psxrecomp/ while leaving lib/*
empty, which is the very state that trips the check. It now names --recursive
and the source-ZIP trap, matching the recomp-ui message.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 1 file (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="runtime/runtime.cmake">

<violation number="1" location="runtime/runtime.cmake:538">
P3: The new guidance in this block is correct (rbengine is a submodule of psxrecomp, not of the game, so non-recursive init leaves it empty), but the identical scenario elsewhere in the same file still shows the old advice: the PSX_NETPLAY FATAL_ERROR right above still suggests `git submodule update --init lib/retcomm-rbengine`, run non-recursively from the game root. Per this new message that command cannot work. Update that sibling block to match (recommend `git submodule update --init --recursive`) so the two diagnostics for the same missing-rbengine failure don't contradict each other.</violation>
</file>

Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic

Comment thread runtime/runtime.cmake
"In an existing clone, run this from the GAME repo root. Note "
"--recursive: rbengine is a submodule of psxrecomp, not of the "
"game, so a non-recursive init leaves it empty.\n"
" git submodule update --init --recursive\n"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: The new guidance in this block is correct (rbengine is a submodule of psxrecomp, not of the game, so non-recursive init leaves it empty), but the identical scenario elsewhere in the same file still shows the old advice: the PSX_NETPLAY FATAL_ERROR right above still suggests git submodule update --init lib/retcomm-rbengine, run non-recursively from the game root. Per this new message that command cannot work. Update that sibling block to match (recommend git submodule update --init --recursive) so the two diagnostics for the same missing-rbengine failure don't contradict each other.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At runtime/runtime.cmake, line 538:

<comment>The new guidance in this block is correct (rbengine is a submodule of psxrecomp, not of the game, so non-recursive init leaves it empty), but the identical scenario elsewhere in the same file still shows the old advice: the PSX_NETPLAY FATAL_ERROR right above still suggests `git submodule update --init lib/retcomm-rbengine`, run non-recursively from the game root. Per this new message that command cannot work. Update that sibling block to match (recommend `git submodule update --init --recursive`) so the two diagnostics for the same missing-rbengine failure don't contradict each other.</comment>

<file context>
@@ -529,7 +529,13 @@ if(PSX_REWIND)
+            "In an existing clone, run this from the GAME repo root. Note "
+            "--recursive: rbengine is a submodule of psxrecomp, not of the "
+            "game, so a non-recursive init leaves it empty.\n"
+            "  git submodule update --init --recursive\n"
             "  or -DRECOMP_RBENGINE_ROOT=/path/to/retcomm-rbengine\n"
             "  or configure with -DPSX_REWIND=OFF to hide Rewind.")
</file context>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant