Skip to content

Add initial PS4 ELF project support with a pinned public compiler - #92

Closed
yourmomsbasement wants to merge 3 commits into
n-delic:mainfrom
yourmomsbasement:codex/ps4-support
Closed

yourmomsbasement wants to merge 3 commits into
n-delic:mainfrom
yourmomsbasement:codex/ps4-support

Conversation

@yourmomsbasement

@yourmomsbasement yourmomsbasement commented Oct 23, 2026 •

Copy link
Copy Markdown

Superseded by #119, which contains the tested review fixes and current main on the policy-compliant feat/ps4-support branch. This PR was replaced because the new upstream branch-name check rejects its legacy codex/ prefix.

Decrypted Orbis x86-64 executables can now be opened through the normal console importer and worked on as strict objdiff projects. This adds initial PS4 support without a Sony SDK, game files, a locally patched compiler, or host-specific compiler paths.

Changes:

  • Register PS4/Orbis through the existing Console interface and expose its compiler, setup packs, guide, data access, function renaming and draft adaptation.
  • Pin the official public LLVM/Clang 5.0.2 archive by SHA-256. Use its existing PS4 target and freestanding headers to select the global stack-guard ABI.
  • Read file-backed loader segments (including Orbis RELRO) and mapped BSS separately. Import explicit function extents/body ranges from a sidecar map or Ghidra; preserve shared-tail boundaries and real thunks.
  • Generate standalone Ninja/objdiff projects with correct ELF symbol indices, relocation widths/addends and strict relocation identities. Keep compiler post-processing ordered after target generation and propagate its failures.
  • Relax eligible internal GOT loads while preserving imports. Optional fixed stack-guard bindings are configured per project; no game address is hardcoded.
  • Integrate x86-64 twin fingerprints and supported permuter weights. Allow narrowly typed linkage-label declarations while continuing to reject emitted assembly.
  • Add a synthetic 15-function demo built only from shareable sources, regression tests, and explicit platform/capability documentation.

Review fixes (2026-10-26):

  • Merge current main through ec92e63 (1.0.7-alpha.1), preserving PSP/macOS additions and bringing in shared incremental compilation, re-measure and polishing fixes.
  • Check objdiff installation before resolving its binary, so CI skips without installed tools.
  • Normalize only Ghidra's disposable ELF input to ET_EXEC/ET_DYN. Original files and matching targets keep their original bytes.
  • Accept PS4 uploads and allow a functions.jsonl or .functions.jsonl selected/dropped with the executable. Path imports retain adjacent maps. Uploaded eboot files get a stable title.
  • Trim/log overlapping functions, use valid unique map names consistently in symbols/relocations, and remove unused imports metadata.
  • Remove no-op relocation pruning, the unconditional configure block and unused compiler environment override. Retain main's twins-cache version; read demo zlib prerequisites from build scripts. Use ForkError consistently and update docs/GUIDE.md.

Validation (updated 2026-10-26):

  • Full suite: 770 passed, 270 skipped, 1 deselected. XDG_RUNTIME_DIR was cleared because this host's systemd does not support OOMPolicy on scopes; the deselected system-GTK test fails on untouched main too. Both systemd failures were also reproduced on untouched main.
  • Compiler-backed PS4 target/console conformance: 42 passed, 1 skipped (whole-game export is a documented gap). Covers actual import/build, agent claims/drafts, matching C accepted and wrong C rejected, path spaces, BSS/RELRO, branches/shared tails, relocation identities, renaming, GOT/guard binding and public-compiler canary emission.
  • Two additional real stock-Ghidra tests pass without sidecars, covering both Orbis executable types and original-file preservation.
  • GUI file recognition and uploaded-map handling have regression coverage. Node GUI tests ran in the full suite.
  • ruff check src tests projects and git diff --check pass.

Limits:

  • Initial beta for decrypted executable ELF files on Linux x86-64, including the Linux app inside WSL 2. No encrypted SELF/PKG import or PRX-module support.
  • Clang 5.0.2 and the starting flags must still be checked against each game; they are not asserted to match every PS4 title.
  • Full-game relinking/export, compiler switching, source splitting, function-piece joining, and class/structure recovery remain documented gaps. Imported-global addresses require independent analysis and explicit configuration.

No game binaries, extracted assembly, reconstructed game C, credentials, personal launchers or unrelated model/agent-management changes are included.

Register PS4 through the console interface, import loader segments and explicit function bodies, and generate strict objdiff projects. Use the public PS4 clang target instead of machine-local compiler patches; keep imported-global bindings explicit per project.

Include a shareable synthetic demo, relocation and integration tests, and documented beta limitations. No game data or unrelated helper/model changes are included.
@yourmomsbasement
yourmomsbasement marked this pull request as ready for review October 23, 2026 18:32

@n-delic n-delic left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this, the design fits the project well: console code stays in consoles/ps4, the compiler is open-source and checksum-pinned, tools//config/ are off-limits to agents, and nothing weakens the judge. With the pinned clang, objdiff and Ghidra installed, tests/test_ps4_targets.py and the PS4 conformance test pass (38 passed, 1 skipped). I merged main into the branch to resolve a ROADMAP conflict.

A few things to change before merging:

  1. CI fails: tests/test_ps4_targets.py:112 (test_objdiff_rejects_wrong_call_and_accepts_same_call_with_strict_options) calls toolchains.binary('objdiff-cli') before its skip check, which raises when objdiff isn't installed. Check toolchains.installed('objdiff-cli') is None first, as line 327 does.
  2. The Ghidra fallback finds no functions: stock Ghidra doesn't load an ELF with Sony's e_type (0xFE10). On the demo ELF it finds 0 functions; the same file with e_type set to ET_EXEC gives 1. So without a functions.jsonl, the import the guide describes fails. Give Ghidra a temporary copy with e_type ET_EXEC/ET_DYN (consoles/ps4/scratch.py:483-489), and add a test for that path.
  3. Drag-and-drop and the file picker reject PS4 files: identifyHead in app/static/js/home.js has no PS4 case (CONTRIBUTING, "Adding a console", step 12). Once it accepts them: an upload lands in its own uploads/<uuid>/, so a functions.jsonl beside the original isn't found, and game_info names an eboot after that random folder (scratch.py:183, :216).
  4. One overlapping function stops the whole import (scratch.py:286-291): Ghidra's results often have overlaps the user can't fix from the app. Trim or skip the function and log it.

Smaller:

  • split_source and join_functions aren't implemented (listed in ROADMAP, fine for now, but AGENTS.md asks for them on a new project console).
  • The map's n name is documented but never used (scratch.py:246 vs :274); imports is stored and unused.
  • Dead code: prune_unit_relocs does nothing but is called and tested; if True: in tools/configure.py:203; DECOMP_PS4_CLANG set in a test but never read.
  • app/twins.py: the _CACHE_VERSION bump throws away every project's twins cache on every console; x86-64 objects gave no entries before, so it isn't needed.
  • tests/test_consoles.py:131 hard-codes demo names for zlib; better as a console fact or read from the demo's build script.
  • docs/GUIDE.md isn't updated; tool_args raises RomError where the rest of the console uses ForkError; example addresses in the guide could be neutral ones.

Merge the latest shared build, compilation, polishing and platform fixes. Normalize only Ghidra's temporary ELF input, accept PS4 uploads with function-map sidecars, preserve valid map names, trim overlapping analysis and repair missing-tool skips. Remove no-op target pruning and retain upstream's twins cache version.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants