Decode STR opcodes via optional .tbl (closes TODO #1) - #1
Conversation
If both is_pc_version=False (default) and radio=None, neither branch sets
self.is_pc_version, causing AttributeError on the line:
if not self.is_pc_version:
while len( data ) % 4 != 0:
data.push_byte( 0 )
Default it to False before the conditional so direct callers (tests,
standalone compile) don't need to pass a radio object.
8060c89 to
6b04d57
Compare
|
Quick follow-up: amended the commit to also wire This effectively also covers the first half of TODO entry 3 (radio dialog scripts) for the binary→JSON direction. The text↔JSON lexer/parser side of TODO 3 remains untouched. |
|
End-to-end test on the pristine MGS1 USA disc 1 RADIO.DAT (no Arabic/translation overlay): Command: Result: 274 radio dialog JSON files written successfully. Sample TALK entry, before/after the .tbl: Without `--tbl`: With `--tbl` (radioChar entries from MGS1 JP): The `\x80\x23` and `\x80\x4e` 2-byte escapes resolve to `#` and `N` per the JP radioChar table — these are the newline/separator markers MGS1 carries over from the Japanese encoding. So the same `.tbl` correctly handles both the stage GCX strings and the radio dialog text in one pass. |
…perly')
Reading side (read_string):
Optional thingy-style .tbl with longest-prefix matching (4 -> 1 byte).
Multi-byte sequences such as MGS1 Japanese radio strings (0x80xx
radioChar / 0x81xx hiragana / 0x82xx katakana / kanji ...) decode
to readable characters. Without a .tbl, behavior is unchanged.
Writing side (encode_string):
Symmetric inverse-tbl lookup so a JSON STR that holds real Unicode
(e.g. "うご") round-trips back to the same bytes (0x81 0x06 0x81 0x14)
the .gcx originally had. Without a .tbl, behavior is unchanged but a
one-line warning is emitted when push_byte would silently truncate a
Unicode codepoint, so .tbl coverage gaps surface instead of corrupting
output silently.
Plumbing:
CLI flag '--tbl <path>' wired into both decompile and compile paths
in main.py (RADIO.DAT GcxData and per-stage GCX in decompile;
RadioComp.gcx and GclComp.gcx in compile). A class-level fallback on
GcxData lets the temporary sub-buffers built inside radio_compile /
gcl_compile use the same inverse table without each module having to
thread the .tbl through its API.
Smoke-tested (round-trip via .tbl):
GcxData(b'\x81\x06\x81\x14').read_string() -> 'うご' (decode)
GcxData().encode_string('うご') -> b'\x81\x06\x81\x14\x00' (encode)
Decompile of pristine MGS1 USA Disc 1 RADIO.DAT (--no-padding --tbl):
\\x80#\\x80N markers resolve to '#' / 'N' across 274 dialogs.
6b04d57 to
87adbd8
Compare
|
Extended the patch to cover the write direction as well. Before: After: when a Smoke test: The CLI plumbing in Side-finding (pre-existing, not in this PR): while testing |
This PR addresses the first item in the README's TODO list — Decode strings properly — and fixes a small bug encountered along the way.
Summary
.tblforread_stringingcx.py. When loaded, multi-byte sequences (radioChar0x80xx/0xC0xx, hiragana0x81xx, katakana0x82xx, punctuation0xd0xx, kanji0xXXYY, …) decode to readable characters via longest-prefix matching (4 → 3 → 2 → 1 byte). Without a.tblthe behavior is unchanged: bytes< 0x80decode viachr(), higher bytes still become\xNNescapes.--tbl <path>flag wired intomain.pysopython3 main.py -d <input> -o <output> --tbl mgs.tbldecodes Japanese MGS1 strings out-of-the-box.GclComp.__init__:self.is_pc_versionwas never set when bothis_pc_version=False(default) andradio=None, socompile_gcl_filecrashed withAttributeErroron the trailing alignment-padding check. Default it toFalsebefore the conditional.Why TBL?
The current
read_stringreturns\xNNfor any byte ≥ 0x80, which makes Japanese strings unreadable in the JSON output. A pluggable.tbl(the same format used by tools like Translhextion) solves this generically: a fan can supply mappings for any encoding (Japanese, Spanish accented chars, modified font tables, etc.) without touching the tool. This is the same mechanism used in many ROM-hacking workflows and is fully opt-in.Tested
--tbland an empty.tbl: identical output to currentmain.Notes
read_string's outer iteration limit (max_iter = 0xff) preserves the original safeguard.set_tbl(None)or omitting the flag disables decoding, so existing test fixtures continue to round-trip byte-identical.