Repository navigation
ref: ik token readers accept data after an embedded NUL #258
Description
The unchanged read_token_file implementations in dump_ref, dump_mtp and dump_draft validate strtol with *end == '\0'. A record containing 1 followed by a NUL and garbage passes because strtol sees only the C-string prefix. The harness appends token 1 and sends it to decode without reporting the malformed record. Compare end with line.c_str() + line.size(), as ref_io.h does, and add an embedded-NUL rejection test. Preserve dump_ref's negative-count mode.
Evidence (at the base of #236):
Filed by Shingler, the review bot, while reviewing #236; the defect is outside that pull request.
Activity
Burntgogi commentedon Oct 15, 2026
Submitted PR #281. Whole-record lexical parsing now has one owner shared by the three ik readers and the existing mainline reader. Isolated compilation/execution of the actual private reader bodies observed the NUL acceptance before the fix and passes 29 file-I/O cases afterward, including dump_ref negative-count mode and the existing prefix/whitespace/sign behavior. The existing ref_io consumer regression kills a C-string-prefix mutant. Combined four-fix T0: 83 units, 0 red. Full linked ik oracles and inference/reference regeneration were not run; the PR and oracle docs explicitly require rebuilding the three harnesses and distinguish that missing evidence from the reader smoke.
juliankang4 commentedon Oct 16, 2026
#281의 수정을 이어받아 #334로 올렸습니다. Burntgogi님의 커밋은 그대로 두었고, 리뷰가 지적한 router trace reader도 같은
parse_token_id를 쓰도록 했습니다. #334가 merge되면 이 이슈가 닫힙니다.(AI 코딩 에이전트가 작성한 코멘트이며, 내용은 제가 확인했습니다.)