Repository navigation
fix(plugin): filter non-object JSONL lines in Store.read_lines and accounting.summarize - #200
Conversation
…repos store.read_lines can return non-dict JSON values (null, arrays, strings) if a ledger file was corrupted or written incorrectly. Code calling .get() on these records crashes with AttributeError. Added isinstance(r, dict) checks before accessing record fields in: - accounting.summarize (filters at entry) - repos._infer_repo_from_state (inline check) - repos._merge_state (filters ledger lists, inline check for events) Same pattern as the distiller fix in microsoft#199.
Muhammad Abdullah Butt (1AbdullahButt)
left a comment
There was a problem hiding this comment.
I checked every caller of Store.read_lines at the PR head (git grep "read_lines("). There are six call sites. cli.py:215 passes its records to accounting.summarize, which this PR now filters. The PR also guards repos.dir_label and the three reads in repos._merge_host. history._session_records already had the check. So no caller is left unguarded. In _merge_host, the filtered d_ledger is what _write_lines writes back, so a merge also drops the corrupt lines from the destination ledger. That seems like the right outcome.
Two small things:
- One guard in
Store.read_lineswould replace all four. If line 112 ofstore.pyonly appends a decoded line whenisinstance(rec, dict), every current and future caller is covered. Thelimitslice would then count only real records; today a corrupt line near the end takes one of thelimitslots. The per-caller checks could then go. - The description names
_infer_repo_from_stateand_merge_state, but the changed functions aredir_labeland_merge_host. Also, the PR adds no test, and the checked boxes don't say what was run. A test that writes a ledger with anullline and callssummarizewould pin it.
Reviewed with help from Claude Code.
A non-object JSONL line (null, a list, a string) came back as a record, so a trailing null with limit=1 hid the last valid record and callers crashed on it. Filter to dict records before the limit slice, so every reader gets dicts and the inline checks in repos.py are not needed. accounting.summarize keeps its own filter for direct lists.
|
Thanks Muhammad Abdullah Butt (@1AbdullahButt), good call moving the filter into |
Summary
Store.read_linescould return non-dict JSON values (null, arrays, strings) if a JSONL log had a malformed line, which crashed callers calling.get()and took slots fromlimit.isinstance(rec, dict)insideStore.read_linesso all callers receive object records andlimitcounts valid records only.isinstance(r, dict)at entry ofaccounting.summarizefor direct list callers.tests/test_store_read_lines.py.Test plan
pytest tests/test_store_read_lines.pytests/suite passes