What's wrong
When DeltaTrack is handed a document it has deliberately decided it cannot compare accurately,
it explains why to a web user and crashes at a command-line user.
The engine detects one such case on purpose. Enrolled bills and public-law prints are typeset
without the printed line numbers in the left margin, and every anchor rule in the PDF parser
(the landmarks it uses to find where a section starts) depends on those numbers. Without them
the whole bill collapses into a single block and the comparison returns a confident wrong
answer. #141 (enrolled PDFs yield no anchors) is why the guard exists: rather than answer
wrongly, src/deltatrack/compare/pdf.py detects the layout and raises UnsupportedLayoutError
carrying a message written for the person who supplied the file:
This document has no printed line numbers, so DeltaTrack can't diff it accurately yet —
enrolled bills and public-law prints are typeset this way. Use the XML version of the bill
instead.
web/app.py catches it and returns that text verbatim as an HTTP 422, which is what the upload
page shows. ./diff_pdf.py catches nothing, so the same file produces a Python traceback whose
final line is that message, preceded by a stack of engine internals.
The same asymmetry applies to malformed input. Posting a file that is not XML to the endpoint
returns start_file: not XML (no leading '<') as an HTTP 415. Passing the same file to
./diff_bill.py compare raises ParseError: syntax error: line 1, column 0 uncaught.
How it surfaced
An audit of the command-line and web diff paths on develop at commit 8fb0fa16, running the
same inputs through both. The probe is docs/research/cli-ui-parity/probes/probe_flags.py on
branch worktree-audit-cli-ui-diff-parity, using an enrolled-bill fixture already committed
under tests/corpus/:
CLI: raises UnsupportedLayoutError UNCAUGHT -> traceback for the user
UI : HTTP 422 -> This document has no printed line numbers, so DeltaTrack can't diff it...
CLI: uncaught ParseError: syntax error: line 1, column 0
UI : HTTP 415 -> start_file: not XML (no leading '<').
Why it matters
The two cases are different in kind, and the distinction is the substance of this issue.
The unsupported-layout case is a designed outcome, not a failure. The tool examined the
document, reached a conclusion, and has a sentence prepared to explain it. Presenting a
designed refusal as a crash is the defect: it tells the user the tool broke when in fact the
tool worked, and it buries actionable guidance ("use the XML version") under a stack trace that
implies there is nothing to be done. An enrolled bill is a common, entirely valid file for
someone comparing appropriations bills to have on disk.
There is a second consequence for anyone scripting the tool. A traceback and a genuine crash
exit the same way, so a batch job comparing many bill pairs cannot distinguish "this document
was declined, move on" from "the tool is broken, stop". Today both are exit code 1.
The malformed-XML case is weaker. That is an unexpected input rather than a designed outcome,
and a traceback for a genuinely broken file is defensible on the command line, where the reader
is more likely to want the detail. It is included here because it is the same missing seam, not
because it carries the same argument.
This is low urgency. Nothing is wrong with the comparison itself, and a command-line user who
reads the last line of the traceback does get the right answer.
What to do
Two directions, and they can be taken separately.
-
Present the designed refusal as a message. Catch UnsupportedLayoutError in the main()
functions of src/deltatrack/diff_pdf.py and src/deltatrack/diff_bill.py, print
exc.message to stderr, and exit non-zero. The user-facing wording already exists and is
already reviewed; only its presentation on this surface is missing.
-
Give a declined document its own exit code, distinct from an unexpected failure, so a
caller can tell the two apart. This is the half that matters for scripted use and it is
worth deciding deliberately, since an exit code is a small interface commitment.
Whether to extend the same treatment to parse errors is a separate call. The argument for is
consistency with the endpoint; the argument against is that a traceback is more useful than a
one-line message when the input really is broken, and command-line users are better placed to
use it. Recommendation is to do 1 and 2, and leave parse errors alone unless someone reports
being confused by one.
Verification
The fix must be shown on a real declined document rather than a synthetic exception, because
the guard's detection is itself part of the path. tests/corpus/ carries committed enrolled-bill
PDFs, and tests/test_pdf_compare.py already exercises the guard through the endpoint
(test_enrolled_upload_returns_specific_message_not_generic_422); the command-line equivalent is
the missing half.
Worth confirming the check can fail: with the handler removed, the test should see a traceback
rather than the message.
Unverified
Whether any wrapper or scheduled job currently keys on the command's exit status. Introducing a
distinct exit code for a declined document is backwards-compatible for anything treating
non-zero as failure, but not for anything matching on the value 1 specifically. No such caller
was found in this repository; callers outside it were not checked.
Part of #691 (the epic making every surface produce the same answer). The refusal it concerns
was introduced by #141 (enrolled PDFs yield no anchors, so the layout is declined rather than
answered wrongly).
Refs #691, #141
What's wrong
When DeltaTrack is handed a document it has deliberately decided it cannot compare accurately,
it explains why to a web user and crashes at a command-line user.
The engine detects one such case on purpose. Enrolled bills and public-law prints are typeset
without the printed line numbers in the left margin, and every anchor rule in the PDF parser
(the landmarks it uses to find where a section starts) depends on those numbers. Without them
the whole bill collapses into a single block and the comparison returns a confident wrong
answer. #141 (enrolled PDFs yield no anchors) is why the guard exists: rather than answer
wrongly,
src/deltatrack/compare/pdf.pydetects the layout and raisesUnsupportedLayoutErrorcarrying a message written for the person who supplied the file:
web/app.pycatches it and returns that text verbatim as an HTTP 422, which is what the uploadpage shows.
./diff_pdf.pycatches nothing, so the same file produces a Python traceback whosefinal line is that message, preceded by a stack of engine internals.
The same asymmetry applies to malformed input. Posting a file that is not XML to the endpoint
returns
start_file: not XML (no leading '<')as an HTTP 415. Passing the same file to./diff_bill.py compareraisesParseError: syntax error: line 1, column 0uncaught.How it surfaced
An audit of the command-line and web diff paths on
developat commit8fb0fa16, running thesame inputs through both. The probe is
docs/research/cli-ui-parity/probes/probe_flags.pyonbranch
worktree-audit-cli-ui-diff-parity, using an enrolled-bill fixture already committedunder
tests/corpus/:Why it matters
The two cases are different in kind, and the distinction is the substance of this issue.
The unsupported-layout case is a designed outcome, not a failure. The tool examined the
document, reached a conclusion, and has a sentence prepared to explain it. Presenting a
designed refusal as a crash is the defect: it tells the user the tool broke when in fact the
tool worked, and it buries actionable guidance ("use the XML version") under a stack trace that
implies there is nothing to be done. An enrolled bill is a common, entirely valid file for
someone comparing appropriations bills to have on disk.
There is a second consequence for anyone scripting the tool. A traceback and a genuine crash
exit the same way, so a batch job comparing many bill pairs cannot distinguish "this document
was declined, move on" from "the tool is broken, stop". Today both are exit code 1.
The malformed-XML case is weaker. That is an unexpected input rather than a designed outcome,
and a traceback for a genuinely broken file is defensible on the command line, where the reader
is more likely to want the detail. It is included here because it is the same missing seam, not
because it carries the same argument.
This is low urgency. Nothing is wrong with the comparison itself, and a command-line user who
reads the last line of the traceback does get the right answer.
What to do
Two directions, and they can be taken separately.
Present the designed refusal as a message. Catch
UnsupportedLayoutErrorin themain()functions of
src/deltatrack/diff_pdf.pyandsrc/deltatrack/diff_bill.py, printexc.messageto stderr, and exit non-zero. The user-facing wording already exists and isalready reviewed; only its presentation on this surface is missing.
Give a declined document its own exit code, distinct from an unexpected failure, so a
caller can tell the two apart. This is the half that matters for scripted use and it is
worth deciding deliberately, since an exit code is a small interface commitment.
Whether to extend the same treatment to parse errors is a separate call. The argument for is
consistency with the endpoint; the argument against is that a traceback is more useful than a
one-line message when the input really is broken, and command-line users are better placed to
use it. Recommendation is to do 1 and 2, and leave parse errors alone unless someone reports
being confused by one.
Verification
The fix must be shown on a real declined document rather than a synthetic exception, because
the guard's detection is itself part of the path.
tests/corpus/carries committed enrolled-billPDFs, and
tests/test_pdf_compare.pyalready exercises the guard through the endpoint(
test_enrolled_upload_returns_specific_message_not_generic_422); the command-line equivalent isthe missing half.
Worth confirming the check can fail: with the handler removed, the test should see a traceback
rather than the message.
Unverified
Whether any wrapper or scheduled job currently keys on the command's exit status. Introducing a
distinct exit code for a declined document is backwards-compatible for anything treating
non-zero as failure, but not for anything matching on the value 1 specifically. No such caller
was found in this repository; callers outside it were not checked.
Part of #691 (the epic making every surface produce the same answer). The refusal it concerns
was introduced by #141 (enrolled PDFs yield no anchors, so the layout is declined rather than
answered wrongly).
Refs #691, #141