Skip to content

ci: the ruff gate reported success while printing its own errors - #30

Merged
EricAndrechek merged 1 commit into
mainfrom
ruff-gate-swallowed
Sep 14, 2026
Merged

EricAndrechek merged 1 commit into
mainfrom
ruff-gate-swallowed

Conversation

@EricAndrechek

Copy link
Copy Markdown
Member

A required check has been passing vacuously. The python job's lint step was:

run: |
  cd python && uv sync … && uv run ruff check src tests && uv run ruff format --check src tests
  uv run python -c 'import chtypes; …'

bash -e does not exit when a command inside an && list fails — only the command following the final && is subject to it. So ruff check failed, ruff format never ran, the list returned non-zero, the next line ran anyway, and the step's status became that of the last command, which passes.

Measured, not reasoned

The job logs for a0a236c, f92e68e, 92c3360 and 2ff98c1 all report python = success while containing E501 Line too long lines. Eleven of them, then fifteen once ruff 0.16.7 added four F541 findings. The gate printed its own failures into its own log on every run and reported green.

Proved the mechanism both directions:

bash -e 'true && false && echo unreached
         echo last'                          -> exit 0   swallowed
bash -e 'true
         false
         echo unreached
         echo last'                          -> exit 1   fails correctly

Why the lint fixes are in the same commit

Repairing the gate alone would turn a required check red. So the fifteen findings it should have caught are fixed here:

  • 4 × F541 (f-string without placeholders) — auto-fixed
  • 11 × E501 — wrapped by splitting string literals at word boundaries, concatenated text unchanged
  • then ruff format

ruff check and ruff format --check both exit 0; the python suite passes 168 tests.

Not mine, and I checked

None of the fifteen came from my earlier commits. The four files involved were last touched by #15, #19 and the 0.1.2 release — and the two lines citing docs/guides/fetch.md already read that path before #22, so my path sweep didn't lengthen them.

Audited for the same shape elsewhere

Every multi-line run: block across all five workflows. Two matched the pattern and are safe: release-python.yml and release-rust.yml's version-agreement steps put the && inside a $( ) substitution, and the following line compares the result and exits 1 — so an empty value still fails. No other instance.

🤖 Generated with Claude Code

A REQUIRED check has been passing vacuously. The `python` job's lint step was

    cd python && uv sync … && uv run ruff check src tests && uv run ruff format --check src tests
    uv run python -c 'import chtypes; …'

and `bash -e` does not exit when a command inside an `&&` list fails — only
the command following the final `&&` is subject to it. So `ruff check` failed,
`ruff format` never ran, the list returned non-zero, the next line ran anyway,
and the step's status became that of the LAST command, which passes.

Measured, not reasoned: the job logs for a0a236c, f92e68e, 92c3360 and 2ff98c1
all report `python` = success while containing `E501 Line too long` lines —
eleven of them, then fifteen once ruff 0.16.7 added four F541 findings. The
gate printed its failures into its own log on every run and reported green.

Proved both directions before and after:

    bash -e 'true && false && echo unreached\necho last'   -> exit 0  (swallowed)
    bash -e 'true\nfalse\necho unreached\necho last'        -> exit 1  (fails)

The fix is one command per line. Fixing the gate alone would turn the required
check red, so the fifteen findings it should have caught are fixed in the same
commit: four F541 auto-fixed, eleven E501 wrapped by splitting string literals
at word boundaries with the concatenated text unchanged, then `ruff format`.
The python suite still passes, 168 tests.

None of the fifteen were introduced by my earlier commits — checked: the four
files involved were last touched by #15, #19 and the release, and the two
lines citing docs/guides/fetch.md already read that path before #22.

Also audited every other multi-line `run:` block across all five workflows for
the same shape. Two matched the pattern and are safe: release-python.yml's and
release-rust.yml's version-agreement steps put the `&&` inside a `$( )`
substitution, and the following line compares the result and exits 1, so an
empty value still fails.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@EricAndrechek
EricAndrechek merged commit 4a0f29e into main Sep 14, 2026
14 checks passed
@EricAndrechek
EricAndrechek deleted the ruff-gate-swallowed branch September 14, 2026 06:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant