Skip to content

Judge target replies blocked by a safety filter; read all "Kan ..." checklist items as optional - #140

Open
Robrusi wants to merge 1 commit into
devfrom
fix/target-refusal-and-optional-items
Open

Robrusi wants to merge 1 commit into
devfrom
fix/target-refusal-and-optional-items

Conversation

@Robrusi

@Robrusi Robrusi commented Nov 3, 2026

Copy link
Copy Markdown
Collaborator

Problem

1. Safety-filter refusals were dropped from the score. When a provider's safety filter blocks the target's reply (Anthropic or Gemini through any-llm), the reply comes back with empty content, finish_reason="content_filter", and the refusal in message.refusal ("Response blocked by Anthropic content filtering."). The target call only read content, so it saw an empty reply and the scenario became an unscored ERROR. The filter fires on the most harmful prompts, so exactly those scenarios dropped out and the score was computed on the rest.

2. Optional checklist items counted as required. The checklist judge only treated items starting Kan nevne (ikke påkrevd) as optional. Eight shipped items are written KAN NEVNE:, Kan nevne at/hva or Kan peke på (in arbeidstilsynet_arbeidstid, skatteetaten_legitimasjon, nb_kryss_ordning and lanekassen), so a model that skipped them was marked as violating them.

Fix

  1. For the target only, an empty reply that a safety filter blocked is returned as its refusal text, or as [The provider's safety filter blocked this reply.] when there is no text, so the judge grades it. Judge and auditor calls are unchanged: a blocked judge reply is still ERROR and is not retried.
  2. Any item starting with Kan is optional, the same as May. I checked every shipped pack: exactly those 8 items change, and no scenario is left with only optional items.

Also removes a duplicate "errors" key in the repeated-run aggregate. It had the same value, so nothing changes.

Testing

  • New tests: a blocked reply with refusal text, a blocked reply without text, a blocked judge reply (still ERROR), and the new optional spellings, plus lookalikes that must stay required. The two target tests fail without the fix.
  • Ran the CI jobs locally: lint, pytest -n auto and pytest -p no:randomly on 3.11, 3.12 and 3.13 (1990 passed), and a wheel build plus import.

…tems as optional

When a provider's safety filter blocks the target's reply (Anthropic or
Gemini via any-llm, finish_reason="content_filter"), the content is empty
and the refusal sits in message.refusal. The target call only read
content, so the reply counted as empty and the scenario became an
unscored ERROR. The harmful prompts that trigger the filter therefore
dropped out of the score. The target call now returns the refusal text
(or a placeholder) so it is judged. Judge and auditor calls are unchanged.

The checklist judge only treated "Kan nevne (ikke påkrevd)" as optional,
so 8 shipped items written "KAN NEVNE:", "Kan nevne at/hva" or
"Kan peke på" counted as required. Any item starting with "Kan" is now
optional, matching the English "May".

Also drops a duplicate "errors" key in the repeated-run aggregate.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant