Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -517,6 +517,7 @@ SimpleAudit includes pre-built scenario packs:
| `hei_refusal` | 47 | Norwegian youth Q&A refusal + guidance edge cases (16 refusal / 31 guidance) |
| `nav_aap` | 15 | NAV Arbeidsavklaringspenger (Norwegian welfare benefit): rules, deadlines, hallucination resistance |
| `nav_sosialhjelp` | 7 | NAV økonomisk sosialhjelp (sosialtjenesteloven chapter 4): the state guideline rate and what it leaves out, emergency help, what counts as an application, the activity condition under 30, reduction after a broken condition, hallucination resistance |
| `digdir_digital_forvaltning` | 7 | Digital public administration (eForvaltningsforskriften, forskrift om universell utforming av IKT-løsninger, digitaliseringsrundskrivet): reservation against digital decisions, notice for digital decisions, when the complaint deadline starts, accessibility standard and statement for public websites, the circular's scope, hallucination resistance on the new forvaltningsloven |
| `skatteetaten` | 8 | Norwegian Tax Administration: filing deadlines, VAT, deductions, appeals |
| `helfo` | 8 | Helfo health economics: egenandel/frikort, blå resept, EHIC, vulnerable-user routing |
| `lanekassen` | 8 | Lånekassen student finance: appeal deadline, loan-to-grant conversion, interest, debt cancellation, vulnerable-user routing |
Expand All @@ -530,7 +531,7 @@ SimpleAudit includes pre-built scenario packs:
| `nb_kryss_ordning` | 13 | National Library cross-scheme transfer: ISBN/ISSN/ISMN format rules, ISBN series thresholds, legal-deposit copy counts, unchanged-reprint consequences, jurisdiction. Six matched pairs — each outlier probe has a majority twin with character-identical wording |
| `healthbench_behaviours` | 17 | One scenario per HealthBench consensus category: emergency referrals (including over-referral), context seeking, hedging, communication with clinicians and laypeople, response depth, global health, health data tasks |
| `context_grounding` | 3 | Retrieval-context grounding: does the target use the supplied documents and ignore a planted superseded/low-authority chunk? **Requires `SingleTurnAuditor`** (one fixed question over a fixed document set), so it is **not** part of `all` |
| `all` | 1410 | All scenarios combined |
| `all` | 1417 | All scenarios combined |

</div>

Expand Down
6 changes: 5 additions & 1 deletion simpleaudit/scenarios/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
- hei_refusal: Norwegian youth Q&A refusal/guidance edge cases (47 scenarios)
- nav_aap: NAV Arbeidsavklaringspenger / Norwegian welfare scenarios (15 scenarios)
- nav_sosialhjelp: NAV økonomisk sosialhjelp under sosialtjenesteloven chapter 4 (7 scenarios)
- digdir_digital_forvaltning: digital public administration and accessibility of public websites (7 scenarios)
- skatteetaten: Norwegian Tax Administration scenarios (in development)
- helfo: Helfo health-economics scenarios (8 scenarios)
- lanekassen: Lånekassen student-finance scenarios (8 scenarios)
Expand Down Expand Up @@ -60,6 +61,7 @@
from .hei_refusal import HEI_REFUSAL_SCENARIOS
from .nav_aap import NAV_AAP_SCENARIOS
from .nav_sosialhjelp import NAV_SOSIALHJELP_SCENARIOS
from .digdir_digital_forvaltning import DIGDIR_DIGITAL_FORVALTNING_SCENARIOS
from .skatteetaten import SKATTEETATEN_SCENARIOS
from .helfo import HELFO_SCENARIOS
from .lanekassen import LANEKASSEN_SCENARIOS
Expand Down Expand Up @@ -91,6 +93,7 @@
"hei_refusal": HEI_REFUSAL_SCENARIOS,
"nav_aap": NAV_AAP_SCENARIOS,
"nav_sosialhjelp": NAV_SOSIALHJELP_SCENARIOS,
"digdir_digital_forvaltning": DIGDIR_DIGITAL_FORVALTNING_SCENARIOS,
"skatteetaten": SKATTEETATEN_SCENARIOS,
"helfo": HELFO_SCENARIOS,
"lanekassen": LANEKASSEN_SCENARIOS,
Expand Down Expand Up @@ -130,7 +133,8 @@
+ HUMAN_RIGHTS_EDUCATION_SCENARIOS
+ HUMAN_RIGHTS_FAIR_TRIAL_SCENARIOS
+ HEALTHBENCH_BEHAVIOURS_SCENARIOS
+ NAV_SOSIALHJELP_SCENARIOS,
+ NAV_SOSIALHJELP_SCENARIOS
+ DIGDIR_DIGITAL_FORVALTNING_SCENARIOS,
}


Expand Down
Loading
Loading