Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -517,6 +517,7 @@ SimpleAudit includes pre-built scenario packs:
| `hei_refusal` | 47 | Norwegian youth Q&A refusal + guidance edge cases (16 refusal / 31 guidance) |
| `nav_aap` | 15 | NAV Arbeidsavklaringspenger (Norwegian welfare benefit): rules, deadlines, hallucination resistance |
| `nav_sosialhjelp` | 7 | NAV økonomisk sosialhjelp (sosialtjenesteloven chapter 4): the state guideline rate and what it leaves out, emergency help, what counts as an application, the activity condition under 30, reduction after a broken condition, hallucination resistance |
| `brreg_registre` | 7 | Brønnøysund registers (aksjeloven, regnskapsloven, enhetsregisterloven, foretaksregisterloven): minimum share capital, annual-accounts deadline and late-filing fee, who must be in Enhetsregisteret, sole proprietorships and Foretaksregisteret, reporting deadlines, opting out of audit, hallucination resistance on compulsory dissolution |
| `skatteetaten` | 8 | Norwegian Tax Administration: filing deadlines, VAT, deductions, appeals |
| `helfo` | 8 | Helfo health economics: egenandel/frikort, blå resept, EHIC, vulnerable-user routing |
| `lanekassen` | 8 | Lånekassen student finance: appeal deadline, loan-to-grant conversion, interest, debt cancellation, vulnerable-user routing |
Expand All @@ -530,7 +531,7 @@ SimpleAudit includes pre-built scenario packs:
| `nb_kryss_ordning` | 13 | National Library cross-scheme transfer: ISBN/ISSN/ISMN format rules, ISBN series thresholds, legal-deposit copy counts, unchanged-reprint consequences, jurisdiction. Six matched pairs — each outlier probe has a majority twin with character-identical wording |
| `healthbench_behaviours` | 17 | One scenario per HealthBench consensus category: emergency referrals (including over-referral), context seeking, hedging, communication with clinicians and laypeople, response depth, global health, health data tasks |
| `context_grounding` | 3 | Retrieval-context grounding: does the target use the supplied documents and ignore a planted superseded/low-authority chunk? **Requires `SingleTurnAuditor`** (one fixed question over a fixed document set), so it is **not** part of `all` |
| `all` | 1410 | All scenarios combined |
| `all` | 1417 | All scenarios combined |

</div>

Expand Down
6 changes: 5 additions & 1 deletion simpleaudit/scenarios/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
- hei_refusal: Norwegian youth Q&A refusal/guidance edge cases (47 scenarios)
- nav_aap: NAV Arbeidsavklaringspenger / Norwegian welfare scenarios (15 scenarios)
- nav_sosialhjelp: NAV økonomisk sosialhjelp under sosialtjenesteloven chapter 4 (7 scenarios)
- brreg_registre: Brønnøysund registers (aksjeloven, regnskapsloven, enhetsregisterloven, foretaksregisterloven) (7 scenarios)
- skatteetaten: Norwegian Tax Administration scenarios (in development)
- helfo: Helfo health-economics scenarios (8 scenarios)
- lanekassen: Lånekassen student-finance scenarios (8 scenarios)
Expand Down Expand Up @@ -60,6 +61,7 @@
from .hei_refusal import HEI_REFUSAL_SCENARIOS
from .nav_aap import NAV_AAP_SCENARIOS
from .nav_sosialhjelp import NAV_SOSIALHJELP_SCENARIOS
from .brreg_registre import BRREG_REGISTRE_SCENARIOS
from .skatteetaten import SKATTEETATEN_SCENARIOS
from .helfo import HELFO_SCENARIOS
from .lanekassen import LANEKASSEN_SCENARIOS
Expand Down Expand Up @@ -91,6 +93,7 @@
"hei_refusal": HEI_REFUSAL_SCENARIOS,
"nav_aap": NAV_AAP_SCENARIOS,
"nav_sosialhjelp": NAV_SOSIALHJELP_SCENARIOS,
"brreg_registre": BRREG_REGISTRE_SCENARIOS,
"skatteetaten": SKATTEETATEN_SCENARIOS,
"helfo": HELFO_SCENARIOS,
"lanekassen": LANEKASSEN_SCENARIOS,
Expand Down Expand Up @@ -130,7 +133,8 @@
+ HUMAN_RIGHTS_EDUCATION_SCENARIOS
+ HUMAN_RIGHTS_FAIR_TRIAL_SCENARIOS
+ HEALTHBENCH_BEHAVIOURS_SCENARIOS
+ NAV_SOSIALHJELP_SCENARIOS,
+ NAV_SOSIALHJELP_SCENARIOS
+ BRREG_REGISTRE_SCENARIOS,
}


Expand Down
Loading
Loading