Is your feature request related to a problem? Please describe.
PyRIT's existing PackageHallucination scenario does not expose Dart, Perl, or Raku. The existing scorer and registered garak package datasets already support these ecosystems, so users cannot select capabilities that PyRIT already contains. This is part of #511.
Describe the solution you'd like
Extend the existing PackageHallucination scenario. Do not add a second scenario.
Required changes:
- Add Dart, Perl, and Raku to the scenario technique enum and language specification map.
- Reuse the registered
garak_dart_packages, garak_perl_packages, and garak_raku_packages datasets.
- Reuse
PackageHallucinationScorer, which already supports these three ecosystems. Do not add a loader or scorer.
- Keep Rust as the bounded default unless maintainers choose another existing default.
- Load a package registry only when its technique is selected. Do not add large registries to implicit default loading.
- Increment the scenario version as required by the scenario conventions.
- Add focused tests for selection, dataset/scorer wiring, and the unchanged default. Update the synchronized Garak scanner documentation.
Follow doc/code/framework.md, the scenario instructions, the dataset instructions, and the applicable test and documentation instructions. Keep dataset selection in the scenario configuration and scoring in the scorer.
Describe alternatives you've considered, if relevant
- A new scenario, new registry loaders, or a new scorer would duplicate existing PyRIT components.
- Loading all registries by default would make a small parity change expensive and would change current behavior.
Additional context
Is your feature request related to a problem? Please describe.
PyRIT's existing
PackageHallucinationscenario does not expose Dart, Perl, or Raku. The existing scorer and registered garak package datasets already support these ecosystems, so users cannot select capabilities that PyRIT already contains. This is part of #511.Describe the solution you'd like
Extend the existing
PackageHallucinationscenario. Do not add a second scenario.Required changes:
garak_dart_packages,garak_perl_packages, andgarak_raku_packagesdatasets.PackageHallucinationScorer, which already supports these three ecosystems. Do not add a loader or scorer.Follow
doc/code/framework.md, the scenario instructions, the dataset instructions, and the applicable test and documentation instructions. Keep dataset selection in the scenario configuration and scoring in the scorer.Describe alternatives you've considered, if relevant
Additional context
garak/probes/packagehallucination.py