Skip to content

FEAT: Add Garak latent injection scenario #2530

Description

Is your feature request related to a problem? Please describe.

PyRIT does not have a scenario for garak's latent-injection probes. These probes test indirect instructions that are embedded in carrier content such as translation text, reports, resumes, fact snippets, and WHOIS data. This is part of the scanner parity work in #511.

A direct port of garak's probe subclasses would not fit PyRIT. It would mix source data, prompt construction, execution, and scoring in the scenario.

Describe the solution you'd like

Add a PyRIT-native latent-injection scenario. Use the structure in PromptInject PR #2509 as the main implementation example. Copy its separation of responsibilities, not garak's runtime mechanisms.

Required scope:

  • Cover the active/lightweight upstream families in latentinjection.py: English-to-French translation, English-to-Chinese translation, report, resume, Eiffel fact snippets, legal fact snippets, WHOIS, WHOIS snippets, and latent jailbreak.
  • Store carrier contexts and injection instruction templates in separate local PyRIT datasets. Preserve expected trigger text, context family, language, and source provenance as seed metadata.
  • Use a small dataset configuration, similar to FEAT: Adding Garak prompt injection scenario #2509, to build AttackSeedGroup values from the datasets.
  • Use the standard PromptSendingAttack. Create bounded atomic attacks per expected trigger and use existing text-matching scorers such as SubStringScorer for exact-trigger variants.
  • Give latent jailbreak a separate path that uses an appropriate existing harm scorer. It does not have the same exact-trigger success contract.
  • Use scenario techniques only for the way that the injection is applied. Keep carrier family and language as dataset metadata or filters; do not make one technique for each garak class.
  • Cap the dataset cross-product, use deterministic selection, and use stable atomic-attack names so resume works.
  • Add exports, focused unit tests, and synchronized .py and .ipynb scanner documentation.

Follow doc/code/framework.md and the applicable files in .github/instructions. Datasets own source content and metadata; techniques describe application; the scenario assembles the campaign; attacks execute it; scorers evaluate responses. Do not port garak's Probe, mixin, hook, detector, or generator structure.

Describe alternatives you've considered, if relevant

  • Porting each garak subclass as a PyRIT technique was considered. This would make content categories control execution and would break PyRIT component responsibilities.
  • Building one large hard-coded prompt list in the scenario was considered. This would prevent dataset reuse and would repeat the design problems that FEAT: Adding Garak prompt injection scenario #2509 avoids.

Additional context

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions