Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 32 additions & 8 deletions 13-guardrails/13-00-guardrails.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,27 @@ codenames and competitor names. Built around `contoso-bank-agent` in the admin F
project, isolated to its own model deployment so other agents on the project are
unaffected.

> **Known service-side limitation (2026-05).** The custom blocklist is currently **not
> attached** to the RAI policy because doing so breaks the Responses API with HTTP 500 on
> any happy-path call (tested empirically: 5/5 fail with any `customBlocklists` entry, 5/5
> succeed with none). The same policy works through Chat Completions API. This is the
> service-side analogue of the Java SDK array-shape issue
> [#49196](https://github.com/Azure/azure-sdk-for-java/issues/49196).
> Practical impact on the demo: Prompt Shields (Jailbreak / Indirect Attack), standard
> safety filters, and Protected Material still work. The blocklist-specific scenarios
> (PII regex, internal codenames, competitor names) currently **do not block**. The
> blocklist resource is still created so it shows in the portal and can be re-attached
> with two lines of code once the service is fixed - see the comment on the RAI policy
> cell in [13-01](13-01-configure-bank-guardrails.ipynb).
>
> The blocklist mechanism itself has been verified end-to-end here previously: the
> cached outputs of [13-03-demo-guardrails.ipynb](13-03-demo-guardrails.ipynb) - captured
> in an earlier run when `customBlocklists` was attached - show all 5 PII inputs and all
> 5 codename/competitor prompts blocking correctly through the Responses API. So when
> the service bug is fixed and the blocklist is re-attached, no further demo verification
> is needed; the cached results are direct evidence the policy + blocklist combination
> behaves correctly.

## What gets demonstrated

| Layer | Mechanism | What the audience sees |
Expand Down Expand Up @@ -40,14 +61,17 @@ pinned to that deployment, so other agents on the project (`storytelling-agent`,
│ Indirect Attack) │
│ • Protected Material │
│ (Text + Code) │
│ • Custom blocklists │
└─────────────┬────────────┘
│ references by name
│ • (customBlocklists │
│ intentionally empty │
│ - service bug, see │
│ note above) │
└──────────────────────────┘

┌──────────────────────────┐
│ bank-demo-blocklist │ ← jailbreak phrases + PII regex +
│ │ codenames + competitors
└──────────────────────────┘
│ (created but NOT │ codenames + competitors. Resource
│ attached to policy) │ exists for portal visibility +
└──────────────────────────┘ re-attachment when bug is fixed.
```

## Run order
Expand Down Expand Up @@ -75,8 +99,8 @@ the Azure portal under `aif-core-{suffix}`:
listed in cell 4 of [13-01](13-01-configure-bank-guardrails.ipynb); flip the *regex*
toggle on for the PII patterns.
2. **Content filters → + Create custom content filter** - name `bank-guardrails-policy`.
Match the configuration in cell 6 of [13-01](13-01-configure-bank-guardrails.ipynb);
reference `bank-demo-blocklist` for both prompt and completion.
Match the configuration in cell 6 of [13-01](13-01-configure-bank-guardrails.ipynb).
Leave the **Blocklists** section empty (see Known limitation above).
3. **Deployments → + Deploy a model** - pick `gpt-4.1-mini` (`2025-04-14`),
name `gpt-4.1-mini-bank-guardrails`, SKU `GlobalStandard` at 30K TPM, set the content
filter to `bank-guardrails-policy` under Advanced.
Expand Down
113 changes: 6 additions & 107 deletions 13-guardrails/13-01-configure-bank-guardrails.ipynb

Large diffs are not rendered by default.

8 changes: 4 additions & 4 deletions 13-guardrails/13-02-create-bank-agent.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@
"name": "stdout",
"output_type": "stream",
"text": [
"Agent created: id=contoso-bank-agent:1 name=contoso-bank-agent version=1\n"
"Agent created: id=contoso-bank-agent:2 name=contoso-bank-agent version=2\n"
]
}
],
Expand Down Expand Up @@ -188,7 +188,7 @@
"name": "stdout",
"output_type": "stream",
"text": [
"The current APY on Sapphire Savings is 4.25%. There is no minimum balance required to open or maintain this high-yield savings account. Let me know if you'd like more details or help opening an account!\n"
"The current APY on Sapphire Savings is 4.25%. There is no minimum balance required to open or maintain this high-yield savings account. If you need more details or want to open an account, just let me know!\n"
]
}
],
Expand All @@ -203,9 +203,9 @@
],
"metadata": {
"kernelspec": {
"display_name": "foundry-samples",
"display_name": "awesome-foundry-nextgen",
"language": "python",
"name": "foundry-samples"
"name": "python3"
},
"language_info": {
"codemirror_mode": {
Expand Down
Loading
Loading