Add ApplyGuardrailScope for security guardrail evaluation tracing#259
Merged
Conversation
Introduces a new OpenTelemetry tracing scope (ApplyGuardrailScope) that captures security guardrail evaluations as spans. This enables observability into content safety, policy enforcement, and risk assessment decisions made during agent operations. Port of microsoft/opentelemetry-distro-dotnet#109 (excluding ETW support). New contracts: - GuardrailDetails: Immutable dataclass capturing guardian evaluation metadata - GuardrailFinding: Represents an individual risk finding with severity and score - GuardrailDecisionType: Enum for guardian decisions (Allow, Audit, Deny, Modify, Warn) - GuardrailRiskSeverity: Constants for risk severity levels - GuardrailTargetType: Constants for guardrail targets (LlmInput, LlmOutput, etc.) New tracing scope: - ApplyGuardrailScope: Context manager scope with record_decision(), record_content_output(), and record_finding() methods Infrastructure updates: - Added microsoft.security.* and microsoft.guardian.* attribute keys to constants - Added apply_guardrail to the exporter operation name filter set - 14 unit tests covering scope creation, finding recording, and edge cases Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
Contributor
There was a problem hiding this comment.
Pull request overview
Adds guardrail evaluation tracing to the observability core package, exposing new guardrail contracts and an ApplyGuardrailScope for OpenTelemetry spans around security/policy checks.
Changes:
- Adds guardrail metadata/finding models and decision/severity/target constants.
- Implements
ApplyGuardrailScopewith span attributes, finding events, decision updates, and output recording. - Updates exports, operation filtering, constants, and unit coverage for the new scope.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
libraries/microsoft-agents-a365-observability-core/microsoft_agents_a365/observability/core/apply_guardrail_scope.py |
Implements guardrail tracing scope and event recording. |
libraries/microsoft-agents-a365-observability-core/microsoft_agents_a365/observability/core/constants.py |
Adds operation and guardrail/security attribute constants. |
libraries/microsoft-agents-a365-observability-core/microsoft_agents_a365/observability/core/__init__.py |
Exports the new guardrail API surface. |
libraries/microsoft-agents-a365-observability-core/microsoft_agents_a365/observability/core/exporters/utils.py |
Includes guardrail spans in exporter operation filtering. |
libraries/microsoft-agents-a365-observability-core/microsoft_agents_a365/observability/core/guardrail_decision_type.py |
Adds guardrail decision enum. |
libraries/microsoft-agents-a365-observability-core/microsoft_agents_a365/observability/core/guardrail_details.py |
Adds guardrail evaluation metadata dataclass. |
libraries/microsoft-agents-a365-observability-core/microsoft_agents_a365/observability/core/guardrail_finding.py |
Adds guardrail finding dataclass. |
libraries/microsoft-agents-a365-observability-core/microsoft_agents_a365/observability/core/guardrail_risk_severity.py |
Adds common risk severity values. |
libraries/microsoft-agents-a365-observability-core/microsoft_agents_a365/observability/core/guardrail_target_type.py |
Adds common guardrail target values. |
tests/observability/core/test_apply_guardrail_scope.py |
Adds unit tests for guardrail scope behavior and attributes. |
The request.content field (InputMessagesParam) can be a string, list[str], or InputMessages object. OTel attributes only accept primitive values or primitive sequences, so structured objects would be silently dropped. Now uses safe_json_dumps for non-string content, matching the pattern used by InferenceScope for input message serialization. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
juliomenendez
approved these changes
May 28, 2026
gwharris7
approved these changes
Jun 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Introduces a new OpenTelemetry tracing scope (\ApplyGuardrailScope) that captures security guardrail evaluations as spans. This enables observability into content safety, policy enforcement, and risk assessment decisions made during agent operations.
Port of microsoft/opentelemetry-distro-dotnet#109 (excluding ETW support, which is not applicable to the Python SDK).
Changes
New Contracts
New Tracing Scope
ecord_finding(), updating decisions via
ecord_decision(), and recording content output via
ecord_content_output().
OpenTelemetry Constants
Tests
Design Notes