[observability] Observability Coverage Report - 2026-09-02

[github-actions[bot]]

Executive Summary

Observability coverage across the 8 analyzed workflow runs is incomplete. Firewall coverage is 83.3% across 6 firewall-enabled runs, and MCP telemetry coverage is 100.0% across 6 MCP-enabled runs. Critical gaps remain where a required primary log is missing entirely.
Most quality issues are not missing files but incomplete signal quality. In particular, 3 firewall-enabled runs had access logs without both allowed and blocked traffic, and 0 MCP-enabled runs had parse or field-quality issues.

Key Alerts and Anomalies

Caution

Critical observability gaps were detected in the analyzed run set.

Critical Issues:

• Firewall log missing for Avenger §33695240310

Warnings:

• Firewall log quality issue for Cache directory setup §33695304111: No blocked requests observed
• Firewall log quality issue for PR Sous Chef §33695143285: No blocked requests observed
• Firewall log quality issue for PR Sous Chef §33694444009: No blocked requests observed

Coverage Summary

Component	Runs Analyzed	Logs Present	Coverage	Status
AWF Firewall (access.log)	6	5	83.3%	Critical
MCP Gateway (gateway.jsonl or rpc-messages.jsonl)	6	6	100.0%	Healthy

Detailed Run Analysis

Firewall-Enabled Runs

Workflow	Run ID	access.log	Entries	Allowed	Blocked	Status
Avenger	§33695240310	❌	0	0	0	Critical
Cache directory setup	§33695304111	✅	59	46	0	Warning
PR Sous Chef	§33695143285	✅	119	64	0	Warning
PR Sous Chef	§33694444009	✅	48	30	0	Warning
Detection Analysis Report	§33694409122	✅	213	116	1	Healthy
Issue Monster	§33694375396	✅	49	18	1	Healthy

Missing Firewall Logs (access.log)

Workflow	Run ID	Date	Link
Avenger	33695240310	2026-09-02	§33695240310

MCP-Enabled Runs

Workflow	Run ID	Telemetry Source	Entries	Servers	Tool Calls	Errors	Status
Avenger	§33695240310	rpc-messages.jsonl	2	1	0	0	Healthy
Cache directory setup	§33695304111	rpc-messages.jsonl	6	2	0	0	Healthy
PR Sous Chef	§33695143285	rpc-messages.jsonl	6	1	2	0	Healthy
PR Sous Chef	§33694444009	rpc-messages.jsonl	4	1	1	0	Healthy
Detection Analysis Report	§33694409122	rpc-messages.jsonl	14	3	3	0	Healthy
Issue Monster	§33694375396	rpc-messages.jsonl	14	1	6	0	Healthy

Missing MCP Telemetry (no gateway.jsonl or rpc-messages.jsonl)

Workflow	Run ID	Date	Link
None	-	-	-

Telemetry Quality Analysis

Firewall Log Quality

• Total access.log entries analyzed: 488
• Domains accessed: 11 unique
• Blocked requests: 2 (0.41%)
• Most accessed domains: -, api.anthropic.com:443, api.openai.com:443

Gateway Log Quality

• Telemetry sources observed: rpc-messages.jsonl
• Total entries analyzed: 46
• MCP servers used: agenticworkflows, github, safeoutputs
• Total tool calls: 12
• Error rate: 0.0%
• Average response time: N/A when request/response pairing was unavailable

Additional Telemetry Presence

• agent-stdio.log: present in 6/8 analyzed runs
• safe_output.jsonl: present in 0/8 analyzed runs
• aw_info.json: present in 0/8 analyzed runs
• awf-config.json: present in 6/8 analyzed runs

Healthy Runs Summary

• Runs with healthy primary observability signals for all enabled components: 4/8
• Detection Analysis Report §33694409122
• Issue Monster §33694375396
• [aw] Failure Investigator (6h) §33695475148
• Daily Trajectory Grader Implementer §33695255552

Recommended Actions

1. Add an automated post-run assertion that every firewall-enabled run publishes at least one access.log file and fail the observability workflow when none is found.
2. Promote gateway.jsonl adoption where possible, but continue treating rpc-messages.jsonl as the canonical healthy fallback during migration.
3. Backfill or standardize aw_info.json publication in downloaded artifacts so firewall and MCP enablement can be confirmed from metadata instead of inferred from file layout.

Historical Trends

Historical trend data was not available in the downloaded artifact set for this daily run, so this report is limited to the current 7-day sample.

References:

• §33695240310
• §33695304111
• §33695143285

Analysis window: Last 7 days | Runs analyzed: 8

Warning

Firewall blocked 2 domains

The following domains were blocked by the firewall during workflow execution:

• ab.chatgpt.com
• api.github.com

[!TIP]
api.github.com is blocked because GitHub API access uses the built-in GitHub tools by default. Instead of adding api.github.com to network.allowed, use tools.github.mode: gh-proxy for direct pre-authenticated GitHub CLI access without requiring network access to api.github.com:

tools:
  github:
    mode: gh-proxy

See GitHub Tools for more information on gh-proxy mode.

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "ab.chatgpt.com"
    - "api.github.com"

See Network Configuration for more information.

Generated by 📊 Daily Observability Report for AWF Firewall and MCP Gateway · codex · gpt54 · 72.2 AIC · ⌖ 10.5 AIC · ⊞ 14.4K · ◷

• expires on Sep 3, 2026, 3:51 PM UTC-08:00

[github-actions[bot]]

Smoke test run 33698479991: starting tests

Warning

Firewall blocked 7 domains

The following domains were blocked by the firewall during workflow execution:

• accounts.google.com
• android.clients.google.com
• clients2.google.com
• contentautofill.googleapis.com
• dc.services.visualstudio.com
• www.google.com
• www.gstatic.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "dc.services.visualstudio.com"
    - "www.google.com"
    - "www.gstatic.com"

See Network Configuration for more information.

📰 BREAKING: Report filed by Smoke Copilot - AOAI (Entra) · copilot · o40mini · 43.1 AIC · ⌖ 3.71 AIC · ⊞ 19.9K · ◷
_{Comment /smoke-copilot-aoai-entra to run again
Add label smoke to run again}