[observability] Observability Coverage Report - 2026-05-21

[github-actions[bot]]

Executive Summary

The broad seven-day fetch returned 30 recent runs. I excluded 4 still in progress and analyzed a capped breadth-first sample of 18 completed runs across 11 workflows, keeping at most 2 runs per workflow.

Required observability coverage was complete in the analyzed sample: firewall telemetry was present in 18/18 firewall-enabled runs and MCP telemetry was present in 18/18 MCP-enabled runs. No run was missing access.log, and no run was missing both gateway.jsonl and rpc-messages.jsonl.

The notable non-critical pattern is that all MCP coverage came from the canonical rpc-messages.jsonl fallback rather than the preferred gateway.jsonl format. That keeps debugging viable, but it means response-time metrics were unavailable in this sample. Firewall logs were present and parseable in all runs, with blocked traffic visible in 10 runs; however, blocked destinations frequently collapsed to (unknown), which reduces domain-level debugging value.

Key Alerts and Anomalies

No critical issues detected.

⚠️ Warnings:

• gateway.jsonl was absent in all 18 analyzed MCP-enabled runs. Coverage remained healthy because every run had valid rpc-messages.jsonl, but per-request duration metrics were unavailable.
• Firewall blocking was visible in 10 runs, but blocked destinations were logged as (unknown) for all 181 blocked requests in the sampled set. That is enough to confirm enforcement, but weak for root-cause analysis.

Coverage Summary

Component	Runs Analyzed	Logs Present	Coverage	Status
AWF Firewall (access.log)	18	18	100.0%	✅ Healthy
MCP Gateway (gateway.jsonl or rpc-messages.jsonl)	18	18	100.0%	✅ Healthy

Sample outcomes: 16 successful, 2 cancelled, 0 failed. Both cancelled runs still retained complete firewall and MCP telemetry.

📋 Detailed Run Analysis

Firewall-Enabled Runs

Workflow	Run ID	access.log	Entries	Allowed	Blocked	Status
Smoke CI	§26196518105	✅	13	2	0	✅ Healthy
Smoke CI	§26194191770	✅	10	2	0	✅ Healthy
Daily Security Red Team Agent	§26196763011	✅	246	66	0	✅ Healthy
AI Moderator	§26196605232	✅	32	9	0	✅ Healthy
PR Sous Chef	§26196485370	✅	127	20	11	✅ Healthy
Daily Reliability Review	§26195453006	✅	326	44	0	✅ Healthy
PR Sous Chef	§26195294104	✅	212	26	19	✅ Healthy
Issue Monster	§26194827931	✅	55	7	5	✅ Healthy
Matt Pocock Skills Reviewer	§26193570792	✅	187	18	18	✅ Healthy
PR Code Quality Reviewer	§26193570711	✅	114	12	11	✅ Healthy
Test Quality Sentinel	§26193570738	✅	428	48	39	✅ Healthy
Design Decision Gate 🏗️	§26193570741	✅	97	14	0	✅ Healthy
Test Quality Sentinel	§26193570190	✅	295	28	24	✅ Healthy
Matt Pocock Skills Reviewer	§26193570192	✅	182	18	15	✅ Healthy
PR Code Quality Reviewer	§26193570189	✅	91	9	9	✅ Healthy
Design Decision Gate 🏗️	§26193570188	✅	157	20	0	✅ Healthy
AI Moderator	§26193342397	✅	44	10	0	✅ Healthy
Daily Regulatory Report Generator	§26191963827	✅	403	32	30	✅ Healthy

Missing Firewall Logs (access.log)

None.

MCP-Enabled Runs

Workflow	Run ID	Telemetry Source	Entries	Servers	Tool Calls	Errors	Status
Smoke CI	§26196518105	rpc-messages.jsonl	6	2	1	0	✅ Healthy
Smoke CI	§26194191770	rpc-messages.jsonl	6	2	1	0	✅ Healthy
Daily Security Red Team Agent	§26196763011	rpc-messages.jsonl	4	1	1	0	✅ Healthy
AI Moderator	§26196605232	rpc-messages.jsonl	12	2	4	0	✅ Healthy
PR Sous Chef	§26196485370	rpc-messages.jsonl	6	1	2	0	✅ Healthy
Daily Reliability Review	§26195453006	rpc-messages.jsonl	52	2	24	0	✅ Healthy
PR Sous Chef	§26195294104	rpc-messages.jsonl	8	1	3	0	✅ Healthy
Issue Monster	§26194827931	rpc-messages.jsonl	20	1	9	0	✅ Healthy
Matt Pocock Skills Reviewer	§26193570792	rpc-messages.jsonl	18	2	7	0	✅ Healthy
PR Code Quality Reviewer	§26193570711	rpc-messages.jsonl	30	2	13	0	✅ Healthy
Test Quality Sentinel	§26193570738	rpc-messages.jsonl	8	1	3	0	✅ Healthy
Design Decision Gate 🏗️	§26193570741	rpc-messages.jsonl	4	1	1	0	✅ Healthy
Test Quality Sentinel	§26193570190	rpc-messages.jsonl	8	1	3	0	✅ Healthy
Matt Pocock Skills Reviewer	§26193570192	rpc-messages.jsonl	16	2	6	0	✅ Healthy
PR Code Quality Reviewer	§26193570189	rpc-messages.jsonl	30	2	13	0	✅ Healthy
Design Decision Gate 🏗️	§26193570188	rpc-messages.jsonl	6	1	2	0	✅ Healthy
AI Moderator	§26193342397	rpc-messages.jsonl	18	2	7	0	✅ Healthy
Daily Regulatory Report Generator	§26191963827	rpc-messages.jsonl	24	3	9	0	✅ Healthy

Missing MCP Telemetry (gateway.jsonl and rpc-messages.jsonl both absent)

None.

🔍 Telemetry Quality Analysis

Firewall Log Quality

• Total raw access.log lines analyzed: 3019
• Routed requests captured by firewall summaries: 566
• Domains accessed: 6 unique allowlisted domains
• Blocked requests: 181 (32.0%)
• Most accessed domains: api.githubcopilot.com:443 (219), api.anthropic.com:443 (103), github.com:443 (48)

Gateway Log Quality

• Telemetry source: rpc-messages.jsonl in all 18 analyzed MCP-enabled runs
• Total JSONL entries analyzed: 276
• MCP servers used: github, mcpscripts, safeoutputs, sentry
• Total tool calls: 109
• Error rate: 0.0%
• Average response time: N/A in this sample because gateway.jsonl was absent and the fallback RPC log does not carry per-call duration fields

Additional Telemetry

• agent-stdio.log: present in 18/18 analyzed runs
• Agent output artifact: present in 18/18 analyzed runs
• aw_info.json: present in 18/18 analyzed runs
• safeoutputs.jsonl: present in 18/18 analyzed runs

Healthy Runs Summary

All analyzed workflows had complete observability coverage: AI Moderator, Daily Regulatory Report Generator, Daily Reliability Review, Daily Security Red Team Agent, Design Decision Gate 🏗️, Issue Monster, Matt Pocock Skills Reviewer, PR Code Quality Reviewer, PR Sous Chef, Smoke CI, Test Quality Sentinel.

Recommended Actions

1. Keep treating rpc-messages.jsonl as sufficient MCP coverage, but restore or add gateway.jsonl where possible so response-time and status metrics are available without extra reconstruction.
2. Improve firewall blocked-request attribution so denied traffic records a concrete destination instead of (unknown) when policy and redaction rules allow it.
3. Preserve the current artifact set for agent-stdio.log, aw_info.json, agent output, and safeoutputs.jsonl; those secondary signals were consistently present and materially improve debugging.

Context: analysis window -7d; candidate runs fetched: 30; in-progress runs excluded: 4; workflow_runs_analyzed: 18; observability_coverage_percentage: 100.0%.

References:

• §26191963827
• §26196518105
• §26196763011

Warning

Firewall blocked 2 domains

The following domains were blocked by the firewall during workflow execution:

• api.github.com
• github.com

💡 Tip: api.github.com is blocked because GitHub API access uses the built-in GitHub tools by default. Instead of adding api.github.com to network.allowed, use tools.github.mode: gh-proxy for direct pre-authenticated GitHub CLI access without requiring network access to api.github.com:

tools:
  github:
    mode: gh-proxy

See GitHub Tools for more information on gh-proxy mode.

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "api.github.com"
    - "github.com"

See Network Configuration for more information.

Generated by 📊 Daily Observability Report for AWF Firewall and MCP Gateway · ● 7.2M · ◷

• expires on May 22, 2026, 12:18 AM UTC

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-05-22T00:18:55.869Z.

Closed by Workflow