[workshop-sim] Workshop Simulation Report — 2026-07-21 (Run #11, 1000×Monte Carlo)

[github-actions]

Overview

• Date: 2026-07-21
• Students simulated: 46 × 1000 Monte Carlo runs
• Workshop steps available: 32/32
• Overall success rate: 27.6% (95% Monte Carlo interval: 27.1%–28.0%)
• Highest-dropout step: 07-first-workflow (20.4% conditional dropout among 25,585 at-risk runs; 95% Monte Carlo interval: 19.9%–20.9%)
• Lowest curriculum quality step: 15-conditional-logic.md (overall score 7.53/10)
• Learning KPI index: 7.49/10 (active_learning 4.80 · checkpoint_quality 8.75 · scaffolding 9.38)
• Model: 2026-07-survival-model-v2 / 2026-07-assumption-model-v1 (parameter hash 3544776554)
• Limitation: synthetic results reflect explicit model assumptions; intervals exclude model and population-assumption uncertainty

Part Summary

Part	Files	Mean Score	Std Dev
Part 1 — core path (lessons 00–14)	20	9.12 / 10.0	±0.73
Part 2 — advanced (lessons 15+)	12	8.75 / 10.0	±0.48
Overall corpus	32	8.99 / 10.0	±0.65

No pages are classified as other.

Critical Findings

1. Setup is a hard filter for all non-advanced learners (Part 1): 16.4% of all 46,000 starting runs drop out at setup, driven by setup-friction. Every mobile and many ui_preferred students fail here because the setup step requires a working terminal or Codespace; no pure-UI alternative exists. This is an access barrier, not a conceptual gap.
2. Copilot access gap drives the most failures at Step 7 (Part 1): copilot-access-missing accounts for 3,659 out of 5,214 dropouts at 07-first-workflow. Learners without an active Copilot plan hit a hard state check that earlier signalling in Step 01 prerequisites could deflect before they invest time reaching Step 7.
3. Concept gap at Step 5 hits beginners and program-managers hardest (Part 1): 05-agentic-workflows-intro.md is the highest-volume dropout step by total count (6,917 failures) with agentic-concept-gap as top category. Its active_learning score is 2.6/10 — the weakest dimension in any content-learning step — and there is no recovery path for learners who finish still confused.
4. Learning quality health is acceptable but active_learning is the system-wide weak dimension: The cohort learning KPI index is 7.49/10, carried by strong checkpoint_quality (8.75) and scaffolding (9.38). active_learning (4.80) lags across almost every step — learners who reach later steps are scaffolded well but lack practice tasks that deepen retention.
5. The most impactful repairs all belong to Part 1 (lessons 00–14): The top-3 dropout steps (07-first-workflow, 05-agentic-intro, 02-setup) are all Part 1. Fixing access barriers here yields the greatest completion-rate uplift before learners invest time in Part 2.

Top Repairs to Prioritize

Note: some student dropout is expected and acceptable. Repairs must maintain or improve the learning KPI index — do not lower the cognitive bar or remove practice to chase headline completion numbers.

1. Add a Copilot plan pre-check callout to 01-prerequisites.md — surfaces a clear gate so learners without access self-select out or activate a plan before reaching Step 7 (completion impact: ↑ · learning KPI impact: ↔)
2. Add a practice exercise and "still confused?" recovery path to 05-agentic-workflows-intro.md — active_learning score is 2.6/10 (lowest in any content-learning step); a short classify-the-task exercise plus a worked example or glossary link reduces agentic-concept-gap dropouts and raises the KPI (completion impact: ↑ · learning KPI impact: ↑)
3. Route mobile and UI-preferred learners to the browser-only Codespace path earlier in the 02-setup group — setup-friction affects 100% of mobile learners; explicit early routing reduces access barriers without changing instructional depth (completion impact: ↑ · learning KPI impact: ↔)

Dropout by step

Step	At-risk runs	Dropouts	Conditional dropout	95% CI	Failure mode	Top reason
07-first-workflow (P1)	25,585	5,214	20.4%	19.9%–20.9%	Access barrier	No active Copilot plan
05-agentic-intro (P1)	34,843	6,917	19.9%	19.4%–20.3%	Learning barrier	Agentic concept not absorbed
02-setup (P1)	46,000	7,557	16.4%	16.1%–16.8%	Access barrier	Setup friction (terminal/environment)
04-actions-intro (P1)	38,443	3,600	9.4%	9.1%–9.7%	Learning barrier	Concept overload
06-install-gh-aw (P1)	27,926	2,341	8.4%	8.1%–8.7%	Access barrier	Extension install friction
12-test-and-iterate (P1)	18,553	1,061	5.7%	5.4%–6.1%	Learning barrier	Test and iterate friction
09-agentic-editing (P1)	19,504	951	4.9%	4.6%–5.2%	Learning barrier	Workflow editing friction
19-research-driven-training-node (P2)	15,458	687	4.4%	4.1%–4.8%	Learning barrier	Research node friction
17-add-mcp-tools (P2)	16,669	694	4.2%	3.9%–4.5%	Learning barrier	MCP tooling friction
18-share-and-reuse (P2)	15,975	517	3.2%	3.0%–3.5%	Learning barrier	Workflow reuse friction
24-self-hosted-runners (P2)	13,635	431	3.2%	2.9%–3.5%	Access barrier	Self-hosted runner friction
08b-interpret-your-run (P1)	20,055	551	2.7%	2.5%–3.0%	Learning barrier	Output interpretation gap
15-conditional-logic (P2)	17,492	441	2.5%	2.3%–2.8%	Learning barrier	Conditional logic friction
20-persistent-memory (P2)	14,771	341	2.3%	2.1%–2.6%	Learning barrier	Memory pattern friction
21-inline-sub-agents (P2)	14,430	326	2.3%	2.0%–2.5%	Learning barrier	Sub-agent friction
16-connect-data-source (P2)	17,051	382	2.2%	2.0%–2.5%	Learning barrier	Data source friction
26-manage-costs-and-budgets (P2)	12,947	273	2.1%	1.9%–2.4%	Learning barrier	Cost guardrail friction
25-audit-and-observability (P2)	13,204	257	1.9%	1.7%–2.2%	Learning barrier	Audit friction
23-ab-experiments (P2)	13,875	240	1.7%	1.5%–2.0%	Learning barrier	Experiment design friction
22-error-handling-and-resilience (P2)	14,104	229	1.6%	1.4%–1.8%	Learning barrier	Resilience friction
08-run-your-workflow (P1)	20,371	316	1.6%	1.4%–1.7%	Access barrier	UI run guidance gap

Learning quality KPIs

Step file	Overall score	active_learning	checkpoint_quality	scaffolding	Learning KPI	Repair priority
05-agentic-workflows-intro.md	7.72	2.6	10	10.0	7.31	High
15-conditional-logic.md	7.53	4.0	10	10.0	7.82	Medium
02a-setup-codespace.md	7.67	4.4	10	5.0	6.60	Medium
02c-setup-browser.md	8.16	6.5	10	5.0	7.36	Medium
17-add-mcp-tools.md	8.32	2.5	10	10.0	7.27	Medium
12-test-and-iterate.md	8.74	3.7	10	10.0	7.71	Low
09-agentic-editing.md	8.71	4.9	10	10.0	8.15	Low
08-run-your-workflow.md	8.78	3.9	10	10.0	7.78	Low
20-persistent-memory.md	8.68	3.9	10	10.0	7.78	Low
18-share-and-reuse.md	8.68	4.7	10	10.0	8.07	Low
Cohort mean	8.99	4.80	8.75	9.38	7.49	—

Curriculum quality metrics

Step file	Overall score	Lowest rubric dimension	Recommended repair focus
15-conditional-logic.md	7.53	active_learning (4.0)	Add a hands-on conditional branch exercise
02a-setup-codespace.md	7.67	active_learning (4.4)	Add a learner-executed verification step
05-agentic-workflows-intro.md	7.72	active_learning (2.6)	Add classify-the-task exercise + recovery path
02c-setup-browser.md	8.16	scaffolding (5.0)	Add step-by-step scaffolded browser path
17-add-mcp-tools.md	8.32	active_learning (2.5)	Add a tool-testing practice task
18-share-and-reuse.md	8.68	active_learning (4.7)	Add a reuse scenario exercise
20-persistent-memory.md	8.68	active_learning (3.9)	Add a memory read/write practice task
24-self-hosted-runners.md	8.69	active_learning (5.6)	Add an enterprise runner configuration task
09-agentic-editing.md	8.71	active_learning (4.9)	Add a tweak-and-re-run exercise
12-test-and-iterate.md	8.74	active_learning (3.7)	Add a structured iteration exercise

Segment breakdowns

Success rate by technical level

Level	Students	Avg success rate
advanced	5	57.5%
actions-user	18	48.0%
github-basic	9	12.7%
beginner	14	0.1%

Success rate by personality

Personality	Students	Avg success rate
confused	5	42.2%
methodical	10	35.2%
curious	16	28.0%
skeptical	5	26.1%
impatient	10	12.5%

Success rate by UI preference

UI preferred	Students	Avg success rate
false (CLI/VS Code)	27	36.3%
true (UI-preferred)	19	15.2%

Notable student journeys (3)

Surprising success — Learner 015 (advanced, confused, devops, CLI): Despite a confused personality (typically a −0.10 confidence penalty), this advanced DevOps engineer achieves a 62.1% success rate. Prior GitHub Actions experience means concept steps and setup are low-friction; confusion manifests as hesitation rather than hard failures, and the terminal path is well-matched to the devops background.

Unexpected dropout — Learner 023 (beginner, curious, program-manager, VS Code, ui_preferred): Despite a curious personality that usually correlates with persistence, this learner achieves near-zero success (0.3%). The beginner + program-manager combination means the Actions mental model assumed by 05-agentic-intro is entirely absent. Even curious persistence cannot bridge a complete knowledge gap without a recovery path.

Content-gap case — Learner 003 (github-basic, skeptical, program-manager, CLI): This learner can commit and push but hits 05-agentic-intro as the primary failure step (0.6% success). The skeptical personality amplifies the agentic concept gap: without a concrete practice task or worked example, skeptical learners disengage rather than accepting the concept on faith. This is the clearest signal that 05-agentic-workflows-intro.md active_learning score of 2.6/10 represents a genuine instructional gap.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by 🔬 Workshop Student Simulator · 178.8 AIC · ⌖ 7.77 AIC · ⊞ 10.8K · ◷

• expires on Jul 22, 2026, 2:50 PM UTC