M365 ServiceDesk Simulator

An interactive Microsoft 365 service-desk training environment — a flight simulator for the helpdesk. Learners work a queue of realistic incidents: interview the requester, pull evidence from simulated Entra ID, Exchange, Teams, SharePoint and Intune consoles, consult the knowledge base, then commit to a diagnosis and a fix. Every action is recorded, and the attempt is assessed against how Microsoft 365 support is actually practised.

It is not a ticket tracker. There is nothing to administer; the tickets are the exercise.

Run it

cd ticket-tracker
python app.py            # http://127.0.0.1:8000
python app.py 9000       # use a different port

Then open http://127.0.0.1:8000/. Press Ctrl+C to stop. The SQLite database simulator.db is created automatically on first run.

Optional, for the AI features. The cheapest way in is Google's free tier — no credit card, 1,500 requests a day, which is roughly 60 full attempts:

pip install -r requirements.txt
export SIM_OPENAI_COMPATIBLE=gemini
export SIM_MODEL=gemini-2.5-flash
export OPENAI_API_KEY=<free key from aistudio.google.com/apikey>

Or with a paid OpenAI key, which needs no host setting:

export OPENAI_API_KEY=sk-...

What a learner does

1. Picks a ticket from the queue — twelve scenarios across identity, mail flow, licensing, collaboration, device policy and security, graded Beginner to Advanced.
2. Interviews the requester in chat. The requester is played by Claude with a set of hidden facts and explicit disclosure rules: they answer what is asked and volunteer nothing. Vague questions get vague answers.
3. Investigates in the simulated admin centres — user properties, sign-in logs with real AADSTS codes, authentication methods, message trace, quarantine, permission inheritance, device compliance, Conditional Access What-If, and more. Read-only actions are free; anything that changes the tenant is marked and logged.
4. Reads the knowledge base — 22 SOP-style articles covering the same ground a real M365 runbook would.
5. Submits a diagnosis, root cause, resolution steps and a reply to the requester.
6. Gets assessed across six dimensions, with the audit trail weighed alongside the write-up.

How assessment works

Two scorers, deliberately:

• The rubric (sim/grading.py) scores what is measurable without judgement: which diagnostic actions were performed, whether the tenant actually ended up fixed, whether anything forbidden was done, whether production was changed before evidence was gathered, whether the written answer names the root cause. This always runs.
• The reviewer (sim/ai.py) is a language model prompted as a principal M365 support engineer and given the scenario's ground truth — root cause, expected actions, red herrings — so its feedback is grounded rather than invented. It reads the action log as the source of truth about what happened and the write-up as the learner's reasoning, and it says so when the two disagree.

The final score is the average of the two per dimension. Neither alone is right: the rubric knows what was clicked but not whether the reasoning held; the model reads the reasoning but should not be the sole authority on what was done.

Dimension	Weight
Investigation & triage	20%
Root cause accuracy	25%
Remediation	25%
Safety & least privilege	15%
Documentation	8%
Customer communication	7%

Hints cost 3 points each, capped at 12.

Analyst profiles

Every name on the leaderboard links to a profile that explains the score rather than just reporting it:

• Where the score comes from — each dimension averaged across all their tickets, so a persistent weakness stands out from one bad afternoon.
• Coaching picture — strongest and weakest dimension, whether scores are improving or slipping, and gaps that have recurred across more than one ticket (a one-off is noise; the third time is a pattern).
• Ticket by ticket — every assessed attempt, expandable to the full assessment: what they did well, what they missed, practice notes, how a senior would have worked it, and what to improve next.

By default this is visible to everyone, which is the point in a cohort — people learn from reading each other's reviews. Set SIM_PRIVATE_FEEDBACK=1 to limit the written detail to the analyst themselves; scores and rankings remain public.

Without an API key the simulator still works end to end. Assessment runs on the rubric alone, hints fall back to scenario-authored ones, and the requester chat is disabled with an explanation. The same fallback covers API errors and rate limits — a broken key never produces a broken app.

Choosing a backend

sim/llm.py is the only file that knows any vendor's API exists. It exposes one call — send a system prompt and a conversation, optionally demand a JSON shape, get text and a token count — and each provider adapts its own SDK to that. So switching backends is an environment variable, and the same attempt can be scored by two different models for comparison.

Because Gemini, Groq, Cerebras, OpenRouter and a local Ollama all speak the OpenAI protocol, SIM_OPENAI_COMPATIBLE reaches every one of them through the same adapter. It accepts a shorthand or a full URL.

Setup	Configuration
Gemini free tier (recommended)	SIM_OPENAI_COMPATIBLE=gemini SIM_MODEL=gemini-2.5-flash OPENAI_API_KEY=<free key>
Groq — fastest replies	SIM_OPENAI_COMPATIBLE=groq SIM_MODEL=llama-3.3-70b-versatile
Local Ollama — no key, no network	SIM_OPENAI_COMPATIBLE=ollama SIM_MODEL=qwen2.5
OpenAI (paid)	OPENAI_API_KEY=sk-...
Claude (paid)	SIM_PROVIDER=anthropic ANTHROPIC_API_KEY=sk-ant-...

On a custom host both jobs use SIM_MODEL unless SIM_CHAT_MODEL is set too — the two-tier OpenAI default would name a model the host does not have.

Not every host implements strict json_schema structured outputs. When one rejects it, the adapter retries in plain JSON mode with the schema described in the prompt, so assessment degrades rather than fails.

Adding a genuinely different provider means implementing complete() on a new class and registering it — no changes to the prompts, the scoring, or the routes.

Adding a scenario

A scenario is one JSON file in scenarios/. No code changes.

It carries both what the learner sees and the ground truth they are graded against: the ticket, the requester, hidden_facts (with the question that unlocks each), red_herrings, root_cause, expected_actions, forbidden_actions, and a tenant block. The tenant block supplies views (what each console action returns) and effects (what the mutating ones change). Actions a scenario says nothing about return a plausible, unhelpful result — so chasing the wrong console costs time rather than breaking the illusion.

sim/scenarios.py validates strictly at startup, so a malformed file fails on launch with a useful message rather than halfway through someone's attempt. Knowledge base articles are Markdown with a small frontmatter block in kb/.

Linting a scenario

Validation proves a file is well-formed. The linter proves it is internally consistent — the checks that do not crash but quietly make a ticket unsolvable or misleading: an action id that does not exist, a fix whose flag no effect sets, a happy path that does not actually win, a forbidden read-only lookup.

python -m sim.linter                      # the live set
python -m sim.linter scenarios/drafts/*.json   # specific files

An empty report and a zero exit code mean the scenario is sound. The same checks run in the test suite over the whole set.

Generating scenarios

sim/generate.py has the model author a new ticket, constrained to the real action catalogue and knowledge-base ids, then puts every candidate through validation and the linter — repairing faults and retrying before it will keep one. It runs once in a while (say monthly) to keep the queue fresh.

python -m sim.generate --count 3                    # least-covered areas
python -m sim.generate --difficulty Advanced --category "Mail flow"
python -m sim.generate --dry-run                     # print, write nothing

Drafts are written to scenarios/drafts/, which the loader does not read — they are invisible to learners until you review one and move it up into scenarios/. Generation keeps the queue fresh; a human still decides what goes live. It needs an API key, like the rest of the AI layer.

Configuration

Variable	Default	Purpose
TICKET_HOST	127.0.0.1	Bind address. 0.0.0.0 serves the local network
SIM_PROVIDER	openai	Which SDK to use: openai or anthropic
SIM_OPENAI_COMPATIBLE	—	gemini, groq, cerebras, openrouter, ollama, or a full base URL
OPENAI_BASE_URL	—	The same thing, if you prefer to give the URL directly
OPENAI_API_KEY	—	Enables assessment coaching, requester chat and hints. Not needed for a local Ollama
ANTHROPIC_API_KEY	—	The same, when SIM_PROVIDER=anthropic
SIM_MODEL	gpt-5.6	Assessment model. Quality matters more than speed — it runs once per attempt
SIM_CHAT_MODEL	gpt-5.6-terra	Requester chat and hints. Runs on every message, so latency is what the learner feels
SIM_ACCESS_CODE	—	Shared code required to start. Set this on anything internet-facing
SIM_MAX_AI_CALLS_PER_ATTEMPT	40	Per-attempt spend guard
SIM_MAX_AI_CALLS_PER_IP_DAY	300	Per-address daily spend guard
SIM_PRIVATE_FEEDBACK	unset	Restrict written feedback on an analyst profile to that analyst. Scores and rankings stay public either way

If you deploy this publicly with an API key set, set SIM_ACCESS_CODE too. Otherwise anyone who finds the URL spends your API credit. The startup banner warns when a key is present and no code is set.

Open it from other devices on your network

By default the server binds 127.0.0.1, so only this machine can reach it. Set TICKET_HOST=0.0.0.0 to serve every device on your local network:

$env:TICKET_HOST="0.0.0.0"; python app.py     # PowerShell

TICKET_HOST=0.0.0.0 python app.py             # macOS/Linux
set TICKET_HOST=0.0.0.0 && python app.py      # Windows cmd

Startup then prints the URLs to type on the other device. The first address listed is the routable one; extra entries are usually virtual adapters (Docker, WSL, Hyper-V) that other devices can't use.

If a device can't connect, it's almost always the network rather than the app:

• Windows Firewall blocks the port by default. Allow it once, from an administrator PowerShell, and confirm the network is set to Private:

  New-NetFirewallRule -DisplayName "M365 Simulator LAN" -Direction Inbound `
    -Protocol TCP -LocalPort 8000 -Profile Private -Action Allow

• Guest Wi-Fi often isolates clients from each other — use the main SSID.
• Mesh systems and extenders may put the device on a different subnet.

Security: LAN mode is plain HTTP. Use it on networks you trust. For access over the internet, put it behind a reverse proxy with TLS and leave the default 127.0.0.1 binding in place.

Deploying

The app is served behind nginx by the systemd unit ticket-tracker. Three things change from the previous version:

1. The database file is new. The old tickets.db is unused — rename it rather than deleting it, and let simulator.db be created on first start.
2. ExecStart needs the virtualenv Python if you want the AI features:

   ExecStart=/home/project/ticket-tracker/.venv/bin/python app.py 8001
   Environment=ANTHROPIC_API_KEY=sk-ant-...
   Environment=SIM_ACCESS_CODE=...

   Without the venv, /usr/bin/python3 app.py 8001 still works — the simulator just runs in rubric mode.
3. Set SIM_ACCESS_CODE before putting an API key on a public host.

cd /home/project/ticket-tracker && git pull
python3 -m venv .venv && .venv/bin/pip install -r requirements.txt
sudo systemctl daemon-reload && sudo systemctl restart ticket-tracker

Tests

python -m unittest discover -s tests -v

Standard library only — no API key and no network needed. They cover scenario validation, the linter's consistency checks, the tenant action engine, and the deterministic grader.

Files

Path	Purpose
app.py	Entrypoint: argument parsing, binding, startup banner
sim/config.py	Paths and environment configuration
sim/db.py	SQLite schema and queries (attempts, actions, messages, submissions, evaluations)
sim/scenarios.py	Scenario loading and validation
sim/linter.py	Semantic checks: a scenario is consistent and winnable
sim/generate.py	Model-authored scenarios, vetted before they are written
sim/tenant.py	The simulated tenant: action catalogue and effect engine
sim/kb.py	Knowledge base index, search and Markdown rendering
sim/grading.py	Deterministic rubric scorer
sim/ai.py	Claude calls: assessment, requester roleplay, hints
sim/render.py	HTML rendering
sim/http.py	Request handler and routing
scenarios/*.json	The training tickets
kb/*.md	Knowledge base articles
assets/	Stylesheet and progressive-enhancement JavaScript

Notes

• All HTML output is escaped and all SQL is parameterised.
• Every page server-renders; JavaScript only removes page reloads. It works with scripting disabled.
• The tenant, users, tickets and errors are fictional. Nothing here touches a real Microsoft 365 tenant.