Policy Scenario Simulator

This repository delivers a modular platform for simulating policy scenarios with populations of LLM-driven agents. Agents plan, act, and reflect inside a deterministic environment that models housing supply, administrative processes, messaging, and macro-economic indicators. The system produces audit trails, run manifests, and quantitative metrics so results can be benchmarked against policy targets.

Features

• Scenario orchestrator that executes full simulation runs, persists manifests, and logs observations/actions for every tick.
• Deterministic agent runtime implementing observe → plan → act → reflect, with persona-aware heuristics for tenants, landlords, and planners.
• Environment simulator that manages housing units, applications, rent adjustments, administrative cases, price indices, and eviction logic.
• Typed tool adapters covering legal lookup, communications, housing market, transport, economy, and administration workflows with real state mutations.
• Population synthesizer seeded for reproducibility.
• Metrics service that computes calibration, realism, and outcome KPIs (eviction rate, affordability, wait times, complaints rate) and stores them with run metadata.
• FastAPI server exposing run creation, retrieval, and evaluation endpoints.
• Typer-based CLI to launch and inspect runs offline.
• Streamlit dashboard for interactive lever tuning with charts and metric summaries.
• Example configurations for the “Rental Stress & Policy Levers” scenario.

Getting Started

Install dependencies and run tests:

pip install -e .
pytest

Launch a sample run via the CLI:

python -m policy_scenario_simulator.cli.main \
  launch-run \
  --scenario-path configs/scenario_rent_cap.json \
  --population-path configs/population_sample.json \
  --lever-values-path configs/lever_values.json \
  --profile-path configs/run_profile_lowcost.json

Start the FastAPI service:

uvicorn policy_scenario_simulator.api.server:app --reload

Launch the Streamlit dashboard:

streamlit run ui/app.py

The UI lets you load the sample configurations (or upload your own JSON files), adjust policy levers via sliders, execute the simulation, and explore metrics and trend charts.

Configuration

The configs/ directory contains sample JSON documents describing scenarios, population priors, and run profiles. Update these with real data sources and secure storage URIs before production use.

Model endpoints and OpenAI credentials

configs/run_profile_lowcost.json now demonstrates how to declare model endpoints backed by OpenAI-compatible APIs. Each entry supports:

• provider: The model host (e.g. openai).
• model or model_env: Either supply the exact model identifier, or reference an environment variable to avoid hard-coding values.
• api_key or api_key_env: Configure the credential inline or via environment variable (recommended).
• base_url or base_url_env: Override the API base URL when targeting Azure OpenAI or other compatible gateways.

If no base URL is provided for the OpenAI provider, the client defaults to https://api.openai.com/v1.

Safety and Governance

• Tool contracts validate every invocation before execution.
• Run manifests capture seeds, tool contract hashes, lever values, and artifact URIs for auditability.
• Metrics evaluation is mandatory—runs are marked complete only after calibration, realism, and outcome scores are persisted.

Next Steps

• Integrate real data sources for legal, housing, and economic signals.
• Extend agent cognition with memory modules and richer deliberation loops.
• Calibrate metrics against historical distributions and expand scenario coverage (transport disruptions, cost-of-living shocks).