Cosimulation that passes information from ExaGO to GridKit

agentigrid/README.md

@@ -56,7 +56,7 @@ Adding a generator supports two modes, chosen from the goal's wording (and named
 
 ### Custom Sweep Metric / Predicate (named registry)
 
-Beyond the default cost metric and standard V-band/loading feasibility, a sweep can select a **named, verified primitive** (`agentigrid/engine/sweep_metrics.py`) — not free-form code. Selection is by name; the registry is extensible (`register_metric` / `register_predicate`) for later capabilities.
+Beyond the default cost metric and standard V-band/loading feasibility, a sweep can select a **named, verified primitive** (`agentigrid/exago_engine/sweep_metrics.py`) — not free-form code. Selection is by name; the registry is extensible (`register_metric` / `register_predicate`) for later capabilities.
 
 - **`metric: "max_delta_v"`** — ranks buses by the worst *system-wide* voltage step `max_b |V_candidate[b] − V_base[b]|` caused by switching in a load block (`search.switched_load_mw`) at the candidate bus. A power-quality flag; the largest step need not be at the switched bus. Requires (and triggers) a one-time base-case solve for the reference voltages.
 - **`feasibility_predicate: "reactive_adequacy"`** — replaces standard feasibility with a reactive-headroom test: a feasible OPF must exist with the added unit forced to `(P = Pmax, Q = Qmax)` (the sweep pins `Qmin = Qmax`). Returns the limiting quantity when it fails.
@@ -78,6 +78,44 @@ agentigrid --tool gridkit --config configs/local_config.yaml \
   "Assume a short circuit fault at each bus of the system, tell me if the system can survive without blackout."
 ```
 
+**Starting from the latest ExaGO steady state.** A fault screen with
+`"start_from": "latest_steady_state"` (the LLM sets it when the goal asks for "the most
+recent steady state"; `--from-exago` forces it for the whole session) uses the GridKit
+case only for the dynamic models; the operating point and network come from the last
+simulation of the newest ExaGO journal in `workdir/exago/` (OPFLOW/PFLOW save their
+solved case, `opflowout.m` / `pflowout.m`, in the iteration folder). Bus voltages,
+machine `p0`/`q0`, loads, shunts and branches are taken from ExaGO; machines of
+generators that are off in ExaGO are removed with their controls. That start gets its
+own no-fault run before any fault. The screen is not run if that simulation did not
+converge, was a sweep, or has an online generator without a dynamic model in the
+GridKit case. The converted case is saved as `from_exago.case.json` in the session
+folder.
+
+**Saved solutions** (`exago.save_solutions` in the config, or
+`--set exago.save_solutions=<value>`). Single solves (base case, `modify`, `select`)
+always keep their solved case. For an action with many solves (sweep, explore,
+boundary, contingency, reserve):
+
+| Value | Kept |
+|---|---|
+| `best` (default) | the best run: cheapest feasible sweep candidate (or best by the sweep's metric), largest hosting capacity (boundary), Pareto-best variant (explore); the most recent if the action has no best (contingency, reserve) |
+| `newest` | the most recent run |
+| `all` | every run |
+| `none` | nothing is written; GridKit cannot start from the latest ExaGO steady state |
+
+Only the solution files are removed; run folders and their input cases stay. After an
+`explore`, pruning waits for the next action, so a variant chosen by `select` keeps
+its solution.
+
+**ContingencyAnalysis.** With `"application": "ContingencyAnalysis"` (the LLM sets it
+when the goal asks for contingency analysis), one GridKit `ContingencyAnalysis` process
+runs all bus faults of the screen, writing `output_<fault index>.csv` per fault in the
+screen's `contingency/` folder; the checks and verdict are the same as with
+`DynamicSimulation`. `ContingencyAnalysis` runs every fault device in the case file, so
+its case copy holds only the requested faults (fault devices shipped with the case,
+such as a demo fault, are left out of the copy). With `DynamicSimulation` they stay in
+the copy but are never switched on.
+
 Link the GridKit binaries and cases first (see [applications/gridkit/README.md](applications/gridkit/README.md)
 and [data/gridkit/README.md](data/gridkit/README.md)). Sessions are written to
 `workdir/gridkit/session_<time>/` (element map, case copies, one folder per fault,
@@ -438,6 +476,13 @@ For transient studies, link GridKit's binaries into `applications/gridkit/` and
 
 Edit `configs/default_config.yaml` or pass `--config path/to/config.yaml`. CLI arguments override config file values.
 
+One config file holds the settings for both tools: the `exago:` and `data:` sections
+for ExaGO, and the `gridkit:` section for GridKit (binary, timeout per run, the
+solver values written into each run's solver `.json`, and the default fault start).
+The LLM, iteration limit, worker count and output folders are shared. PJM pass/fail
+limits are not configurable; they live in `agentigrid/gridkit_engine/criteria.py`
+with their sources.
+
 Set your API key as an environment variable:
 ```bash
 export ANTHROPIC_API_KEY="your-key-here"

agentigrid/agentigrid/cli.py

@@ -6,7 +6,7 @@
 import logging
 import sys
 import threading
-from dataclasses import fields
+from dataclasses import fields, replace
 from datetime import datetime
 from pathlib import Path
 from typing import Any
@@ -208,6 +208,16 @@ def build_parser() -> argparse.ArgumentParser:
         help="Simulation tool: exago (steady state, .m case) or gridkit "
              "(transient stability, .case.json case). Default: exago",
     )
+    parser.add_argument(
+        "--from-exago",
+        action="store_true",
+        default=False,
+        help="GridKit only: start from the most recent ExaGO steady state "
+             "(last simulation of the newest journal in workdir/exago) instead of "
+             "the case's own operating point, for every fault screen. Without it, "
+             "the LLM chooses per fault screen (start_from), e.g. when the goal "
+             "asks for the most recent steady state.",
+    )
     parser.add_argument(
         "--version",
         action="version",
@@ -358,7 +368,8 @@ def _listen():
                 controller.request_stop()
                 print("[Steering] Stop requested.")
             elif lower == "save":
-                save_dir = Path(f"workdir/saved_session_{datetime.now().strftime('%Y%m%d_%H%M%S')}")
+                stamp = datetime.now().strftime('%Y%m%d_%H%M%S')
+                save_dir = controller._config.output.workdir / f"saved_session_{stamp}"
                 controller.save_session(save_dir)
                 print(f"[Steering] Session saved to: {save_dir}")
             elif lower == "status":
@@ -389,7 +400,7 @@ def _listen():
 
 def run_search(cfg: AppConfig, goal: str, quiet: bool = False) -> None:
     """Run the LLM-driven search loop."""
-    from agentigrid.engine.agent_loop import AgentLoopController
+    from agentigrid.exago_engine.agent_loop import AgentLoopController
 
     controller = AgentLoopController(cfg, quiet=quiet)
 
@@ -400,6 +411,16 @@ def run_search(cfg: AppConfig, goal: str, quiet: bool = False) -> None:
     controller.run(cfg.search.base_case, goal)
 
 
+def exago_output(cfg: AppConfig) -> AppConfig:
+    """ExaGO runs write to an ``exago/`` subfolder of the output folders,
+    as GridKit runs write to ``gridkit/``."""
+    return replace(cfg, output=replace(
+        cfg.output,
+        workdir=Path(cfg.output.workdir) / "exago",
+        logs_dir=Path(cfg.output.logs_dir) / "exago",
+    ))
+
+
 def run_gridkit(cfg: AppConfig, args: argparse.Namespace) -> None:
     """Run a GridKit transient-stability study (``--tool gridkit``)."""
     from agentigrid.gridkit_engine.agent_loop import GridkitAgentLoop
@@ -415,8 +436,21 @@ def run_gridkit(cfg: AppConfig, args: argparse.Namespace) -> None:
     if not case_file.exists():
         logger.error("Case file does not exist: %s", case_file)
         sys.exit(1)
+    from agentigrid.gridkit_engine.from_exago import (
+        SteadyStateError, find_latest_steady_state,
+    )
+
+    steady_state = None
+    if args.from_exago:
+        try:
+            steady_state = find_latest_steady_state(exago_output(cfg).output.workdir)
+        except SteadyStateError as exc:
+            logger.error("%s", exc)
+            sys.exit(1)
     if args.dry_run:
         print(f"GridKit settings: {settings}")
+        if steady_state:
+            print(f"Start from: {steady_state.describe()}")
         return
     quiet = getattr(args, "quiet", False) or False
     if not quiet:
@@ -431,8 +465,9 @@ def run_gridkit(cfg: AppConfig, args: argparse.Namespace) -> None:
     from agentigrid.gridkit_parsers.case_parser import CaseFileError
 
     try:
-        session = GridkitAgentLoop(cfg, settings=settings, quiet=quiet).run(case_file, args.goal)
-    except (FileNotFoundError, CaseFileError) as exc:
+        session = GridkitAgentLoop(cfg, settings=settings, quiet=quiet).run(
+            case_file, args.goal, steady_state=steady_state)
+    except (FileNotFoundError, CaseFileError, SteadyStateError) as exc:
         logger.error("%s", exc)
         sys.exit(1)
     if session.termination_reason in ("llm_error", "base_case_failed", "parse_failures"):
@@ -447,7 +482,7 @@ def run_regenerate_report(argv: list[str] | None = None) -> None:
     renders from deterministic journal data only.
     """
     from agentigrid.backends import create_backend
-    from agentigrid.engine.regenerate_report import regenerate_from_journal
+    from agentigrid.exago_engine.regenerate_report import regenerate_from_journal
 
     sub = argparse.ArgumentParser(
         prog="agentigrid regenerate-report",
@@ -470,7 +505,7 @@ def run_regenerate_report(argv: list[str] | None = None) -> None:
         overrides["llm.backend"] = args.backend
     if args.model:
         overrides["llm.model"] = args.model
-    cfg = load_config(Path(args.config), cli_overrides=overrides)
+    cfg = exago_output(load_config(Path(args.config), cli_overrides=overrides))
     setup_logging(cfg.output.logs_dir, verbose=cfg.output.verbose)
 
     generate_analysis = not args.no_analysis
@@ -508,6 +543,7 @@ def main(argv: list[str] | None = None) -> None:
     if args.tool == "gridkit":
         run_gridkit(cfg, args)
         return
+    cfg = exago_output(cfg)
 
     # Set up logging
     verbose = cfg.output.verbose
@@ -538,7 +574,7 @@ def main(argv: list[str] | None = None) -> None:
             sys.exit(1)
         if not quiet:
             print(f"Resuming session from: {resume_dir}")
-        from agentigrid.engine.agent_loop import AgentLoopController
+        from agentigrid.exago_engine.agent_loop import AgentLoopController
         controller = AgentLoopController(cfg, quiet=quiet)
         if sys.stdin.isatty():
             _start_stdin_listener(controller)

agentigrid/agentigrid/config.py

@@ -29,6 +29,7 @@
         "env_script": None,
         "timeout": 600,
         "mpi_np": 1,
+        "save_solutions": "best",
     },
     "data": {
         "data_dir": "./data/exago",
@@ -98,8 +99,27 @@
         "near_optimal_abs_tol": 5.0,
         "network_summary_max_generators": 40,
     },
+    "gridkit": {
+        "binary_dir": "./applications/gridkit",
+        "dynamicsimulation_binary": None,
+        "timeout": 600,
+        "study": {
+            "tmax": 15.0,
+            "dt_monitor": 0.01,
+            "rel_tol": 1.0e-7,
+            "abs_tol": 1.0e-9,
+            "dt_fixed": 0.0,
+            "max_steps": 0,
+            "max_order": 5,
+        },
+        "fault": {
+            "start": 1.0,
+        },
+    },
 }
 
+SAVE_SOLUTIONS = ("all", "best", "newest", "none")
+
 # ---------------------------------------------------------------------------
 # Frozen dataclasses
 # ---------------------------------------------------------------------------
@@ -117,6 +137,11 @@ class ExagoConfig:
     env_script: Optional[Path]
     timeout: int
     mpi_np: int = 1
+    # Solved cases written by OPFLOW/PFLOW (opflowout.m / pflowout.m): "all" keeps
+    # every one; "best" / "newest" keep one per multi-solve action (the best, else
+    # the most recent / the most recent); single solves are always kept; "none"
+    # writes none.
+    save_solutions: str = "best"
 
 
 @dataclass(frozen=True)
@@ -210,6 +235,39 @@ class ReportConfig:
         # top-N by Pmax. Bounds system-prompt size at scale.
 
 
+@dataclass(frozen=True)
+class GridkitStudyConfig:
+    """Keys written one-to-one into GridKit's solver .json
+    (GridKit/application/PhasorDynamics/README.md)."""
+
+    tmax: float = 15.0          # simulation length, s (PJM M-14B G.2.2: 10-15 s)
+    dt_monitor: float = 0.01    # recording interval, s
+    rel_tol: float = 1.0e-7
+    abs_tol: float = 1.0e-9
+    dt_fixed: float = 0.0       # 0 = adaptive time step
+    max_steps: int = 0          # 0 = IDA default, negative = unlimited
+    max_order: int = 5          # IDA method order, 1-5
+
+    def solver_options(self) -> dict[str, Any]:
+        """Solver keys other than tmax/dt_monitor (those are set per run)."""
+        return {f.name: getattr(self, f.name) for f in fields(self)
+                if f.name not in ("tmax", "dt_monitor")}
+
+
+@dataclass(frozen=True)
+class GridkitFaultConfig:
+    start: float = 1.0          # fault_on time, s, when the goal gives none
+
+
+@dataclass(frozen=True)
+class GridkitConfig:
+    binary_dir: Path = Path("./applications/gridkit")
+    dynamicsimulation_binary: Optional[Path] = None   # defaults to {binary_dir}/DynamicSimulation
+    timeout: float = 600        # seconds per GridKit run
+    study: GridkitStudyConfig = field(default_factory=GridkitStudyConfig)
+    fault: GridkitFaultConfig = field(default_factory=GridkitFaultConfig)
+
+
 @dataclass(frozen=True)
 class AppConfig:
     """Top-level immutable configuration for AgentiGrid."""
@@ -220,6 +278,7 @@ class AppConfig:
     search: SearchConfig
     output: OutputConfig
     report: ReportConfig = field(default_factory=ReportConfig)
+    gridkit: GridkitConfig = field(default_factory=GridkitConfig)
 
 
 # ---------------------------------------------------------------------------
@@ -333,12 +392,38 @@ def load_config(
 
     report = _build_section(ReportConfig, merged.get("report", {}), config_root, set())
 
-    cfg = AppConfig(exago=exago, data=data, llm=llm, search=search, output=output, report=report)
+    gridkit = _build_gridkit(merged["gridkit"], config_root)
+
+    cfg = AppConfig(exago=exago, data=data, llm=llm, search=search, output=output, report=report,
+                    gridkit=gridkit)
 
     _validate(cfg)
     return cfg
 
 
+def _build_gridkit(raw: dict, root: Path) -> GridkitConfig:
+    """Build the ``gridkit:`` section. GridKit ignores unknown solver keys
+    without a word, so unknown keys here are warned about, not dropped silently."""
+    nested = {"study": GridkitStudyConfig, "fault": GridkitFaultConfig}
+    for name, cls in [("gridkit", GridkitConfig), *nested.items()]:
+        section = raw if name == "gridkit" else raw.get(name) or {}
+        unknown = set(section) - {f.name for f in fields(cls)}
+        if unknown:
+            logger.warning("Unknown key(s) in gridkit config %s: %s (ignored)",
+                           "" if name == "gridkit" else f"section '{name}'", sorted(unknown))
+    flat = {k: v for k, v in raw.items() if k not in nested}
+    base = _build_section(
+        GridkitConfig, flat, root, {"binary_dir", "dynamicsimulation_binary"},
+    )
+    # An empty "study:" line in YAML is null, which would replace the defaults.
+    parts = {name: _build_section(cls, {**DEFAULTS["gridkit"][name], **(raw.get(name) or {})},
+                                  root, set())
+             for name, cls in nested.items()}
+    return GridkitConfig(binary_dir=base.binary_dir,
+                         dynamicsimulation_binary=base.dynamicsimulation_binary,
+                         timeout=base.timeout, **parts)
+
+
 def _validate(cfg: AppConfig) -> None:
     """Emit warnings for missing paths; does not raise."""
     if not cfg.exago.binary_dir.exists():
@@ -349,6 +434,10 @@ def _validate(cfg: AppConfig) -> None:
     if not cfg.data.data_dir.exists():
         logger.warning("Data directory does not exist: %s", cfg.data.data_dir)
 
+    if cfg.exago.save_solutions not in SAVE_SOLUTIONS:
+        logger.warning("Unknown exago.save_solutions '%s' (expected one of %s)",
+                       cfg.exago.save_solutions, SAVE_SOLUTIONS)
+
     valid_backends = {"openai", "anthropic", "ollama", "ollama-cloud"}
     if cfg.llm.backend not in valid_backends:
         logger.warning("Unknown LLM backend '%s' (expected one of %s)", cfg.llm.backend, valid_backends)
@@ -452,3 +541,13 @@ def _validate(cfg: AppConfig) -> None:
             "sweep_max_workers=%d is negative; treating as auto (min(cpu_count, 16)).",
             cfg.search.sweep_max_workers,
         )
+
+    study, fault = cfg.gridkit.study, cfg.gridkit.fault
+    if not 1 <= study.max_order <= 5:
+        logger.warning("gridkit.study.max_order=%s is outside 1-5; GridKit will refuse to run.",
+                       study.max_order)
+    if study.dt_monitor <= 0:
+        logger.warning("gridkit.study.dt_monitor=%s must be positive.", study.dt_monitor)
+    if fault.start >= study.tmax:
+        logger.warning("gridkit.fault.start=%s s is not before gridkit.study.tmax=%s s.",
+                       fault.start, study.tmax)

agentigrid/agentigrid/engine/__init__.py → agentigrid/agentigrid/exago_engine/__init__.py

@@ -4,20 +4,20 @@
 
 import logging
 
-from agentigrid.engine.agent_loop import AgentLoopController, SearchSession
-from agentigrid.engine.commands import ModCommand, parse_command
-from agentigrid.engine.executor import SimulationExecutor, SimulationResult
-from agentigrid.engine.explore import ExploreCache, VariantResult
-from agentigrid.engine.journal import JournalEntry, ObjectiveEntry, ObjectiveRegistry, SearchJournal
-from agentigrid.engine.metric_extractor import available_metrics, extract_all_metrics, extract_metric
-from agentigrid.engine.modifier import ModificationReport, apply_modifications
-from agentigrid.engine.objective_parser import build_objective_extraction_prompt, parse_objective_extraction
-from agentigrid.engine.schema_description import command_schema_text
-from agentigrid.engine.session_io import save_session, load_session
-from agentigrid.engine.validation import ValidationResult, validate_command
-from agentigrid.parsers.matpower_model import MATNetwork
+from agentigrid.exago_engine.agent_loop import AgentLoopController, SearchSession
+from agentigrid.exago_engine.commands import ModCommand, parse_command
+from agentigrid.exago_engine.executor import SimulationExecutor, SimulationResult
+from agentigrid.exago_engine.explore import ExploreCache, VariantResult
+from agentigrid.exago_engine.journal import JournalEntry, ObjectiveEntry, ObjectiveRegistry, SearchJournal
+from agentigrid.exago_engine.metric_extractor import available_metrics, extract_all_metrics, extract_metric
+from agentigrid.exago_engine.modifier import ModificationReport, apply_modifications
+from agentigrid.exago_engine.objective_parser import build_objective_extraction_prompt, parse_objective_extraction
+from agentigrid.exago_engine.schema_description import command_schema_text
+from agentigrid.exago_engine.session_io import save_session, load_session
+from agentigrid.exago_engine.validation import ValidationResult, validate_command
+from agentigrid.exago_parsers.matpower_model import MATNetwork
 
-logger = logging.getLogger("agentigrid.engine")
+logger = logging.getLogger("agentigrid.exago_engine")
 
 __all__ = [
     "AgentLoopController",