Make the engine hostable in-process: context factory seam, run ids and progress events

CHANGELOG.md

@@ -7,8 +7,91 @@ RayMigrator follows Semantic Versioning where applicable.
 
 ## [Unreleased]
 
+### Added
+
+- Typed progress events for hosts. `MigrationContext.ProgressObserver` takes an
+  `IMigrationProgressObserver` (namespace `Raycoon.RayMigrator.Core.Progress`)
+  that receives `RunStartedEvent` (after the run row exists, with its id),
+  `FileStartedEvent` and `FileCompletedEvent` for every file on every target
+  (migration, rollback and baseline, with the record id and a `FileOutcome`),
+  `BlockCompletedEvent` for every executed SQL block, and `RunCompletedEvent`
+  with the result. Every completion follows a start for the same file and
+  target; a rollback without a rollback file reports `Skipped`; the block
+  events of the atomic path follow the commit; and `FromRelease` is reported
+  in Simulate and Validate as well. A host shows progress without parsing
+  log text. An observer that throws is logged and the run continues. The
+  default observer ignores every event; the CLI does not use them.
+- `BaselineResult.BaselinedRecords` counts the records a baseline wrote, one
+  per file and target that was pending, also when the baseline failed midway.
+  `BaselinedFiles` keeps counting files per release and target group.
+
+### Changed
+
+- `IMigrationContextFactory` is the entry point of a host for one run, the CLI
+  pipeline included. `Create` prepares options a host built in memory with the
+  new `RayMigratorOptionsPreparer`, which gives them what options bound from
+  the `appsettings*.json` files always got: `{ENV:VARIABLE}` placeholders are
+  resolved in every string of the options graph (a missing variable is an
+  error that names the variable and the path), the product defaults are
+  merged down, every enum-typed value is checked, the data annotations run
+  and write their documented defaults into null properties, and the rule
+  catalog is applied. `CreateFromValidatedOptions` skips the preparation for
+  options the options framework already validated. Both fill the
+  `DalSpecificPropertiesDictionary` for the repository, the product's targets
+  and the database logging, check the schema names, and make the context
+  current (`IMigrationContextAccessor` and `MigrationLoggingContext`). A host
+  that built the context itself got none of this: pre-built options kept null
+  timeouts that later failed with an `InvalidOperationException`, every
+  engine used the SQL Server block delimiter `GO`, and the Serilog enricher
+  saw no run properties. The factory now lives in `Raycoon.RayMigrator.Services`,
+  takes the accessor and a logger, and no longer takes a version argument (the
+  engine knows its own). The accessor is a singleton in both host modes; the
+  API accessor isolates per async flow through its `AsyncLocal` anyway.
+  `ProductAliasGuard` explains an unknown product alias with the same message
+  the CLI showed, now also for hosts. The preparer checks connection strings
+  that contain placeholders as they were written, so a credential a
+  placeholder supplies is not reported as hardcoded (#30).
+- The result contracts of `IMigrationService` name the repository run and the
+  target. `MigrationOperationResult`, `BaselineResult` and `MigrationRunInfo`
+  carry `MigrationRunId`, the id of the run row (0 when the run mode writes no
+  row); the random `RunId` Guid that was never stored is gone, as is the
+  `InitiatedBy` placeholder that nothing ever filled. `MigrationFileResult`
+  carries `TargetAlias` and `MigrationRecordId` wherever the entry belongs to
+  one target (TargetByTarget order, failures, rollbacks). `MigrationRunInfo`
+  carries `Environment` and `FromRelease`; `MigrationHistory`,
+  `MigrationStatusInfo` and `BaselineResult` carry `Environment`.
+- `MigrateUpAsync`, `MigrateDownAsync` and `BaselineAsync` refuse a request
+  whose `TargetReleaseVersion` or `Environment` differs from the
+  `MigrationContext`, like the run-mode check does: the run row stamps the
+  target release and the environment from the context, so a host that let
+  them diverge recorded a release the run never aimed at, or ran one
+  environment's files against another environment's rows. The settings
+  snapshot of a migrate-up run records the `AllowOutOfOrder` value of the
+  request, which is the value the run used.
+- `MigrationService` reads the configuration of the current run from its
+  `MigrationContext` instead of from `IOptions<RayMigratorOptions>`, and
+  `TemplateCache` no longer takes the options: a host builds the cache once
+  per process and calls `ValidateConfigurationAgainstTemplateCache` for each
+  configuration it runs. A host that creates one context per run from
+  pre-built options (RayMigrator Studio) can now execute migrations; before,
+  the service read the products and the CLI tools from the options registered
+  in the container, so a context built from other options failed with
+  "Sequence contains no matching element". The CLI behaves as before. Breaking
+  for hosts that construct `MigrationService` or `TemplateCache` themselves.
+
 ### Fixed
 
+- `GetHistoryAsync` and `GetStatusAsync` read the runs of the requested
+  environment only. `Repository_MigrationRun_Select` filtered by product
+  alone, so the history listed the runs of every environment of the product
+  (with zero counts, because the record history is per environment) and the
+  status reported the last run of whichever environment ran last. The
+  template takes `@EnvironmentId` in all five engines.
+- `MigrationRun.FromReleaseVersion` holds the release the run started from:
+  the highest Migrated release of the product and environment, read right
+  before the run row is inserted, or `NULL` for a first run. Every run row
+  stored the literal text `FromReleaseVersion` before, because the parameter
+  was bound to its own name instead of a value.
 - `RULE_7_3` (hardcoded credential) no longer fires for a connection string
   that supplies its credential through an `{ENV:VARIABLE}` placeholder. The
   CLI ran the rule catalog on the resolved values, so a placeholder resolving

CLAUDE.md

@@ -66,7 +66,8 @@ Seven commands, lowercase kebab-case: `migrate-up`, `migrate-down`, `validate-ha
 - **Configuration** sits under the `RayMigrator` root section: `Repository`, `ProductDefaults` → `TargetGroupDefaults` → `TargetDefaults`, `Products` → `TargetGroups` → `Targets`, `DatabaseLogging`, `CliTools`, `Serilog`. The target-group order option is `TargetMigrationOrder` (`FileByFile` / `TargetByTarget`); `TargetGroupMigrationOrder` on a product orders the groups. Arrays whose elements carry an `Alias` merge by alias across the `appsettings*.json` hierarchy (ADR-021).
 - **Placeholders**: `{ENV:VAR}` in configuration and CLI values, `{CFG:SchemaName}` / `{CFG:TableBaseName}` in SQL templates. Runtime values are ADO parameters.
 - **SQL templates** are `Content` files, not embedded resources. Result contract is `SELECT 'code[,code...],message'`; the first code comes from `Shared/Constants/TemplateResultCode.cs`, further integer codes are template-specific.
-- **Options validation** uses DataAnnotations plus `Ray*Attribute`s in Core; enum options are strings resolved through `ParsedEnumOption<T>`. Cross-field rules live in the `Validation` project and are shared with the wizard.
+- **Options validation** uses DataAnnotations plus `Ray*Attribute`s in Core; enum options are strings resolved through `ParsedEnumOption<T>`. Cross-field rules live in the `Validation` project and are shared with the wizard. Options built in memory get the same treatment from `RayMigratorOptionsPreparer`.
+- **Hosting seam**: a host creates one `MigrationContext` per run through `IMigrationContextFactory` (`Services`), which prepares and activates it; `MigrationService` reads the configuration from the context, not from `IOptions`. Progress goes through `MigrationContext.ProgressObserver`. See `Docs/02-core-concepts/migration-context.md`.
 - **Docs and repo**: `CLAUDE.md` is tracked. `.claude/` (skills, agents, settings) and `CLAUDE.local.md` are local and gitignored. `Docs/arc42/` is mirrored to the GitHub Wiki on every push to `main`.
 - **Release** follows [Docs/11-development/release-process.md](Docs/11-development/release-process.md). Do not tag or push to `main` outside that process.
 

Docs/01-architecture/component-responsibilities.md

@@ -151,16 +151,16 @@ All request types are in `Models/Requests.cs`, all result types are in `Models/R
 | Result DTO | Purpose |
 |-----|---------|
 | `OperationResult` | Abstract base class (Success, ErrorMessage, ErrorCode, Messages, ExecutedAt, Duration). `ErrorCode` is `int?`: negative = SQL template ResultCode, positive = C# backend ErrorCode, null = unclassified. |
-| `MigrationOperationResult` | Result of migration up/down (extends `OperationResult`). Adds `RunId` (Guid), `ProductAlias`, `Environment`, `Operation`, `Result`, `TotalMigrations`, `SuccessfulMigrations`, `FailedMigrations`, `CurrentRelease`, `MigrationResults` (List<MigrationFileResult>). |
-| `MigrationFileResult` | Result of a single migration file execution (FileName, ReleaseVersion, TargetGroup, Success, ErrorMessage, ExecutedAt, Duration). |
+| `MigrationOperationResult` | Result of migration up/down (extends `OperationResult`). Adds `MigrationRunId` (the repository run row, 0 without one), `ProductAlias`, `Environment`, `Operation`, `Result`, `TotalMigrations`, `SuccessfulMigrations`, `FailedMigrations`, `CurrentRelease`, `MigrationResults` (List<MigrationFileResult>). |
+| `MigrationFileResult` | Result of a single migration file execution (FileName, ReleaseVersion, TargetGroup, TargetAlias (null when the entry summarizes all targets of the group), MigrationRecordId, Success, ErrorMessage, ExecutedAt, Duration). |
 | `ValidationResult` | Result of hash validation (TotalFiles, ValidFiles, InvalidFiles, MissingFiles, Issues). |
 | `HashValidationIssue` | Individual hash validation issue (FileName, IssueType, ExpectedHash, ActualHash, Details). |
 | `HashUpdateResult` | Result of hash update (UpdatedFiles, NewFiles, RemovedFiles, UpdatedFileNames). |
-| `BaselineResult` | Result of baseline operation (TargetReleaseVersion, BaselinedFiles). |
-| `MigrationStatusInfo` | Current migration status for a product (CurrentRelease, LastMigrationDate, TotalMigrationsExecuted, PendingMigrations, LastRunResult, TargetGroups). |
+| `BaselineResult` | Result of baseline operation (Environment, MigrationRunId, TargetReleaseVersion, BaselinedFiles). |
+| `MigrationStatusInfo` | Current migration status for a product and environment (Environment, CurrentRelease, LastMigrationDate, TotalMigrationsExecuted, PendingMigrations, LastRunResult, TargetGroups). |
 | `TargetGroupStatus` | Status of a target group (Alias, DatabaseType, CurrentRelease, ExecutedMigrations, LastMigrationDate, Targets). |
-| `MigrationHistory` | Migration run history (Runs: List<MigrationRunInfo>). |
-| `MigrationRunInfo` | Information about a migration run (MigrationRunId, RunId, StartedAt, CompletedAt, Operation, Result, RunMode, InitiatedBy, TotalMigrations, SuccessfulMigrations, FailedMigrations, ToRelease). |
+| `MigrationHistory` | Migration run history of a product and environment (Environment, Runs: List<MigrationRunInfo>). |
+| `MigrationRunInfo` | Information about a migration run (MigrationRunId, Environment, StartedAt, CompletedAt, Operation, Result, RunMode, TotalMigrations, SuccessfulMigrations, FailedMigrations, FromRelease, ToRelease). |
 | `FixIssuesResult` | Result of Fix command (WasSimulated, OrphanedRunsFound, OrphanedRunsFixed, OrphanedRuns: List<OrphanedRunInfo>). |
 | `OrphanedRunInfo` | Information about an orphaned MigrationRun entry (MigrationRunId, Environment, EnvironmentId, StartedAt, MinutesRunning, MigrationRunModeId, WasFixed). |
 

Docs/01-architecture/dependency-injection.md

@@ -127,7 +127,7 @@ Services created per operation:
 
 ### Options Pattern
 
-Configuration sections are bound to strongly-typed options classes with DataAnnotations validation:
+Configuration sections are bound to strongly-typed options classes with DataAnnotations validation. The binding is the CLI pipeline's way of loading and validating the files; the services never consume `IOptions<RayMigratorOptions>`. They read the configuration of the current run from `MigrationContext.RayMigratorOptions`, which the context factory receives from the pipeline (`CreateFromValidatedOptions`) or prepares itself for a host that built the options in memory (`Create`, see [MigrationContext](../02-core-concepts/migration-context.md#hosting-the-engine-in-process)).
 
 ```csharp
 // Registration with validation (JSON mode)
@@ -364,30 +364,22 @@ public class TemplateExecutor
 }
 ```
 
-### TemplateCache (Decoupled from MigrationContext)
+### TemplateCache (Decoupled from MigrationContext and from the options)
 
 ```csharp
 public class TemplateCache
 {
-    private readonly RayMigratorOptions? _options;
     private readonly bool _revealSensitiveData;
     private readonly ILogger _logger;
 
-    public TemplateCache(
-        IOptions<RayMigratorOptions>? options,
-        bool revealSensitiveData,
-        ILogger<TemplateCache> logger,
-        bool validateConfiguration = true)
+    public TemplateCache(bool revealSensitiveData, ILogger<TemplateCache> logger)
     {
-        _options = options?.Value;
         _revealSensitiveData = revealSensitiveData;
         _logger = logger;
         Initialize();
-        if (validateConfiguration && _options != null)
-            ValidateConfigurationAgainstTemplateCache(_options);
     }
 
-    /// Can be called on-demand when Products/Repository config becomes available
+    /// Called by the host for every configuration it runs; the CLI pipeline calls it once after the options are bound
     public void ValidateConfigurationAgainstTemplateCache(RayMigratorOptions options) { ... }
 
     /// Gets all available DAL names (e.g. "SqlServer", "PostgreSQL", "MariaDb", "MySql")

Docs/01-architecture/design-decisions.md

@@ -79,14 +79,13 @@ All four files are searched in the directory specified by `--config-dir` (or the
 - `{ENV:VariableName}` - Environment variables
 - `@ParameterName` - SQL parameters (parameterized queries)
 
-### TemplateCache with Deferred Validation
+### TemplateCache without options
 
-**Decision**: `TemplateCache` accepts a `validateConfiguration` parameter (default `true`). When set to `false`, template loading occurs without validating that configured `DatabaseType` values have matching templates.
+**Decision**: `TemplateCache` takes no options. It loads every template at construction, and the host calls `ValidateConfigurationAgainstTemplateCache(options)` for each configuration it runs.
 
 **Rationale**:
-- **Flexibility**: When Products and Repository configuration are loaded from an alternative source, they may not be available at `TemplateCache` construction time. The cache loads all available templates at startup but defers validation until `ValidateConfigurationAgainstTemplateCache(options)` is called explicitly after configuration is loaded.
-- **Reusability**: The same `TemplateCache` class works for different configuration modes without branching logic.
-- **Public validation method**: `ValidateConfigurationAgainstTemplateCache(options)` and `GetAvailableDatabaseTypes()` are public, allowing external callers to validate on demand.
+- **One cache, many configurations**: A host that runs one configuration per request (RayMigrator Studio) builds the cache once per process; a constructor bound to one options object would tie the cache to the first configuration.
+- **Explicit validation**: The CLI pipeline validates once after the options are bound; a host validates whenever a configuration enters. `GetAvailableDatabaseTypes()` stays public for diagnostics.
 
 ## MigrationContext Pattern
 

Docs/01-architecture/patterns.md

@@ -178,15 +178,14 @@ public class TemplateExecutor
 }
 ```
 
-The `TemplateCache` (in `Raycoon.RayMigrator.Core.Templates`) loads and caches all SQL templates at startup. It supports deferred validation via the `validateConfiguration` parameter, which is set to `false` when Products configuration is not yet available at cache initialization:
+The `TemplateCache` (in `Raycoon.RayMigrator.Core.Templates`) loads and caches all SQL templates once per process. It takes no options: the host validates each configuration it runs against the cache explicitly, so one cache serves many configurations:
 
 ```csharp
 public class TemplateCache
 {
-    public TemplateCache(IOptions<RayMigratorOptions>? options, bool revealSensitiveData,
-        ILogger<TemplateCache> logger, bool validateConfiguration = true) { /* ... */ }
+    public TemplateCache(bool revealSensitiveData, ILogger<TemplateCache> logger) { /* ... */ }
 
-    // Called on-demand when configuration becomes available
+    // Called by the host for every configuration it runs (the CLI pipeline once, after the options are bound)
     public void ValidateConfigurationAgainstTemplateCache(RayMigratorOptions options) { /* ... */ }
 
     // Returns all loaded DAL types (e.g. "SqlServer", "PostgreSQL", "MariaDb", "MySql", "Sqlite")
@@ -515,7 +514,7 @@ public abstract class OperationResult
 
 public class MigrationOperationResult : OperationResult
 {
-    public Guid RunId { get; set; }
+    public int MigrationRunId { get; set; }   // MigrationRun.Id; 0 when the run mode writes no run row
     public string ProductAlias { get; set; } = string.Empty;
     public string Environment { get; set; } = string.Empty;
     public MigrationOperation Operation { get; set; }