Add SAP HANA provider with mandatory real-engine CI - #182
Merged
jogibear9988 merged 32 commits intoSep 22, 2026
Merged
jogibear9988 merged 32 commits into
jogibear9988 merged 32 commits into
Conversation
jogibear9988
force-pushed
the
codex/migrator-hana-ci-qualification
branch
from
September 22, 2026 18:12
bb4683e to
551f3d4
Compare
jogibear9988
force-pushed
the
codex/migrator-hana-ci-qualification
branch
from
September 22, 2026 18:17
551f3d4 to
1d79d29
Compare
jogibear9988
changed the title
jogibear9988
force-pushed
the
codex/migrator-hana-ci-qualification
branch
from
September 22, 2026 18:32
1d79d29 to
7046f88
Compare
Member
Author
|
@codex review |
Introduce additive runner options for tag any/all matching, ordered unversioned profiles, maintenance stages, activation and lock leases. Acquire locks before history refresh and keep version history isolated from auxiliary migrations. Preserve per-migration defaults; add no-transaction and verified-provider whole-session execution with callbacks deferred until commit. Validation: rebuilt solution; SQLite 156 tests and Unit 73 tests passed. Behavioral coverage checks rollback boundaries, post-commit ordering, profile history, tag matching and lock release. Native lock implementations and tooling follow separately.
Expose offline and connected SQL generation with opt-in legacy migration capture and explicit rejection of connection access. Add session-owned SQL Server, PostgreSQL and MySQL/MariaDB locks with timeout and release leases. Package a .NET tool for list, status, validation, migrate, rollback, plan and SQL output. Keep Microsoft dependency injection/options/logging dependencies in an optional package and omit sensitive exception/SQL details in tooling output. Validation: solution rebuilt; Unit 77 and SQLite 159 passed. Added DI activation and CLI argument/list checks, legacy preview opt-in/read-only checks, and live independent-session locking tests for the database CI matrix.
A locally installed CLI could list and generate offline SQL but failed to connect because the core factory fallback selected its historical driver assembly. Register each bundled driver's factory explicitly before creating a provider. Validation: rebuilt solution; connected SQLite CLI migrate/status/rollback smoke passed. Added a CLI regression with a disposable file database that verifies version history and final table removal; Unit 83 and SQLite 161 passed.
Omit free-form provider Log/Warn text so SQL and secrets cannot escape through the optional logging adapter. Distinguish CLI parser and known capability/lock errors from arbitrary migration-body failures. Reject InitializeOnce overrides in preview rather than executing initialization hooks. Pass independent initial-history snapshots to Started/Finished logging. Validation: rebuilt solution; Unit 86 and SQLite 167 passed. Added regressions for three migration exception types returning execution exit code 1, secret/brace logging, initialization-dependent preview rejection and stable lifecycle history arguments. Addresses all four initial PR 177 review findings.
Release the lock in a finally block and retain a secondary release exception in the original failure Data. A release failure after successful execution still propagates. Add a regression covering a failed migration and failed lease disposal together. Validation: solution build; Unit 89 passed; SQLite 172 passed.
Add an additive RollbackTo entry point that validates the refreshed execution plan after acquiring the deployment lock. Route the CLI rollback command through it so an empty database and a higher target cannot execute Up migrations. Validation: solution build, Unit 89 passed, SQLite 173 passed; the new connected CLI regression verifies no user table or version is applied. Addresses review 4072146578.
The documentation PR reproduced an Informix registry timeout followed by a successful retry, but coverage downloaded an empty startup-only artifact left by the first attempt. Enable upload-artifact overwrite for each uniquely named database suite so the gate consumes the current attempt's test results. Validation: inspected run 35735397572 attempt 2: all database jobs passed, while duplicate test-results-Informix artifacts (277 and 16046 bytes) caused the missing-suite failure. Fresh CI validates the complete artifact flow.
… locks Add a two-connection runner regression on SQL Server, PostgreSQL, MySQL and MariaDB. Seed the second runner with stale empty history, hold the first inside its migration, start the second acquisition, and verify only one Up executes after history reload. Confirm the final version and that the lock can be acquired again. Coordination uses events instead of timing sleeps. Validation: solution build and Unit 96 passed. The new concurrency cases require their four live provider CI jobs before claiming coverage.
Even when one worker faults, await the remaining worker before dropping the temporary history table or disposing its connection. This keeps the failure path of the concurrency regression deterministic.
Retain the already-validated callback CurrentMigration context, auxiliary-only history preservation and regression tests when replaying the tooling branch on the updated runner/provider bases. These fixes had equivalent patches earlier in the old stack; rebase patch deduplication otherwise omitted their final tooling integration. Preserve the original file encodings and matrix settings. The resulting source tree matches the previously tested bc35e0e exactly; only the comparison and homepage additions already merged on master are new here.
Update README, static homepage and the comparison newly merged to master with effective scopes, fluent operations, preview limitations, transaction modes, profiles, maintenance, locking, CLI and optional DI/logging. Pin source evidence and distinguish under-review source features from released packages. Add a compiled quick-start that verifies read-only preview, migration and automatic reversal. Validation: compiled and ran the SQLite quick-start; packed and locally installed the tool; offline SQL generation succeeded. Inspected homepage at desktop/mobile widths. Connected CLI smoke exposed a missing driver-factory registration and is being corrected in the tooling PR before docs completion.
Pin Migrator claims to 8d8818e and distinguish under-review source APIs from released packages. Document native SQLite column removal, retained autoincrement high-water state, typed fluent coverage and its validation limits. Explain initialization preview rejection and down-only rollback. Record exact local and live CI evidence without attributing earlier results to later commits. Normalize documentation line endings to avoid unrelated diff noise. Validation: solution build and compiled FluentQuickStart preview/migration/reversal passed; homepage previously inspected at desktop and mobile sizes.
…nventory Record all 81 issue identities with verified closures, pending fixes and explicitly incomplete historical reproduction work. Update the source-pinned comparison and runner guide for SQL Server GO scripts, Oracle caller-owned legacy sequences and SQL Server uniqueness ownership. Record the observed Windows native-library path limitation. Validation: build, Unit 96 passed, SQLite 176 passed; compiled quick start passes. Locally packed tool 9.0.0-upgrade-review.1 generated offline SQL, then migrated/status-checked/rolled back SQLite from a short installation path. No package was published.
Pin the comparison to bb88165 and distinguish local validation from pending live checks. Document explicit SQL Server uniqueness adoption, auxiliary-only version preservation and callback context. Record verified Oracle cleanup and metadata CI evidence without closing partial issues. Validation: rebuilt solution; compiled FluentQuickStart preview, migration and reversal passed. Integrated runner validation: 93 unit and 184 SQLite tests passed.
Map 43 historical reports to named passing NUnit cases from master run 35715528132. Record separate source evidence, behavior differences and remaining gaps for the other historical reports. Keep related coverage distinct from complete reproduction; no new issue closures are inferred from broad suite success.
…erage Refresh the pinned source revision, record native lock concurrency evidence for four engines and explain the new PostgreSQL relation lookup and time mapping. Keep broader schema support and original-report reproduction limits explicit. Validation: integrated solution build, 93 unit tests and 184 SQLite tests passed. New PostgreSQL regressions await live PR CI.
Replace pending PostgreSQL validation notes with the actual live regression result. Link the complete database/unit matrix and coverage gate for bc35e0e while retaining the distinction between verified behavior and incomplete audit scenarios. Documentation-only evidence update.
Add primary-key and unique-constraint definitions with ordered columns; rename Unique to UniqueConstraint. Render named keys/checks in complete table definitions, reject malformed keys before DDL and expose fluent equivalents. Add pure dialect constraint rendering and atom-level identifier quoting. Add structured readers for SQLite, SQL Server, PostgreSQL, Oracle and MySQL/MariaDB, with explicit diagnostics for unsupported readers. Replace SQLite regex-based constraint-name recovery with a tokenizer handling quoted identifiers and nested expressions. Keep legacy flag removal as the next isolated step. Document each implemented breaking API change and current reader limitations in the 12.1-to-13 migration guide. Validation: solution build, 93 unit tests and 190 SQLite tests passed; live provider matrix is required on the PR.
The live Oracle matrix reproduced ORA-01745 because :table is a reserved bind identifier. Use lookup_table and lookup_schema consistently in SQL and command parameters for each catalog query. The ordered composite-constraint regression exercises this path on Oracle and the other supported catalogs. Validation: solution build and all 93 unit tests pass locally. Oracle integration validation runs in the PR matrix; the preceding head passed the other ten test jobs.
Carry named primary-key definitions in SQLite table snapshots and emit them when rebuilding instead of deriving key order from legacy column flags. Copy columns before clearing transitional flags, preserve physical column order when altering attributes, and retain identity key names and sequence high-water values. Update rename and explicit primary-key removal paths. Reject removal of a member of a named key before changing the original table, and document the explicit removal sequence in the 12.1-to-13 migration guide. Validation: solution build, 93 unit tests and 192 SQLite tests pass. New regressions verify composite key order, constraint names, persisted rows, duplicate/null enforcement, explicit removal, physical column order and identity sequence continuity.
Remove ColumnProperty, flag-taking constructors and AddColumn overloads, column primary-key membership properties, and the obsolete SchemaBuilder API. Columns expose independent nullability, identity, unsigned and explicit collation attributes; table keys and unique/check constraints remain named ordered objects. Consolidate authoring on MigrationBuilder and update examples and the operation coverage inventory. Render column clauses through ColumnAttribute dialect mappings. Stop implicit unique-constraint ownership and removal during ChangeColumn, remove SQL Server collation inference, and change Oracle columns in place. SQLite preserves named and unnamed constraints independently of column attributes, reports physical column order, distinguishes AUTOINCREMENT, and preserves table uniqueness when removing indexes. Add structured constraint readers for Db2, Firebird, Informix and Sybase for live matrix validation. Document the source/API breaks and concrete replacements in the 12.1-to-13 guide. Convert legacy flag tests to schema and persisted-data assertions. Local validation: complete solution builds; 87 unit and 192 SQLite tests pass with no skips. Live provider validation is required on the stacked PR.
Replace the generic INFORMATION_SCHEMA foreign-key lookup with parameterized native readers for SQL Server, PostgreSQL, MySQL/MariaDB, Db2 and Firebird. Add native Informix and ASE readers, preserving composite column pairing and keeping unique indexes separate from declared constraints in metadata. Render Informix inline constraint names using its suffix syntax. Treat Oracle DEFAULT NULL as absence of a value, including string columns. Make tests request unique constraints explicitly after ChangeColumn and remove unrelated unsigned attributes from PostgreSQL identity/index fixtures. Use the same pure SQLite CREATE TABLE renderer for imperative execution and structured preview, including named identity keys, uniqueness and collation. Verify generated SQL against SQLite rather than only asserting its text. Validation: solution build; 87 unit tests and 193 SQLite tests pass locally. The new composite-FK catalog regression runs in six live database matrix jobs. This follow-up addresses failures found in PR #181 run 35756650482; live database verification is required on the new head before the PR is ready.
…horing APIs Introduce RawSql.Insert for trusted default expressions without changing literal string escaping. SQLite renders the required expression parentheses and preserves typed expressions, escaped literals, GUIDs and numeric defaults through catalog reads and table reconstruction. Replace the new string-only collation property with a typed Collation definition. Both imperative columns and fluent builders accept the same semantic presets or named provider escape hatch. Resolve supported comparisons in SQL Server, MySQL, MariaDB, PostgreSQL and SQLite dialects; reject unmapped requests and non-text columns before execution. Never substitute SQLite ASCII folding for Unicode case-insensitivity. Document expression trust, function availability, provider/version requirements, language and ordering limits, PostgreSQL ICU configuration and metadata changes in the 12.1-to-13 migration guide. Account for MySQL/MariaDB catalog equivalence between unique indexes and unique constraints in the composite-FK regression. Validation: solution build; 87 unit and 195 SQLite tests pass locally. Added live case/accent comparison tests for MySQL/MariaDB and SQL Server, plus SQL Server NEWID defaults in imperative and fluent creation. Live jobs must pass on this head.
The new real SQL Server collation/default regression exposed that SQL Server rejects bracket-delimited collation names in CREATE TABLE. Validate the allowed collation-name token before emitting it without identifier brackets, preserving injection rejection for named inputs and semantic presets. Validation: solution build passes. Regression is the existing case/accent and NEWID test that failed in PR #181 run 35764844526; verify its new CI result.
… metadata
Catalog inspection previously attempted to parse function expressions as CLR
numbers, GUIDs or dates, or returned them as quoted text during schema copying.
Recognize expression defaults before literal-specific conversion and retain them
as RawSql. Skip PostgreSQL identity defaults and preserve existing supported
literal casts. Decode escaped string literals without stripping their contents.
Add a live regression on all three engines that creates LOWER('ABC') as an
expression and as a literal, inspects metadata, recreates a table from its column
definitions, and checks persisted results from imperative and fluent creation.
Validation: solution build passes. PR CI verifies the new round trips against the
real engines; this is a new check beyond the earlier green v13 database matrix.
…nspection Treat a quoted default as a string literal only when the entire expression is one escaped SQL literal. Concatenated defaults such as 'A' || 'B' remain RawSql and retain executable meaning when a table is reconstructed. Stop unquoted DDL tokens before adjacent line/block comments so primary-key keywords and constraint names are not lost when external schemas omit whitespace. Validation: rebuilt solution; all 197 SQLite tests pass, including new behavioral regressions for concatenated defaults and comment-adjacent named keys through a rebuild, persisted values and duplicate-key enforcement.
Read SQL Server nonclustered primary-key metadata from the backing index type. Use a parameterized Oracle foreign-key catalog reader with schema filtering, position-aligned composite keys and independent delete/update action reporting. Copy caller arrays in foreign-key constructors consistently with other keys. Reject Oracle included-column and clustered-index requests that were previously ignored silently. Add the corresponding migration-guide entry so callers can choose an ordinary index or an explicit Oracle-specific operation. Validation: solution build passes. Add live SQL Server nonclustered-key round-trip, Oracle composite-FK/action metadata, and Oracle unsupported-index diagnostics. The provider-specific CI results must pass before this follow-up is accepted.
Compare current FluentMigrator runner projects with our database matrix. Record Redshift, Snowflake and Db2 for IBM i as deferred until real test endpoints and credentials exist; distinguish Db2 LUW and PostgreSQL from those engines and exclude providers dropped upstream. Add an isolated HANA Express qualification job using a pinned official container and the SAP .NET driver. Require real DDL, identity/constraint creation, persisted data, transaction rollback and native catalog queries, with bounded startup, failure artifacts and unconditional container cleanup. This probe makes no provider-support claim; admission requires passing CI and subsequent migration provider integration tests. Validation: qualification .NET project builds locally with no warnings or errors. Actual engine availability is deliberately validated in GitHub Actions.
…ation The first actual GitHub runner attempt pulled the official image successfully but OCI refused SAP's documented kernel.shmmni=524288 setting. Set the namespace limit to Linux's supported 32768 maximum so the container can start. Keep the actual-engine readiness and .NET behavior tests mandatory; no success is inferred from image availability. Evidence: qualification run 35765175072 failed at container creation, before any database tests. The next run must reach the real HANA checks.
The pinned official HANA Express image and SAP .NET driver passed real DDL, data, transaction rollback and catalog access in run 35766200488. Add Hana dialect/provider registration, native type rendering, positional parameters, schema-aware catalog readers, explicit constraints/FK actions, ordinary indexes, column/table alterations, history support and structured create/add preview. Replace the prerequisite-only workflow with a mandatory Hana job in the complete database matrix. Add eight actual-engine tests covering owned/caller connections, imperative/fluent/preview behavior, constraints and metadata, schema qualification, data rollback, restart/downgrade/read-only planning and unsupported capabilities. Keep SAP dependencies in tests/hosts; the core has no driver package dependency. Do not claim transactional DDL, whole-session transactions, native locks, tenant administration or unimplemented specialized schema features. Document the exact provider scope and retain qualification evidence. Other additional FluentMigrator engines remain deferred until real CI infrastructure is available. Validation: solution build; 87 unit and 197 SQLite tests pass locally. The newly mandatory Hana matrix job must validate the provider itself before support is accepted; the prerequisite probe alone is not sufficient.
The first mandatory HANA matrix run passed six of eight scenarios. Resolve the two failures: SAP exposes HanaFactory.Instance with a private constructor, and HANA does not permit LOWER(...) as a column default. Factory discovery now honors the standard public singleton field/property before constructor fallback. Exercise CURRENT_TIMESTAMP raw defaults through imperative, fluent and generated SQL paths, including catalog round trips and persisted timestamp values. Use HANA's complete ALTER column definition for default changes, replace removed defaults with DEFAULT NULL, and emit NULL when relaxing nullability. Extend the actual-engine alteration test to verify both changed/defaultless inserts and NOT NULL -> NULL transitions. This follows SAP's documented ALTER syntax. Validation: solution build and 87 unit tests pass locally. Existing actual-engine run 35767879821 identified the failures; all other database jobs passed. HANA changes require a fresh mandatory matrix run before provider admission.
jogibear9988
force-pushed
the
codex/migrator-hana-ci-qualification
branch
from
September 22, 2026 18:43
7046f88 to
fbe9c58
Compare
… tests SAP removes credentials from HanaConnection.ConnectionString after opening. The owned-connection test incorrectly reused that sanitized property and failed authentication even though the singleton factory was now loaded correctly. Keep the original disposable CI input in the fixture and pass it directly to ProviderFactory.Create. Do not log it or change driver credential-retention settings. All seven other actual-engine scenarios passed in run 35769035270, including imperative/fluent/preview timestamp defaults, metadata, default removal, nullability changes, foreign keys, DML rollback and history restart. The complete matrix must pass on this commit before admitting HANA support.
jogibear9988
changed the title
jogibear9988
marked this pull request as ready for review
September 22, 2026 19:01
Member
Author
|
Final head eabec55 passed all 13 required checks in https://github.com/dotnetprojects/Migrator.NET/actions/runs/35770116342. SAP HANA passed all eight actual-engine scenarios with zero skips; the complete test-assignment audit also passed. The provider is ready for review, with its DDL/locking/CLI limitations documented. No merge or publication performed. |
jogibear9988
added this pull request to stack #179
September 22, 2026 19:09
Base automatically changed from
codex/migrator-v13-column-model
to
codex/migrator-fluent-operations
September 22, 2026 19:11
An error occurred while trying to automatically change base from
codex/migrator-fluent-operations
to
master
September 22, 2026 19:12
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add SAP HANA only with a mandatory test job against SAP's actual engine. The provider is stacked on #181; no extra database is admitted through SQL-string tests, a substitute engine, or skipped credential-gated tests.
Validation: final head eabec55 passed all 13 checks in run 35770116342. The real SAP HANA job passed all eight behavioral scenarios with zero skips. The full matrix also validates the existing ten database families, 87 unit tests, 197 SQLite cases and complete/disjoint test assignment. Earlier actual-engine failures drove the singleton-factory, default SQL and credential-safe fixture corrections; their tests now pass.
Limits: HANA DDL may autocommit; whole-session transactional DDL/native migration locking are rejected. The packaged CLI does not bundle the SAP driver; use the library in a host referencing it. No merge or publication.