Skip to content

Preserve SQLite dependencies and fix provider SQL handling - #174

Merged
jogibear9988 merged 19 commits into
masterfrom
codex/migrator-provider-corrections
Sep 22, 2026
Merged

jogibear9988 merged 19 commits into
masterfrom
codex/migrator-provider-corrections

Conversation

@jogibear9988

@jogibear9988 jogibear9988 commented Sep 22, 2026 •

Copy link
Copy Markdown
Member

Builds on #173. Correct provider behavior while preserving the imperative API and custom-provider signatures.

  • Preserve SQLite triggers, independent FK actions and AUTOINCREMENT high-water state during supported rebuilds; validate integrity before commit, restore FK settings, reject unsupported reconstruction, and select native rename/drop when eligible.
  • Preserve caller definitions/arrays, carry decimal precision and scale, escape filtered-index literals, and correct SQL Server schema/default metadata and error propagation.
  • Add independent FK actions with provider-specific restrictions, nullable scalar/content-size APIs, explicit file/resource scripts and SQL Server GO batching.
  • Use explicit SQL Server uniqueness ownership and Oracle legacy-sequence ownership; do not delete unrelated objects by name. Round-trip SQL Server TIME/defaults/values and single-column uniqueness metadata on SQL Server, PostgreSQL and Oracle.

Validation: full eleven-job database/unit matrix and coverage gate passed at bdc8ac3 in run https://github.com/dotnetprojects/Migrator.NET/actions/runs/35737814671, including the live regressions that previously failed. Latest follow-up adds nullable-result/array tests: local build, Unit 79 and SQLite 154 pass; its own PR CI is required before merge.

Fixes #52, fixes #53, fixes #98, fixes #101, fixes #102, fixes #139, fixes #145, fixes #161.

Partial/related: #33 (MATCH policy), #48 (broader schema qualification), #54 (remaining inline quoting), #132 (historical unmarked uniqueness), #134 (unsupported SQLite properties), #140/#141 (explicit legacy cleanup and table-owned triggers), #162 (provider-specific time representations). See the issue audit in the documentation stack; these references do not imply complete resolution.

Stack: #173 → this PR → #175 → #177 → #178. Merge only after review, in dependency order, retargeting the next PR after its base merges. This task does not authorize merging or publishing packages.

PostgreSQL follow-up (7234e9f): parameterized relation-based column, constraint and existence metadata supports qualified/quoted names and the search path; native time-without-time-zone metadata/defaults round-trip as TimeSpan. All three new PostgreSQL regressions passed in run https://github.com/dotnetprojects/Migrator.NET/actions/runs/35742976746 . Remaining database jobs and the coverage gate are checked separately.

Copilot AI lite review requested due to automatic review settings September 22, 2026 12:32

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Unresolved SQLite, Oracle, SQL Server, and cross-provider foreign-key handling issues block approval.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 2 High severity · 3 Medium severity

Open (5)
What changed in this PR

Improves SQLite rebuild preservation and provider-specific SQL handling, with added APIs and regression coverage.

Changes:

  • Preserves SQLite triggers and foreign-key actions during rebuilds.
  • Fixes provider metadata, defaults, escaping, and identifier handling.
  • Adds nullable helpers, embedded-script support, and regression tests.
File Summary
src/​Migrator/​Providers/​TransformationProvider.cs Adds foreign-key action support and script handling.
src/​Migrator/​Providers/​Impl/​SqlServer/​SqlServerTransformationProvider.cs Fixes schema metadata and default removal.
src/​Migrator/​Providers/​Impl/​SQLite/​SQLiteTransformationProvider.cs Improves rebuilds, constraints, triggers, and escaping.
src/​Migrator/​Providers/​Impl/​PostgreSQL/​PostgreSQLTransformationProvider.cs Escapes filtered index literals.
src/​Migrator/​Providers/​Impl/​Oracle/​OracleTransformationProvider.cs Updates literal escaping and identifier validation.
src/​Migrator/​Framework/​ProviderExtensions.cs Adds nullable and resource-script helpers.
src/​Migrator/​Framework/​IForeignKeyActions.cs Defines the independent foreign-key action API.
src/​Migrator.Tests/​Providers/​SQLite/​SQLiteTransformationProvider_AddForeignKeyTests.cs Updates SQLite foreign-key assertions.
src/​Migrator.Tests/​Providers/​Generic/​Generic_ChangeColumnTestsBase.cs Enables default-removal regression coverage.
src/​Migrator.Tests/​ProviderCorrectionTests.cs Adds SQLite rebuild regression tests.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/Migrator/Providers/Impl/SQLite/SQLiteTransformationProvider.cs Outdated
Comment thread src/Migrator/Providers/Impl/SQLite/SQLiteTransformationProvider.cs Outdated
Comment thread src/Migrator/Providers/Impl/SqlServer/SqlServerTransformationProvider.cs Outdated
Comment thread src/Migrator/Providers/TransformationProvider.cs

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Unresolved moderate issues remain in SQLite rebuilds, Oracle identifier validation, and generic foreign-key action handling.

Review effort: Lite
Findings: 2 High severity · 3 Medium severity

Open (5)

Copilot AI review requested due to automatic review settings September 22, 2026 12:45

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Six moderate unresolved findings affect Oracle defaults, nullable conversion, SQLite safety, and unsupported FK update actions.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1 High severity · 3 Medium severity

Open (4)
Resolved since last review (3)

Comment thread src/Migrator/Framework/ProviderExtensions.cs Outdated
Comment thread src/Migrator/Providers/Impl/SQLite/SQLiteTransformationProvider.cs Outdated
Copilot AI review requested due to automatic review settings September 22, 2026 12:52

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Unresolved SQL Server, Oracle, and SQLite provider issues remain.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 2 High severity · 2 Medium severity

Open (4)
Resolved since last review (2)
Previously missed (1)

In code that hasn't changed since last review

Medium severity Handle unsupported Oracle foreign-key delete actions

src/​Migrator/​Providers/​TransformationProvider.cs:1562

The Oracle special case only suppresses ON UPDATE. It still emits ON DELETE RESTRICT or ON DELETE SET DEFAULT for this overload, but Oracle's existing overload emits no action clause and these actions are not valid Oracle DDL. A caller using the new API will therefore get a provider error; add an Oracle-specific override that rejects or normalizes unsupported delete actions and retains its identifier checks.

Comment thread src/Migrator/Providers/TransformationProvider.cs Outdated
Copilot AI review requested due to automatic review settings September 22, 2026 12:59

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Unresolved moderate issues affect FK API exposure, provider action SQL, Oracle limits, and SQLite rebuild fidelity.

Review effort: Lite
Findings: 1 High severity · 2 Medium severity

Open (3)
Resolved since last review (1)
Previously missed (3)

In code that hasn't changed since last review

Medium severity Synchronize Oracle identifier length limits

src/​Migrator/​Providers/​Impl/​Oracle/​OracleTransformationProvider.cs:940

This guard now permits 128-byte Oracle identifiers, but OracleDialect.MaxFieldNameLength is still 30 (src/Migrator/Providers/Impl/Oracle/OracleDialect.cs:72). That leaves the public dialect limit inconsistent with AddTable validation, so callers that rely on the dialect metadata still reject identifiers this change claims to support; update both limits together.

Medium severity Preserve SQLite foreign-key MATCH clauses

src/​Migrator/​Providers/​Impl/​SQLite/​SQLiteTransformationProvider.cs:1538

GetForeignKeyConstraints stores SQLite's PRAGMA match value in ForeignKeyConstraint.Match, but this rebuild DDL writes only OnDelete and OnUpdate. An existing MATCH FULL/PARTIAL foreign key is therefore recreated without its MATCH clause during AddColumn/ChangeColumn and other rebuilds, changing the dependency definition; preserve the clause or reject unsupported matches.

Medium severity Handle Oracle unsupported delete actions

src/​Migrator/​Providers/​TransformationProvider.cs:1562

The Oracle special case only suppresses ON UPDATE, but Oracle inherits this overload while its existing AddForeignKey implementation emits no action clauses. Calls with an unsupported delete action such as Restrict or SetDefault now generate ON DELETE ... SQL that Oracle rejects instead of following the provider's existing behavior; validate/handle onDelete in an Oracle-specific override as well.

Copilot AI review requested due to automatic review settings September 22, 2026 13:17

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot AI review requested due to automatic review settings September 22, 2026 13:22

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot AI review requested due to automatic review settings September 22, 2026 13:24

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot AI review requested due to automatic review settings September 22, 2026 13:26

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot AI review requested due to automatic review settings September 22, 2026 13:51

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Remove Oracle defaults explicitly when changing a column with no requested default, and treat SQL Server DBNull constraint lookups as absent. Preserve SQLite triggers across case-insensitive table lookup and reject unique/FK name collisions consistently. Route independent FK actions through Db2, Informix, and Sybase overrides with explicit update-action validation.

Validation: rebuilt solution; SQLite suite 146 passed including trigger casing and constraint collision regressions. Unit suite run separately. Live Oracle and server coverage remains required in PR CI.
Oracle string changes already rebuild the column without its prior default. Setting DEFAULT NULL before that rebuild exposed the legacy default parser on the temporary column and failed three live Oracle tests. Keep explicit default removal for non-string in-place modifications, which retain defaults otherwise.

Validation evidence: PR 174 Oracle CI run 35729312158 identified all three failing string-change tests; the newly enabled default-removal test itself passed. New live Oracle CI must verify this correction. Informix failure in that run was an image registry timeout, not a test failure.
Carry precision/scale from Column through dialect mapping instead of dropping them in the legacy scalar overload. Retain existing dispatch for definitions without precision. Correct ForeignKeyConstraint helper direction and preserve independently specified actions, rejecting unknown action strings and copying caller arrays.

Validation: rebuilt solution; Unit 72 passed and SQLite 148 passed including decimal data round trips. Added mapping/direction unit regressions and a shared provider test for persisted fractional precision; live provider results remain required. Addresses PR 175 precision review and related normal-API definition defects.
Mark unique constraints created by ChangeColumn with an extended property and remove only those explicitly owned objects on subsequent changes. User-created constraints, including legacy-looking names, remain intact. Clone incoming column definitions, preserve precision and replace defaults without mutating caller objects.

Validation: rebuilt solution and Unit 72 passed. Enabled the previously ignored issue 132 regression and added a live SQL Server preservation/non-mutation test. Live SQL Server CI must pass before this ownership change is considered verified; unmarked historical constraints are intentionally not inferred by name.
Use native DROP COLUMN on SQLite 3.35+ for columns without represented key/index/check dependencies. Preserve the reconstruction path for legacy/dependent cases. Let SQLite reject trigger/view dependencies atomically and propagate the error instead of retrying a lossy rebuild.

Validation: rebuilt solution; SQLite 149 passed. New Microsoft.Data.Sqlite regressions verify unrelated trigger preservation with foreign keys enabled and dependent-trigger rejection without data loss. Existing System.Data.SQLite removal regressions also pass.
Capture sqlite_sequence high-water state before rebuilding and restore it for retained identity columns, including empty tables after deletion. Reject fallback column removal when trigger definitions would need rewriting, before mutating the original table.

Validation: rebuilt solution; SQLite 150 passed. Regression inserts/deletes ID 100, rebuilds the empty table and verifies the next generated ID is 101.
Return values already assignable to nullable scalar T before using Convert.ChangeType, covering Guid and other non-IConvertible structs. Emit ON DELETE before ON UPDATE in both generic foreign key paths. Add a live SQL Server regression that verifies cascade update and set-null delete independently.

Validation: build, Unit 73 passed and SQLite 150 passed. SQL Server behavior is delegated to the PR live database matrix before its review threads are resolved.
Add optional IScriptBatchProvider and an explicit script execution path while keeping ExecuteNonQuery untouched. SQL Server splits standalone GO lines using quote, identifier and nested-comment state. Unsupported SQLCMD directives and GO repetition fail before execution. File and embedded-resource loaders use the same path; custom providers can implement their own splitter without changing ITransformationProvider.

Validation: build, Unit 77 passed, SQLite 151 passed. Tests cover multiline strings/comments/identifiers, rejected directives, real file/resource data loading and a live SQL Server GO regression for PR CI.
Stop SQLite table creation from stripping primary-key flags on caller columns, and stop generic/Oracle changes from clearing caller uniqueness or nullability. Use an internal definition copy that retains subtype state. Regressions reuse a composite-key definition for a second table and verify nullable composite inserts plus unique-flag preservation.

Validation: solution build, Unit 78 passed and SQLite 152 passed.
Move GetColumns_UniqueButNotPrimaryKey_ReturnsFalse from the SQLite-only fixture to the shared metadata fixture as requested by issue #102. Preserve its assertion that uniqueness is distinct from a primary key and does not imply NOT NULL.

Validation: solution build and SQLite 152 passed. Other inherited provider fixtures run in PR CI before the issue is marked fixed.
Map DbType.Time to native TIME for the modern SQL Server dialect, parse time metadata and format bounded TimeSpan defaults invariantly. Preserve the explicitly selected SqlServer2005 dialect's historical DATETIME mapping. Add a live metadata-copy/default/parameter round-trip regression for issue #162.

Validation: solution build and Unit 78 passed. The live SQL Server PR job must pass before claiming provider coverage.
Remove the unconditional table-name-derived DROP SEQUENCE and swallowed exceptions. Oracle continues to clean up its table-owned triggers and native identity objects. Add RemoveTableWithOwnedSequences for caller-owned legacy sequences, validating names and existence before destructive DDL. Failures are observable and unrelated lookalike sequences remain intact.

Validation: solution build and Unit 78 passed. New live Oracle regressions cover preserved unrelated sequences and explicit cleanup of a legacy sequence/trigger pair; provider CI must verify them. Addresses the unsafe legacy cleanup behind #140 and #141.
…ssions

Read single-column UNIQUE constraints in SQL Server, PostgreSQL and Oracle metadata while leaving composite members individually non-unique. Preserve independent nullability and primary-key flags. Correct Oracle flag assignments and bind TimeSpan values with DbType.Time in centralized parameter construction.

Reproduced failures in CI run 35737057890: shared uniqueness metadata failed on three engines and SQL Server TimeSpan insertion was rejected. Local validation: build, Unit 78 passed, SQLite 153 passed. Added a composite-uniqueness regression; live CI must verify the fixes.
Add real SQLite regressions for empty and all-NULL scalar/content-size queries, then populated values. Assert identifier quoting returns a separate array without changing caller input. Replace the obsolete foreign-key TODO with the compatibility distinction between legacy and independent-action overloads.

Validation: solution build, Unit 79 passed and SQLite 154 passed. Covers #53, #98 and #145 without changing legacy return signatures.
…cit legacy adoption

Mark column-level unique constraints created by AddTable and every property-bearing AddColumn path, not only ChangeColumn. Keep explicitly named user constraints independent. Add an explicit adoption API for caller-owned historical single-column constraints, validating their table/column shape and making repeated adoption idempotent.

Validation: solution build and Unit 79 passed before the additional composite-adoption assertion. Live SQL Server tests cover create/add/remove uniqueness, legacy adoption, composite rejection and preserving unrelated user-owned constraints; CI must pass before resolving #132.
…ions

Quote reserved or non-simple constraint identifiers as single atoms with escaped closing delimiters. Route SQLite inline UNIQUE/CHECK, SQL Server nonclustered keys and Informix/Sybase constraints through identifier helpers. Delegate Oracle's legacy FK overload to the independent-action path so CASCADE is honored, and reject unsupported deletion actions explicitly.

Validation: build, Unit 79 passed, SQLite 155 passed. SQLite regression verifies enforcement with spaced constraint names and reserved column names; live Oracle cascade deletion regression is included for CI.
Normalize only files already changed by this stack to each file's master line-ending convention, reducing unrelated review noise without changing source behavior.
… trips

Replace public-only existence and column/constraint lookup with parameterized PostgreSQL relation resolution. Keep schema and catalog case exact, honor the connection search path, scope constraint lookup to the requested table, and order returned columns. Parse the native time-without-time-zone type and literal defaults.

Add live regressions for same-named tables across schemas, quoted names containing apostrophes, search-path resolution and TimeSpan defaults/binding. Local solution build, 79 unit tests and 159 SQLite tests passed; PostgreSQL verification requires PR CI.
Copilot AI review requested due to automatic review settings September 22, 2026 18:12
@jogibear9988
jogibear9988 force-pushed the codex/migrator-provider-corrections branch from 7234e9f to a09a74a Compare September 22, 2026 18:12

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Base automatically changed from codex/migrator-runner-safety to master September 22, 2026 19:10
@jogibear9988
jogibear9988 merged commit 4749f5d into master Sep 22, 2026
12 checks passed
@jogibear9988
jogibear9988 deleted the codex/migrator-provider-corrections branch September 22, 2026 19:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment