Skip to content

Support non-key filtered indexes, fluent fallback modes and metadata round-trips - #200

Open
jogibear9988 wants to merge 1 commit into
masterfrom
codex/filtered-index-support
Open

jogibear9988 wants to merge 1 commit into
masterfrom
codex/filtered-index-support

Conversation

@jogibear9988

Copy link
Copy Markdown
Member

Fixes #199.

A unique index on IpaUserIdentifier filtered by IpaUserIdentifier IS NOT NULL AND Archive = 0 previously failed because shared validation required every filter column to be an index key, and null filter values were unsupported. SQL Server, PostgreSQL and SQLite now support this definition through both the classic and fluent APIs, and can read back the supported index definition for recreation.

Changes

  • Validate filter columns against table columns and render null/DBNull.Value comparisons as IS NULL or IS NOT NULL.
  • Add Index.UnsupportedFilterBehavior with UnsupportedIndexFilterBehavior.Throw (default) and Ignore, exposed through fluent .OnUnsupportedFilter(...).
  • Apply the policy to providers without implemented filter support, the SQL Server 2005 dialect, and unsupported cases of Oracle's existing expression emulation. Supported providers still apply filters in Ignore mode; other invalid options and database errors are not suppressed.
  • Share filter formatting and catalog parsing. Read back null checks, non-key predicates, quoted identifiers and escaped strings, including strings containing AND.
  • Preserve keys, included columns and flags through metadata and fluent snapshots. Scope PostgreSQL metadata to the requested table, preserve column order, and emit INCLUDE before WHERE.
  • Match SQLite index SQL by exact name and preserve filters through table reconstruction. Test recreating an index from its read-back definition through classic and fluent APIs.
  • Update the manual, generated documentation and runner guide.

Usage

migration.Create.Index("UX_ActiveUsers").OnTable("Users")
    .WithColumns("IpaUserIdentifier").Unique()
    .WithFilter(
        new FilterItem { ColumnName = "IpaUserIdentifier", Filter = FilterType.NotEqualTo, Value = null },
        new FilterItem { ColumnName = "Archive", Filter = FilterType.EqualTo, Value = 0 })
    .OnUnsupportedFilter(UnsupportedIndexFilterBehavior.Throw);

Read back with Database.GetIndexes("Users") or Schema.Table("Users").Indexes() in fluent migrations.

Behavior and limits

Ignore creates an unfiltered index. For a unique index, uniqueness then applies to all rows. The fallback policy is an authoring option and is not stored in database metadata.

Oracle retains its limited non-unique, key-column expression emulation; expression read-back as FilterItems remains unsupported. SQL preview still rejects filtered indexes, including Ignore mode. Predicates that cannot be represented by FilterItems, such as OR, fail explicitly.

Validation

Rebased onto current master and verified:

  • dotnet build Migrator.slnx --no-restore -v quiet: passed.
  • Unit suite: 799 passed, 0 skipped.
  • SQLite suite: 421 passed, 0 skipped.
  • Documentation generation consistency, local links, anchors and paired sample structure: passed.
  • Diff whitespace check with the repository's CRLF content accounted for: passed.
  • Live SQL Server/PostgreSQL round-trip tests added for CI; not run locally because the servers are unavailable. Local SQL generation and catalog-reader regression tests passed.

…round-trips

Fixes #199: allow SQL Server, PostgreSQL and SQLite filter predicates to
reference table columns outside the index keys. Render null and DBNull
comparisons as IS NULL/IS NOT NULL and escape string filter literals.

Add UnsupportedIndexFilterBehavior.Throw (default) and Ignore to Index,
with OnUnsupportedFilter(...) in the fluent builder. Unsupported providers
can explicitly create an unfiltered index; supported filters remain active
and other invalid options or database errors are not suppressed. Honor the
policy for SQL Server 2005 and Oracle's limited expression emulation too.

Share predicate formatting and catalog parsing across native filtered-index
providers. Preserve null checks, quoted identifiers, escaped strings and
conjunctions when reading index definitions. Keep included columns before
WHERE in PostgreSQL, preserve key order and scope metadata to the requested
table. Match SQLite index definitions by exact index name and preserve the
filter across table rebuilds. Carry the fallback policy and metadata through
fluent snapshots, including definitions with null included-column arrays.

Add regression coverage for the reported unique-index example, unsupported
provider modes, classic/fluent round-trips, catalog metadata, SQLite rebuilds
and reuse of read-back definitions. Update the generated manual and runner
guide, documenting unfiltered uniqueness, preview and Oracle limitations.

Validation after rebasing onto current master:
- Solution build succeeded.
- Unit suite: 799 passed; SQLite suite: 421 passed.
- Final filtered-index/fluent checks: 59 passed.
- Documentation generation, consistency and link/sample-structure checks passed.
- Live SQL Server/PostgreSQL round-trip tests added for CI; servers unavailable locally.
Copilot AI lite review requested due to automatic review settings September 25, 2026 16:29

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Filtered indexes - throws

2 participants