Skip to content

Fixes 26509: Entity Reference custom property filters return no results in Advanced Search - #33253

Open
anuj-kumary wants to merge 2 commits into
mainfrom
fix/custom-property-entity-reference-search
Open

Fixes 26509: Entity Reference custom property filters return no results in Advanced Search#33253
anuj-kumary wants to merge 2 commits into
mainfrom
fix/custom-property-entity-reference-search

Conversation

@anuj-kumary

@anuj-kumary anuj-kumary commented Sep 13, 2026

Copy link
Copy Markdown
Member

Describe your changes:

Advanced Search filters built on an Entity Reference custom property never returned results — the asset was indexed, the property was populated, and the filter still came back empty every time.

There were two independent causes. Either one on its own was enough to break the filter, which is why fixing only the first did not make the symptom go away.

1. The filter and the search index disagreed about where the value lives.

When a custom property holds a reference to another entity, the index stores the reference's parts separately — its name, its fully qualified name and its display name each land in a different place. The query builder shows the display name in the UI and offers display names in the value picker, but always looked the chosen value up against the name. Those two are only the same string by coincidence, so the comparison failed for any reference where they differ.

2. The offered values and the stored values differ in letter case.

The value picker is populated from a source that lower-cases everything it returns, while the index preserves the original capitalisation. The comparison was exact and case-sensitive, so a reference whose display name was not already entirely lower-case could never be matched — including the one the user had just selected from the list.

The fix

The filter now reads the same part of the reference that the UI is displaying, and compares it ignoring case. Reloading a previously saved filter was updated to stay in step with the new query shape, so saved queries still open correctly in the builder.

No backend or schema changes; this is entirely in the search-filter layer.

Type of change:

  • Bug fix

High-level design:

N/A — small change.

Tests:

Use cases covered

  • Filtering on an Entity Reference custom property by display name returns the matching assets.
  • Filtering by the reference's name or fully qualified name reads the correct stored value.
  • A reference whose display name is capitalised is matched by the value offered in the picker.
  • A saved filter containing an Entity Reference condition reloads into the builder unchanged.
  • Every other custom property type continues to resolve to the same stored value as before.

Unit tests

  • I added unit tests for the changed logic.
  • Added cases covering each part of a reference a filter can name, and the case-insensitive match.
  • Added a contract test that pins, for every supported custom property type, which stored value the filter resolves to. The pre-existing round-trip coverage could not catch this class of bug because it only ever proved the filter agreed with itself; the new test compares against what the indexer actually writes.

Backend integration tests

  • Not applicable (no backend changes).

Ingestion integration tests

  • Not applicable (no ingestion changes).

Playwright (UI) tests

  • Not added. The failure is in how the filter is built rather than in the interaction, and it is covered by the unit tests above.

Manual testing performed

  1. Seeded a custom property of every supported type on the Table entity and populated all of them on a sample asset, including an Entity Reference pointing at a user whose name, display name and capitalisation all differ.
  2. Confirmed the values were indexed as expected.
  3. Reproduced the original failure on the unfixed build — the filter returned zero results for a value picked straight out of the dropdown.
  4. Verified the fixed build returns the asset for the same selection, and re-checked the originally reported scenario.
  5. Re-ran the filters for the remaining custom property types to confirm none of them changed behaviour.

UI screen recording / screenshots:

To be attached.

Checklist:

  • I have read the CONTRIBUTING document.

  • My PR title is Fixes <issue-number>: <short explanation>

  • My PR is linked to a GitHub issue via Fixes #<issue-number> above.

  • I have commented on my code, particularly in hard-to-understand areas.

  • For JSON Schema changes: not applicable.

  • For UI changes: I attached a screen recording and/or screenshots above.

  • I have added tests and listed them above.

  • I have added a test that covers the exact scenario we are fixing.

Advanced Search filters built on an Entity Reference custom property never
matched anything. Two independent causes:

- The filter looked the chosen value up against a different part of the
  reference than the one the UI offers, so the comparison only ever
  succeeded when those two happened to be the same string.
- The values offered in the picker are lower-cased at source, while the
  index preserves the original capitalisation, and the comparison was
  case-sensitive. Any reference not already entirely lower-case could
  never be matched.

The filter now reads the part of the reference the UI is showing and
compares it ignoring case. Reloading a saved filter was updated to match,
so existing saved queries still open correctly.
@anuj-kumary anuj-kumary added the safe to test Add this label to run secure Github workflows on PRs label Sep 13, 2026
@anuj-kumary anuj-kumary self-assigned this Sep 13, 2026
@anuj-kumary anuj-kumary added the skip-pr-checks Bypass PR metadata validation check label Sep 13, 2026
@github-actions

Copy link
Copy Markdown
Contributor

❌ PR checklist incomplete

This PR cannot be merged until the following are addressed on its linked issue:

  • No GitHub issue is linked. Link an issue in the Development section of the PR (or add Fixes #12345 to the description). For a same-org cross-repo issue, add Fixes open-metadata/<repo>#123 to the description.

The fields live on the linked issue in the Shipping project (open the issue → right sidebar → Projects). After you set them, re-run this check (or push a commit) — issue/project changes do not re-trigger it automatically.

Maintainers can bypass this check by adding the skip-pr-checks label.

@github-actions github-actions Bot added the UI UI specific issues label Sep 13, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Jest test Coverage

UI tests summary

Lines Statements Branches Functions
Coverage: 72%
72.77% (101630/139652) 57.49% (61449/106879) 58.72% (20155/34323)

@sonarqubecloud

Copy link
Copy Markdown

@github-actions

Copy link
Copy Markdown
Contributor

✅ Playwright Results — workflow succeeded

Validated commit 0e5d3dee4e017301537b08b1f854cf2edaa4b87e in Playwright run 34740306849, attempt 1.

✅ 4496 passed · ❌ 0 failed · 🟡 10 flaky · ⏭️ 1 skipped · 🧰 0 lifecycle flaky

Performance

Blocking targets: ✅ met · Optimization targets: 🟡 in progress

Shard-job maxima below are not the full workflow wall time; the linked run includes build, fixture, planning, and reporting.

🕒 Full workflow signal wall (to summary) 51m 36s

⏱️ Max setup 5m 18s · max shard execution 20m 9s · max shard-job elapsed before upload 23m 12s · reporting 24s

🌐 217.92 requests/attempt · 2.24 app boots/UI scenario · 39.77% common-shard skew

Optimization targets still in progress:

  • Common shard skew was 39.77% (convergence target: at most 15%).
  • Browser traffic was 217.92 requests per attempt (convergence target: fewer than 200).
  • Application boot ratio was 2.24 per UI scenario (10659 boots / 4765 scenarios; convergence target: at most 1).
Shard Passed Failed Flaky Skipped Lifecycle failed Lifecycle flaky
✅ Shard advanced-search-01 130 0 0 0 0 0
✅ Shard chromium-01 138 0 0 0 0 0
🟡 Shard chromium-02 157 0 2 0 0 0
✅ Shard chromium-03 152 0 0 0 0 0
✅ Shard chromium-04 151 0 0 0 0 0
✅ Shard chromium-05 166 0 0 0 0 0
✅ Shard chromium-06 141 0 0 0 0 0
✅ Shard chromium-07 154 0 0 0 0 0
✅ Shard chromium-08 177 0 0 0 0 0
✅ Shard chromium-09 147 0 0 0 0 0
🟡 Shard chromium-10 129 0 2 0 0 0
✅ Shard chromium-11 126 0 0 0 0 0
🟡 Shard chromium-12 147 0 1 0 0 0
🟡 Shard chromium-13 181 0 1 1 0 0
✅ Shard chromium-14 175 0 0 0 0 0
✅ Shard chromium-15 174 0 0 0 0 0
✅ Shard chromium-16 129 0 0 0 0 0
✅ Shard chromium-17 155 0 0 0 0 0
✅ Shard chromium-18 155 0 0 0 0 0
✅ Shard chromium-19 191 0 0 0 0 0
✅ Shard chromium-20 200 0 0 0 0 0
✅ Shard chromium-21 161 0 0 0 0 0
✅ Shard chromium-22 175 0 0 0 0 0
🟡 Shard chromium-23 167 0 2 0 0 0
✅ Shard chromium-24 147 0 0 0 0 0
✅ Shard chromium-25 145 0 0 0 0 0
✅ Shard data-asset-rules-01 65 0 0 0 0 0
✅ Shard domain-isolation-01 16 0 0 0 0 0
✅ Shard global-state-01 34 0 0 0 0 0
🟡 Shard import-export-01 111 0 1 0 0 0
✅ Shard import-export-02 38 0 0 0 0 0
🟡 Shard ingestion-01 40 0 1 0 0 0
✅ Shard ingestion-02 53 0 0 0 0 0
✅ Shard reindex-01 28 0 0 0 0 0
✅ Shard search-01 12 0 0 0 0 0
✅ Shard search-rbac-01 29 0 0 0 0 0
🟡 10 flaky test(s) (passed on retry)
  • Pages/UserDetails.spec.tsAdmin user can edit teams from the user profile (shard chromium-02, 1 retry)
  • Pages/UserDetails.spec.tsCreate team with domain and verify visibility of inherited domain in user profile after team removal (shard chromium-02, 1 retry)
  • Features/GlobalPageSize.spec.tsPage size should persist across different pages (shard chromium-10, 1 retry)
  • Pages/Lineage/LineageFilters.spec.tsVerify LineageSearchSelect in lineage mode (shard chromium-10, 1 retry)
  • Pages/Lineage/DataAssetLineage.spec.tsverify create lineage for entity - Table (shard chromium-12, 1 retry)
  • Pages/Entity.spec.tsTier Add, Update and Remove (shard chromium-13, 1 retry)
  • Features/AIMode/CustomPropertiesPanel.spec.tsdeletes a custom property via the confirmation modal (shard chromium-23, 1 retry)
  • Pages/Users.spec.tsAdmin soft & hard delete and restore user from profile page (shard chromium-23, 1 retry)
  • Features/MetricBulkImportExportEdit.spec.tsAdmin bulk edits only selected metric rows (shard import-export-01, 1 retry)
  • Features/TestSuitePipelineRedeploy.spec.tsRe-deploy all test-suite ingestion pipelines (shard ingestion-01, 1 retry)

📦 Download artifacts

How to debug locally
# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip    # view trace

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

safe to test Add this label to run secure Github workflows on PRs skip-pr-checks Bypass PR metadata validation check UI UI specific issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant