Skip to content

[Compatibility]: MySQL functional indexes on JSON expressions are unsupported #28053

Closed

Description

@Ariznawlll

Description

MatrixOne accepts the CREATE INDEX syntax up to the expression key part, then rejects a MySQL-supported functional index over a JSON extraction expression. This prevents indexing JSON predicates without adding a separate generated column to the schema.

Environment

  • Branch: main
  • Commit: 95d04586eb80864499d0fde27aeb912b463ae4cd
  • Deployment: local single-CN MatrixOne
  • MySQL control: isolated MySQL 8.0 socket instance

Steps to reproduce

CREATE DATABASE json_functional_index;
USE json_functional_index;
CREATE TABLE docs(id INT PRIMARY KEY, doc JSON);
CREATE INDEX idx_kind ON docs ((CAST(doc ->> '$.kind' AS CHAR(20))));

Actual behavior

MatrixOne returns:

ERROR 20101 (HY000): internal error: unsupported index which using expression as keypart

The table remains unchanged.

Expected behavior

The same DDL is accepted as in MySQL 8.0. After inserting JSON documents, an equality predicate on the identical expression can use the functional index.

Stability and controls

  • Reproducer: 3/3 MatrixOne runs returned the same error.
  • MySQL 8.0 control: created idx_kind; EXPLAIN SELECT id FROM docs WHERE CAST(doc ->> '$.kind' AS CHAR(20)) = 'alpha' selected idx_kind.
  • MatrixOne workaround control: a stored generated column kind AS (doc ->> '$.kind') plus INDEX(kind) works and produces Index Table Scan; it requires a schema change and is not syntax-compatible with MySQL functional indexes.

Code analysis

pkg/sql/plan/build_index_util.go:159-163 explicitly rejects every expression key part in checkIndexKeypartSupportability with the reported error.

Regression coverage

After implementation, add deterministic DDL/DML/EXPLAIN coverage for a JSON expression key part to the index BVT suite.

Activity

  1. added this to the MO-202609 milestone on Sep 3, 2026
  2. added theissue type on Sep 3, 2026
  3. added
    severity/s1High impact: Logical errors or data errors that must occur
    and removed on Sep 4, 2026
  4. VioletQwQ-0 commented on Sep 4, 2026

    @VioletQwQ-0
    Collaborator

    已完成 triage。该问题具有稳定复现、MySQL 8 对照和明确代码入口,并已进入当前里程碑,因此从 needs-triage 提升为 severity/s1。

    首期范围如下:

    • 支持普通持久表上的非唯一、单表达式、普通 BTREE 功能索引。
    • 支持 CREATE TABLE 内联定义、CREATE INDEX 和 ALTER TABLE ADD INDEX。
    • 表达式必须是确定性的行内标量表达式,返回现有普通索引可接受的类型。
    • 首期优化器只匹配语义完全相同的等值谓词,并保留原始谓词作为 residual 校验。
    • 暂不支持 UNIQUE、复合或混合 key part、多值 JSON、前缀、ASC/DESC、临时表和 cluster table。cluster table 支持在 fix(plan): route cluster table inserts through multi-update #28088 合入后单独扩展。

    实现采用“一条功能索引对应一个内部隐藏 generated column”的方式,复用现有 generated-column DML 计算和普通二级索引维护;隐藏列及其索引必须原子创建、回填和删除,且 SHOW CREATE TABLE、SHOW INDEX、clone、checkpoint/restore 不得暴露内部列名。

    合入依赖为:

    1. 先合入功能索引设计 RFC。
    2. 合入 [Compatibility]: COPY ALTER cannot combine ADD COLUMN and ADD INDEX in one statement #28052 的 generated-column + index 原子 DDL 基础能力。
    3. 合入 fix(json): preserve plain text in JSON_UNQUOTE #28090,固定 JSON_UNQUOTE / ->> 的文本类型和 charset 语义。
    4. 最后合入 [Compatibility]: MySQL functional indexes on JSON expressions are unsupported #28053 的完整实现 PR。

    实现 PR 使用 issue #28053 关联,不自动关闭 Issue。合入后转入 phase/testing,由 QA 在单 CN、多 CN、重启及备份恢复环境给出明确结论后再决定关闭。

  5. Ariznawlll commented on Oct 22, 2026

    @Ariznawlll
    CollaboratorAuthor

    Current-main data point (dev 3a1e42b430, 2026-10-22): this rejection now has a narrower, misleading surface worth knowing for the fix.

    Non-JSON functional indexes work end-to-end on the same build — CREATE INDEX idx_lower ON fi((lower(name))) creates, maintains, and is used both by point lookups and by EXPLAIN (dual filter + Index Table Scan), with UPDATE consistency and SHOW CREATE round-trip preserved.

    The JSON expression form specifically dies at DDL time with:

    CREATE TABLE ji (id INT PRIMARY KEY, d JSON);
    CREATE INDEX jidx ON ji((CAST(d->'$.k' AS UNSIGNED)));
    -- ERROR 20105 (HY000): not supported: functional index expression must have
    --                      session-independent semantics

    Two observations:

    1. The message is imprecise: CAST(d->'$.k' AS UNSIGNED) is fully session-independent (pure function of the row); whatever check produces this error is misclassifying the JSON cast/extract expression — likely a whitelist of permitted functional-index expressions that includes string functions but not JSON arrow/cast nodes.
    2. The queries that such an index would serve still work unindexed (correct rows: WHERE CAST(d->'$.k' AS UNSIGNED) = 12 returns the right row; UPDATE-consistency checks pass), so this is purely "cannot create the acceleration index", consistent with this issue's compatibility scope.

    Regression checks for the eventual fix: (a) DDL acceptance of the CAST form, (b) index actually used by the CAST(...)=N predicate per EXPLAIN, (c) post-UPDATE index consistency — same three probes that currently pass for lower() on this build.

  6. XuPeng-SH commented on Oct 25, 2026

    @XuPeng-SH
    Contributor

    QA handoff: bounded JSON functional-index behavior on pinned main

    Tested the original issue DDL against the isolated local MatrixOne service:

    • Source: aeaa273daf588a9988ba423915d3ae6b8c82124a; running git_version() returned aeaa273daf.
    • Binary SHA-256: 450363d5ec5e693bdd7ded44565b3b0ab1addcb807d3f9569c5d5eedeb1486e4.
    • One local CN, MySQL wire protocol, 5,003 deterministic synthetic JSON rows (seed 20261025).
    • CREATE INDEX idx_kind ON docs ((CAST(doc ->> '$.kind' AS CHAR(20)))) succeeded and persisted in SHOW CREATE TABLE.
    • Unforced EXPLAIN selected Index Table Scan on docs.idx_kind; indexed and IGNORE INDEX result sets matched the independent row oracle.
    • UPDATE and DELETE maintained the index: matching IDs changed from [420] to [0, 420], then [0].
    • The deliberately unbounded JSON-text index was rejected with 20105 and left no index metadata.

    The temporary database was dropped and the owned service was shut down cleanly. This verifies the issue's bounded CHAR(20) compatibility case on one local CN; it does not cover multi-CN, restart, backup/restore, or release deployments. Keeping the issue open for tester disposition.

  7. slyunnn commented on Oct 26, 2026

    @slyunnn
    Collaborator

    Regression and extended verification for #28053 (2026-10-26)

    Verification Environment

    • MatrixOne latest main: 861a2c4cb68961d8b6b01f18e0aa6ac52a3a505f — Fix decimal rounding and MySQL text-to-float casts (#31191). The running binary reported git_version() = 861a2c4cb6.
    • Local topology: 2 CN / 1 TN / 1 LogService / Proxy; built from the pinned source with Go 1.27.1.
    • Proxy 16833, CN1 26151, and CN2 26152 were reachable. The original DDL and plan checks ran through Proxy; the indexed lookup was also run directly through Proxy, CN1, and CN2.
    • Client: MySQL CLI over TCP using the MySQL wire protocol.

    What Was Covered

    • OK Reproduced the issue's bounded JSON-text functional index DDL: CREATE INDEX idx_kind ON docs ((CAST(doc ->> '$.kind' AS CHAR(20)))) succeeded. SHOW INDEX exposed the expression, and EXPLAIN selected Index Table Scan on docs.idx_kind.
    • OK Inserted matching/non-matching JSON rows; the indexed equality lookup returned the expected row.
    • OK Verified index maintenance: after UPDATE, the match moved to the new JSON value; after DELETE, it no longer matched.
    • OK The same indexed lookup returned the expected row through Proxy, CN1, and CN2.
    • OK The unbounded JSON-text expression boundary was rejected with ERROR 20105.
    • Not covered Full distributed BVT rerun, restart, backup/restore, and mixed-version rollout.

    Findings

    The original bounded JSON-text functional-index scenario is fixed on current main. The narrow fix is merged as PR #28274, commit 8f9c50468ea06cd52554f2a5ef94cfb137d930f6 — it permits bounded JSON-to-text expressions while retaining rejection of unsupported/unbounded forms.

    MOTR / BVT Coverage

    No new MOTR case was added: this run supplied direct multi-CN SQL evidence, and the implementation PR already added the focused distributed SQL regression. The full BVT suite was not rerun.

    Final Conclusion

    Fixed for the issue's bounded JSON-text functional-index case on the pinned latest main. Large-data regression is not needed: this is an expression-eligibility, index-plan, and DML-maintenance behavior, not a row-count or resource-boundary issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

kind/bugSomething isn't workingseverity/s1High impact: Logical errors or data errors that must occur

Type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions