Skip to content

[Bug]: Python SQL builder clone turns INSERT/UPDATE/DELETE into SELECT #30876

Closed

Description

@MatrixAdventurer

由 LW 授权的 ChatGPT Personal 智能体提交。

Is there an existing issue for the same bug?

Branch Name

main (no exact upstream dev branch exists; explicitly authorized main fallback).

Commit ID

cf48add

Other Environment Information

Python 3.12.14, SQLAlchemy 1.4.54; actual public SDK builder SQL executed through native SQLite DBAPI in disposable in-memory fixtures. No production database or account was accessed.

Actual Behavior

MatrixOneSQLBuilder.clone() copies SELECT/WHERE state but leaves _query_type at its reset default SELECT and drops INSERT/UPDATE payloads. Consequently cloning an INSERT, UPDATE or DELETE produces a SELECT instead of the intended mutation; executing it leaves the rows unchanged. Extending the cloned INSERT with values() or UPDATE with set() also raises a wrong-operation ValueError.

Expected Behavior

A clone must preserve the original operation and payload while allowing independent mutations/reset. Preserve the existing parameter-object semantics; isolate mutable builder-owned list/row containers without deep-copying arbitrary user values.

Steps to Reproduce

  1. Create an in-memory table with rows (1,"old",0),(2,"keep",0).
  2. Construct an INSERT, bulk INSERT, UPDATE or DELETE with the public builder.
  3. Execute builder.clone().build(), or its parameter-substitution equivalent.
  4. Observe SELECT output and unchanged table state instead of the expected write.
    Initial regression suite against original production: 10 failed /2 passed (two SELECT controls pass). Final 13 regressions cover native SQL results, both output modes, bulk NULL, independent values/set additions/reset, and original bulk-input aliases. All 13 pass with the five-field repair; owning test file 51 passed. Independent review reproduced original/fixed behavior and explicitly Approved the complete final delta.

Additional information

Final candidate: 2 files, +78/-0, including tests; no new dependency/generated source/server/schema/API signature. Complete local SDK offline suite: 1311 passed +86 subtests, three existing warnings. Fatal Flake8, wheel/sdist build and Twine checks pass. These are SDK SQL-generation checks, not live MatrixOne or full online certification; hosted CI remains to be recorded.
Authenticated MatrixAdventurer (id85048713), own fork MatrixAdventurer/matrixone -> matrixorigin/matrixone and push permission were freshly verified. Planned feature branch fix/python-builder-clone-mutations-20261022 only in that fork; no upstream/default ref push. All branch/commit/PR operations and read-only checks are by the signed execution subject. No automatic merge, deployment or issue closure.

由 LW 授权的 ChatGPT Personal 智能体提交。

Activity

  1. XuPeng-SH commented on Oct 22, 2026

    @XuPeng-SH
    Contributor

    QA verification update (2026-10-22): the fix is present on open PR #30877 at head 28ed91a784de1994f797de07c79ac4f99a287474 (not merged yet). The change copies the builder operation and mutation payloads while isolating the builder-owned lists/rows. The reviewer submitted an APPROVED review on this exact head and reports 51 owning-file tests passed; GitHub run 39809700496 shows all six Python SDK offline matrix jobs and CI Required successful. The live MatrixOne/online jobs were skipped. I inspected the diff and found no further actionable issue. I could not rerun pytest locally because pytest and SQLAlchemy are unavailable in this runtime. Added Ariznawlll as tester while retaining the existing XuPeng-SH assignee; this issue remains open pending merge and post-merge verification.

  2. Ariznawlll commented on Oct 23, 2026

    @Ariznawlll
    Collaborator

    测试版本

    • 官方 main:0594cdb955d39afbdd417a1b398534f6d5284d02,2026-10-23(北京时间)。
    • 已复核正文、研发评论和实际修复 diff;#30877 已 MERGED,merge commit eb2168fa473516a5a05538186c5250d48b5ded20 已通过祖先关系检查,包含在上述 main 中。不是对未合入分支的验收。
    • 隔离 macOS arm64,1 CN / 1 TN / 1 LogService、DISK-V2、真实 MySQL TCP。服务二进制 build info 为上述完整 SHA、vcs.modified=false,启动后 git_version() 为 0594cdb955。Python SDK 直接从同一 clean checkout 导入;Python 3.11.4 / SQLAlchemy 2.0.45 / PyMySQL。

    原问题验证

    真实 SDK Client.execute() 执行公共 builder 的 clone().build() 或 clone().build_with_parameter_substitution(),不以 mock SQL 或 SQLite 代替本轮 MO 数据验证。两种入口各覆盖 INSERT、bulk INSERT、UPDATE、DELETE、SELECT 控制,每项均随独立场景重复 3 次。

    初始数据为 (1,'old',0),(2,'keep',0):

    • INSERT clone 实际新增 (3,"新 O'Reilly",1),affected_rows=1。
    • bulk clone 实际新增 (3,'',1),(4,NULL,0),affected_rows=2。
    • UPDATE clone 实际把 id=1 改为 (1,"新 O'Reilly",1),id=2 不变,affected_rows=1。
    • DELETE clone 实际删除 id=1,仅剩 (2,'keep',0),affected_rows=1。
    • SELECT clone 正确返回 (2,'keep',0),全表不变。

    每次还断言生成语句的操作类型,以及 original/clone 的 SQL 和参数内容未被执行修改。新回归最终 3/3、0 skip,3.18 秒。

    负对照:加载修复 merge 的第一父提交中的原始 sql_builder.py 模块,在同一当前服务与 SDK 执行器上运行;四种旧 cloned write 均生成 SELECT、两行原数据不变,3/3。此对照只替换旧 builder 模块,不声称运行了完整旧 SDK 或旧服务。

    Happy Path 覆盖

    1. 两种输出入口覆盖 Unicode、单引号、空字符串、bulk NULL、WHERE 参数顺序、完整行集和实际 affected_rows。
    2. INSERT clone 独立追加第二行,original 后续追加第三行不污染 clone;clone.reset 不清空 original。UPDATE clone 独立追加 flag=7,original 后续追加不匹配 WHERE 不污染 clone;DELETE 同样验证独立 WHERE。均在真实表执行、3/3。
    3. insert_values() 后 clone 拍下 columns/rows 容器状态;随后修改调用方 columns、原 row 和外层 rows,clone 仍插入原本的 (3,'snapshot',1),(4,NULL,0),3/3。
    4. 任意用户参数对象保留 identity,不要求深拷贝;这是纯 Python 对象语义检查,不向数据库发送任意对象。
    5. SDK 参数元组入口保持 Decimal('-123.456789')、微秒 2026-01-02 03:04:05.123456;UPDATE 后 Decimal/二进制列为真正 SQL NULL。二进制 clone payload 另经真实 PyMySQL 参数绑定验证:HEX 为 00FF41、Decimal 为 1.250000、微秒为 .000001,3/3。
    6. 真实 client.session():插入第三行后主动抛本地异常,事务内计数=3、回滚后恢复原两行;另一个正常退出事务提交 id=1 的 label='commit',3/3。

    Unhappy Path 覆盖

    1. original/clone 的无 values INSERT、无 SET UPDATE 均抛 ValueError;表仍为初始两行,3/3。
    2. bulk INSERT 中一条新行、一条重复主键:拒绝且新行不残留;UPDATE 把 id=1 改成已有 id=2:拒绝且两行不变。两种输出入口分别 3/3。
    3. cloned INSERT 对 NOT NULL 的 flag 传 NULL:拒绝且两行不变,两种输出入口分别 3/3。
    4. 每次服务端错误后重新核对完整行、builder SQL/参数不变,并执行 SELECT 1 确认连接继续可用。
    5. 独立数据库采用随机名称,finally 中关闭 SDK pool、删除数据库并查 catalog 确认为 0;任务服务正常退出 0,日志未见 panic/fatal/data-race 特征。未做并发压力、网络断连或进程故障注入,本问题为同步 builder 状态复制错误。

    白盒验证

    修复只增加 _query_type、INSERT columns/每行容器、UPDATE SET columns/values 的复制,符合本 issue 的浅拷贝语义。已有 tests/offline/test_vector_operations.py 51 passed;完整 SDK offline 1310 passed + 95 subtests passed(2.37 秒)。没有新增或修改 MO Go 测试,也没有向 MO 新提 PR。

    特别区分已知相邻缺陷:首次扩展测试通过 SDK Client 传 Python bytes 时返回 1064。独立对照中,未 clone 和 clone 各 3/3 同样失败,表无残留;原生 DBAPI 绑定相同 bytes 成功。该问题已包含在 #30059 的 binary/typed literal 验收范围,修复 #30073 当前仍 OPEN、未合入。因此不重复报 bug、不把它当作 #30877 修复失败,也不声称 SDK 二进制渲染已修复。失败日志保留;MOTR 没有把 1064 写成成功 golden,而是明确通过原生绑定单独验证 clone payload。

    回归测试

    • 独立 MOTR #268,commit 5ca6b3da4332bf296d86ab85328823fd2dd5122e,从官方 main 13a6aad987e93be1883daa54f08a63d4cc5fe5a2 建分支,未追加 select without from clause is not supported #247。
    • 场景 script/14_issue_regression/issue_30876_sdk_clone.py 与手写配对 .out,MOTR_PYTHON_SDK_TESTS=1 时执行真实 SDK;本轮没有 skip。
    • MOTR framework 428 passed + 435 subtests(26.92 秒),11 条测试收集 warning。未运行完整 MOTR 产品/issue suite、全部在线 SDK、跨平台本地矩阵;不把框架 UT 当作产品全套通过。
    • 修复 exact head d042ae4620b837239a67c7e957d53b5017d79673 的 ALL CI 39825997931 SUCCESS,六个 Python/SQLAlchemy offline 作业及 CI Required 成功;online/kernel 作业为 skipped。
    • 新回归的 MOTR CI 39881129940 为 failure:四个作业均 0 steps,实际 annotation 为 The job was not started because an Actions budget is preventing further use.。这是预算阻断、没有执行测试,未记为 CI 通过或产品失败;没有额外触发昂贵 CI。

    测试证据

    • 可重放的独立场景源码。执行入口:MOTR_PYTHON_SDK_TESTS=1 python -m motrlib.runner --config <protected-config> --branch main --repeat 3 -j 1 -v 14_issue_regression.issue_30876_sdk_clone;PYTHONPATH 指向与服务同 SHA 的 SDK。
    • 本地 /tmp/mo-hourly-0047-Yigxc9/ 保存 motr-final.log、首次失败 motr-initial-binary.log、diagnostic.log(旧 builder 及 bytes 对照)、sdk-focused.log、sdk-offline.log/xml、framework.log、service.log 和修复 CI 元数据。输出与公开评论未包含连接凭据。
    • 本验收结论仅针对 [Bug]: Python SQL builder clone turns INSERT/UPDATE/DELETE into SELECT #30876 的 clone 操作/状态复制,不扩大到整个 SDK 参数渲染、GPU、分布式性能或恢复能力。

    测试结论

    测试通过。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions