Repository navigation
[Bug]: Python SDK JSON paths containing quotes break SQL statement boundaries #30765
Description
由 LW 授权的 ChatGPT Personal 智能体提交。
Is there an existing issue for the same bug?
- Searched JSON-path/SDK issues and open PRs, and inspected fix(python-sdk): preserve SQL parameter boundaries and safe literals #30073's changed files. Its parameter renderer does not change sqlalchemy_ext/json_functions.py; no duplicate JSON-path literal fix found.
Branch Name
main (no exact upstream dev branch exists; authorized main fallback applies).
Commit ID
Final PR head: eac18da
Base: 57681d4
Other Environment Information
Python 3.12.14; SQLAlchemy 1.4.54, 2.0.55 and 2.1.6. Deterministic native SQLite DBAPI regression plus actual MatrixOne v4.2.5 service validation. See PR #30766 and the signed evidence comments for commands, fixed commits and validation scope.
Actual Behavior
Six SDK JSON helpers wrap paths using literal_column(f"'{path}'"). A valid path selecting the key owner's, $."owner's", breaks the SQL string boundary and fails parsing. json_extract_string, json_extract_float64, json_set, json_insert and json_replace have the same defect. SQL-looking paths are also emitted as SQL syntax. No successful production exploit or privilege bypass is claimed.
Expected Behavior
Bind paths as ordinary SQLAlchemy literal(path) values so the driver/dialect preserves the exact path without breaking the SQL boundary. Preserve executemany support and dialect-escaped literal_binds compilation.
Steps to Reproduce
The committed offline engine/DBAPI regression yielded 12 failed / 18 passed against the original production module. After the fix, the complete JSON test file passes 50/50, including three executemany cases.
Actual MatrixOne v4.2.5 service regressions independently reproduced six SQL1064 failures before the fix and six passes after it, on each of SQLAlchemy 1.4.54, 2.0.55 and 2.1.6. These are six targeted online tests, not the whole online suite.
Full offline suites passed respectively 1258, 1254 and 1262 tests (+86 subtests each); fatal Flake8, build and Twine checks passed. Auditable fork run: https://github.com/MatrixAdventurer/matrixone/actions/runs/39692412953 .
All six upstream Python SDK matrix jobs and CI Required passed on the final PR head: https://github.com/matrixorigin/matrixone/actions/runs/39692878618 . Path/dispatch-skipped kernel and full-online jobs are not claimed as passes.
Additional information
Delivered PR: #30766 .
Complete product patch: three files, +76/-14; production helpers and existing offline/online tests. Final independent review explicitly Approved the final remote head. The validated and rebased commits have identical complete clients/python trees; evidence reuse and upstream-only changes are documented in the PR.
PR is ready, assigned to XuPeng-SH and review requested from XuPeng-SH; actual metadata was read back. Own-fork branch operations, readiness and assignment/review-request operations were performed by the signed execution subject and documented in the PR. No automatic merge, deployment or issue closure.
All read-only verification and delivery operations in this record are performed by the signed execution subject.
由 LW 授权的 ChatGPT Personal 智能体提交。
Activity
MatrixAdventurer commentedon Oct 21, 2026
由 LW 授权的 ChatGPT Personal 智能体提交。
候选已保存在本人fork独立分支 fix/python-json-path-literals-20261022,固定head 8541db1,base 3b5ccf1,完整tree10056a620c0924fba46a67e5690ec0df734e5b31等于本地固定commit beb1885d0dabb182ad7d5bd3d96c374914950140。只涉及3个产品/回归文件 +76/-14;未写默认分支,未创建上游PR。
独立审查已发现并推动修正:literal_execute会限制SQLAlchemy executemany,因此最终使用普通literal(path),由既有PyMySQL/aiomysql参数处理转义,显式literal_binds编译仍正确;原有批量写入能力保留,并新增三个使用原生SQLite JSON函数的实际双行写入控制。online string提取用例也已改为字符串输入,符合服务端语义。最终JSON单文件50/50通过,完整版本矩阵及在线门禁正在验证;此前中间版本的全量结果不冒充最终版本通过。
下一步在本人fork的独立验证分支运行临时MatrixOne服务,读回六个真实Client.execute回归的修复前失败、修复后通过证据。验证工作流仅为可审计执行工件,不进入产品PR;必要offline/online回归本身均在上述3文件完整补丁内。最终独立复审Approve及所需门禁未完成前不发布正式修复PR。分支创建、连接器commit及只读核验主体同上署名。
MatrixAdventurer commentedon Oct 21, 2026
由 LW 授权的 ChatGPT Personal 智能体提交。
已启动可审计的临时服务验证:run39692412953,验证分支head0149a13782e0edbe919a2575e7c1ada8eb9b3c85。工作流以只读权限、无继承凭据checkout固定产品head8541db1514be735cd20c6153eeff7be6fe0d62da,在独立容器内运行官方MatrixOne4.2.5,并为SQLAlchemy1.4/2.0/2.1分别验证真实六项online RED→GREEN、全量offline与打包。原实现失败的oracle拒绝setup error/skips,避免把环境失败当回归证据。该执行工件仅在本人fork验证分支,不进入三文件产品PR。
最终产品代码保持ordinary literal,SQLAlchemy2.0.55全量1254passed+86subtests、2.1.6全量1262passed+86subtests及fatal flake8已实际通过;最终1.4全量正运行。源码复审三个阻塞已全部修正,独立review的最终交付Approve仍等待在线及矩阵证据。尚未发布上游PR,不声称在线/完整CI通过。分支与工作流创建/更新均由本署名主体执行。
fix(python-sdk): bind JSON path strings safely
MatrixAdventurer commentedon Oct 21, 2026
由 LW 授权的 ChatGPT Personal 智能体提交。
真实MatrixOne验证已完成:run39692412953三组均SUCCESS。独立review已读原始日志确认server为8.0.30-MatrixOne-v4.2.5,原实现六项均因SQL1064引号语法失败、修复后六项均通过,均无setup error/skip。SQLAlchemy1.4.54/2.0.55/2.1.6全量offline分别1258/1254/1262passed,各86subtests;fatal Flake8、wheel/sdist与Twine均通过。独立review对固定已测8541db1明确Approve。此证据仅覆盖六项定向在线回归,不冒充完整在线套件/内核CI。
发布前main前进到57681d4(仅内核backup及UT调度文件,Python SDK与其CI未变),已将相同补丁同步到该最新基线:最终head eac18da,完整tree4117c874feebbb7371a429b1ef432b410cf71094等于本地fc98f4c2307d438c62cc4ba119fd3af9f39988e4。整个clients/python目录tree在已测与最终版本均为b015c54087ab3fd340eeec402a451a3019dc26a2,证据按仓库semantic-validity规则复用;独立review正在读回最终同步head。
旧已测head保存在本人fork archive/30765-before-sync-20261022,源分支只以expected_sha租约更新。最终产品修复仍3文件+76/-14,没有拆分或省略必要测试;验证workflow为独立fork运行工件,不进入产品PR。以上归档/更新操作和只读核验由本署名主体执行。
fix(python-sdk): bind JSON path strings safely (#30766)
XuPeng-SH commentedon Oct 22, 2026
QA follow-up for PR #30871 — merged 2026-10-22
PR #30871 is now merged. Final PR head:
5c082c036105a248fb82dbd2211cada48bfd11ae; merge commit:cf48adddcda41c2ae754904af76b034a83b83163. The final PR diff contains the Python JSON-path implementation and its offline/online regression tests.The exact-head review and retained validation report available before merge are for
dcf46bdbd5a7a9fd65b23f4ea73869380d98ee2c. They report the special-key fix and nested-path coverage, 80 focused offline tests and 43 online JSON tests passing after excluding three failures also reproduced on base. The submitted review isCOMMENTEDwith “Assessment: approve” and is from the PR author, not an independent formalAPPROVEDreview. These test results are reported evidence, not tests run by this QA agent.For the final merged SHA, combined status is empty; the only associated workflow returned is Iceberg Connector E2E and does not verify the Python SDK fix. This environment lacks pytest/SQLAlchemy and a reachable MatrixOne service, so I could not rerun tests on the merged head.
The bracket-access defect appears addressed by the merged change. Keep #30765 open for tester confirmation on main and preserve both existing assignees (XuPeng-SH and Ariznawlll); no duplicate issue or assignment change is needed.
Ariznawlll commentedon Oct 22, 2026
测试版本
mainc232b6d2a1b5a94ac624c4954586569841c1e153(执行前后与 GitHub 官方 main 核对)a538c4bccd367d227440721990a04695d6add842已进入上述 main原问题验证
直接调用 #30766 修复的六类 JSON helper,以带撇号的键
owner's及正确 JSON 路径$."owner's"运行现有test_json_path_quoted_key_roundtrip:6 个参数化在线用例 × 3 轮 = 18/18 成功。对应json_extract、json_extract_string、json_extract_float64、json_set、json_insert、json_replace的原始 SQL 字符串边界缺陷在这些入口上未再复现。本地完整 JSON offline 文件也连续 3 轮各 50/50 通过。但同一 SDK JSON 类型公开的方括号入口仍失败。每轮新建表插入
{"owner's":"ok"},用已引用路径json_extract_string(data, '$."owner\'s"')查询:3/3 返回ok。改为data["owner's"].astext:SDK 生成$.owner's(未按 JSON path 语法引用特殊成员),真实 MatrixOne 3/3 返回错误码 20301:此项不是 1064 字符串边界错误,也未证明 SQL 注入;它是同一 quoted-key 功能经
JSON[key]入口仍不可用。#30766 只为直接 helper 传入已引用路径加回归,未覆盖键名转义。现有 QA 静态提示 指向同一代码路径;本轮补上了真实服务的独立动态复现。Happy Path 覆盖
ok;插入、连接复用和临时表清理成功。Unhappy Path 覆盖
data["owner's"].astext:3/3 稳定返回 20301,未返回应有的ok;对照查询成功,排除了数据不存在、连接或服务不可用。白盒验证
当前 main 的
JSONPathElement构造$.{key},嵌套访问继续直接拼接;本轮 SQLAlchemy 实际对象读回_current_path="$.owner's",与服务端 20301 一致。未运行 Python race;此问题是确定性的路径值错误。回归测试
测试证据
go test ./pkg/tests/issues -run '^TestIssue30765OnlinePythonSDK$' -count=1 -json -timeout=10m,探针内启动真实嵌入集群并逐轮调用 SDK online 六例及方括号/引用路径对照;三轮实际输出均为六点 PASS、控制ok、方括号 20301。测试结论
测试不通过。
MatrixAdventurer commentedon Oct 22, 2026
由 LW 授权的 MOI 智能体提交。
针对本问题完成了客户端侧修复与回归测试,请 @XuPeng-SH 审阅。
根因
clients/python/matrixone/sqlalchemy_ext/json_type.py中JSONPathElement.__init__与__getitem__用 f-string 直接拼接路径(f'$.{key}'、f'{self._current_path}.{key}'),未对成员名做 JSON path 引用处理。服务端pkg/container/bytejson/utils.go的isIdentifier只接受[A-Za-z]、_、$、非 ASCII 字节(≥0x80)与非首位数字;其余成员(owner's、a"b、含.、首字符为数字、空串等)必须以双引号包裹、并按strconv.Unquote兼容转义。因此col["owner's"]生成$.owner's,被服务端以 20301(invalid json path)拒绝。修复(3 个文件,+100/-2)
_quote_json_member():逐字符复刻服务端isIdentifier判定,合法标识符裸输出,其余双引号包裹并转义("、\、控制字符),与path.go经strconv.Unquote反解的语义对齐;空串输出""。__init__与__getitem__的路径生成均经该函数渲染。tests/offline/test_json_offline.py:新增TestSpecialCharacterKeys,7 组参数化键的生成路径断言 + 嵌套路径断言(离线可跑)。tests/online/test_json_online.py:新增TestSpecialCharacterKeysOnline,真实执行specifications["owner's"]与嵌套访问(修复前直接 20301)。验证情况(如实)
python -m py_compile通过;_quote_json_member从源码提取后独立执行 10 组用例(owner's/a"b/a\b/x.y/007/中文键/brand/空串/\n/\x7F)+ 嵌套拼接断言,全部通过,输出与服务端isIdentifier及strconv.Unquote语义逐条核对一致(本 Issue 涉及服务端 path literal 化的部分此前已修复,此处不涉及再改服务端);__init__/__getitem__中除新函数外无残留裸 f-string 路径拼接。clients/python/tests/offline与在线 gate。完整 patch(仓库根目录
git apply即可)PR 状态说明
当前 GitHub 绑定渠道无推送权限,无法创建分支与 PR;此前实测上游 matrixone 仅有 main 与 release 分支、无 dev 分支,按授权约束不擅自改投 main。待具备推送渠道并确认目标分支后,可按此 patch 立即提交 PR。本评论不合并、不关闭本 Issue。
fix(python): quote JSON object keys in bracket access (#30871)
XuPeng-SH commentedon Oct 22, 2026
已按行为验证(
main@3ad9adbb4c),转测保留1. 修复确实在 main 上:PR #30766(
fix(python-sdk): bind JSON path strings safely)的提交是 HEAD 的祖先,可git merge-base --is-ancestor验证。2. 源码级前后对照(同一文件版本差异):
覆盖
json_extract_string / json_extract_float64 / json_set / json_insert / json_replace五个 helper(另有一处多路径分支同样改为literal(path))。3. 编译级实测(SQLAlchemy 2.0.43,
literal_binds):单引号被正确转义为
'',path 作为完整字面量出现,不再截断语句;对注入形态的路径($."x') or 1=1 --")同样是转义后整体传递,不会跳出字符串边界。4. 回归:本机未跑完整
clients/python/tests/offline/test_json_offline.py(离线套件在本地超过 400s 未结束,我按超时终止,未取得结论),因此不引用该套件的结果。上述结论仅覆盖源码差异与 SQL 生成层面。保留
Ariznawlll作为唯一 assignee 进行回归验证;如果线上(真实连接)执行仍有失败用例,请按 QA 结果转回作者。Ariznawlll commentedon Oct 22, 2026
测试版本
main,完整 commitb74ceced223bcf9ae0007445a3dd0f9ddb0cbc48,执行前及补充验证前均从官方 REST 核对。a538c4bccd367d227440721990a04695d6add842、后续 fix(python): quote JSON object keys in bracket access #30871 mergecf48adddcda41c2ae754904af76b034a83b83163。两 PR 均 MERGED,两者均经git merge-base --is-ancestor确认进入上述 main。本轮在第二次修复合入后才验收,不将前一轮失败当成新修复失败。go version -m为上述完整 SHA 且vcs.modified=false,服务git_version()为b74ceced22;Python SDK 从相同干净源码导入。原问题验证
json_extract、json_extract_string、json_extract_float64、json_set、json_insert、json_replace)使用带撇号路径$."owner's":现有原始 6 个在线参数化用例连续 3 轮通过。两套 SQLAlchemy 环境分别执行,未出现原 SQL1064。JSON["owner's"].astext现在真实返回ok,独立重复 3/3;显式引用路径 helper 同样返回ok。x.y不得误取嵌套x→y,items[0]不得误取数组,特殊键必须返回独立 Python oracle 中对应值。1daa5e459bf5f58a9b21218a1082cb1a5bdf5562,连同一当前服务,显式 helper 3/3 返回ok,方括号 3/3 返回 20301 invalid json path;当前 SDK 两入口均 3/3 返回ok。旧 SDK 失败仅为 RED 控制,不作为新修复失败或重新改派依据。Happy Path 覆盖
*、数组形态、数字开头、控制字符/NUL、中文及非 BMP、SQL 形态文本。六 helper、json_insert已有键保持 7 / 缺失键插入 11、json_set/json_replace已有键更新为 9 均精确断言完整 JSON。7.0(float)、文本child(str)、布尔 True(bool);SQL NULL、JSON null、缺失成员及空结果分别核对,均作为合法边界而非错误场景。Unhappy Path 覆盖
$.[的json_setUPDATE,要求出现 invalid json path;之后逐行完整内容与修改前完全相同,guard 表仍为(42),同连接继续查询 quoted-key 成功。MOTR 三轮均通过,未将错误输出录为成功 golden。test_extract_with_filter、test_filter_with_dict_syntax的 quoted-string 比较得到 0 行而非 1 行;test_json_replace_only_existing期望字符串'499.99',实际为数字499.99。同一服务上使用合入前 SDK 重跑原文件,得到 29 通过 / 同样 3 失败,与研发记录一致。这三项未变化,不是本轮 quoted-key 修复回归;保留原断言,不把它们统计为 PASS。JSONPathElement 仍有 SQLAlchemy cache 警告,本轮未作性能结论。白盒验证
回归测试
a568a7cbe3957a6b63209db8166bae0a7acce646,从官方 MOTR main13a6aad987e93be1883daa54f08a63d4cc5fe5a2建分支,未追加 select without from clause is not supported #247。--repeat 3 -j 1实际 3/3 通过;独立 Python 值 oracle,无错误 golden。MOTR_PYTHON_SDK_TESTS=1且安装匹配 SDK;未启用时 exit 77 明确 SKIP,不算通过;启用后的缺依赖或断言失败不会静默跳过。MOTR 框架离线测试 428/428(26.381 秒),不等于 428 个产品场景。The job was not started because an Actions budget is preventing further use.,是 Actions 预算阻塞,不是测试断言失败,也没有 CI PASS。测试证据
/tmp/mo-30765-merged-IFma5M/:audit.json、motr.log、online_focused_*.log、sdk21_online_*.log、online_full.log、pre_online_full.log、pre_quoted.log、post_quoted.log、sdk_offline.log、motr_offline.log及服务日志。未使用共享 TKE/128/129。本 issue 原始六 helper 字符串边界缺陷与前轮未修好的 bracket 特殊键路径,均在 #30871 已合入的 main 上完成独立真实服务验收及 MOTR 回归;前轮 20301 阻塞已消除。结论仅针对 #30765,不代表完整 JSON online 套件或发布线通过。
测试结论
测试通过。