From e090499557729bc05a89dbce47dc1d0216421d85 Mon Sep 17 00:00:00 2001 From: chamsechan Date: Mon, 14 Sep 2026 23:01:03 +0800 Subject: [PATCH] fix(studio): deliver pipeline composition authoring and acceptance fixes (RFC-0057) --- doc/CHANGELOG.md | 7 + doc/README.md | 1 + .../0057-pipeline-composition-experience.md | 545 ++++++ doc/rfcs/README.md | 1 + src/tools/CMakeLists.txt | 4 +- src/tools/alg_pipeline_tool.cpp | 77 +- src/tools/pipeline_authoring.cpp | 1714 +++++++++++++++++ src/tools/pipeline_authoring.h | 84 + tests/CMakeLists.txt | 1 + tests/tooling/studio_browser_test.mjs | 120 +- tests/tooling/studio_fix_workflow_test.mjs | 107 +- tests/tooling/test_pipeline_studio.py | 1159 ++++++++++- tools/pipeline_studio/README.md | 34 +- tools/pipeline_studio/server.py | 427 +++- tools/pipeline_studio/web/app.js | 436 ++++- tools/pipeline_studio/web/graph.js | 4 +- tools/pipeline_studio/web/index.html | 11 + tools/pipeline_studio/web/styles.css | 16 + tools/pipeline_studio/web/workbench.js | 113 +- 19 files changed, 4653 insertions(+), 208 deletions(-) create mode 100644 doc/rfcs/0057-pipeline-composition-experience.md create mode 100644 src/tools/pipeline_authoring.cpp create mode 100644 src/tools/pipeline_authoring.h diff --git a/doc/CHANGELOG.md b/doc/CHANGELOG.md index e6ff1132..499fb6a2 100644 --- a/doc/CHANGELOG.md +++ b/doc/CHANGELOG.md @@ -1,5 +1,12 @@ # Changelog +## 2026-09-14 Pipeline 编排与部署预检验收修复（RFC-0057） + +- 增加八类原生作者操作、128 动作/4 MiB 有界请求和 `fix-deps` 预览/原地修复；写入失败保留原文件，普通扇出保持已有默认 key，歧义目标和非法请求返回结构化错误。 +- Studio 图修改统一调用原生操作，提供默认绑定重置、独立依赖编辑及单次撤销。校验、运行和保存支持一次应用合法表单；修复使用页内审阅，应用前复核草稿和工具版本。 +- 预检无需执行 Demo，结果随方案或运行设置改变而过期。已有部署文件可显式关联，预检/运行/保存使用相同候选配置，保留无关模型覆盖和输出池；原始 JSON 路径变更提供明确的部署意图选择。 +- 增加短写、请求边界、默认扇出、双文件冲突、部署一致性及真实浏览器回归。RFC 仍分阶段实施，未宣称模型效果或真实开发者体验指标已验收。 + ## 2026-09-14 投产前框架兼容入口与配置收口（第一批瘦身落地，RFC-0056） - **旧入口与兼容别名清理**： diff --git a/doc/README.md b/doc/README.md index 5f7bc0e2..f4735954 100644 --- a/doc/README.md +++ b/doc/README.md @@ -46,6 +46,7 @@ | 需要了解什么 | 入口 | | --- | --- | | 尚待完成的开发者试用与生产验收 | [方案开发者验收计划](plans/solution_developer_acceptance.md) | +| 降低方案编排心智负担的实施顺序与验收 | [RFC-0057：Pipeline 编排体验](rfcs/0057-pipeline-composition-experience.md)（In Implementation） | | 架构与接口为何这样设计 | [RFC 索引](rfcs/README.md)，优先列出进行中的 RFC | | 用户可感知的版本变化 | [Changelog](CHANGELOG.md) | | 特定 RFC 当时的验证证据 | [评审与验收归档](rfcs/reviews/README.md) | diff --git a/doc/rfcs/0057-pipeline-composition-experience.md b/doc/rfcs/0057-pipeline-composition-experience.md new file mode 100644 index 00000000..3879f1f4 --- /dev/null +++ b/doc/rfcs/0057-pipeline-composition-experience.md @@ -0,0 +1,545 @@ +# RFC 0057：降低 Pipeline 编排认知负担的实施方案 + +- **RFC 编号**：0057-pipeline-composition-experience +- **创建日期**：2026-09-14 +- **文档状态**：In Implementation +- **关联分支**：`fix/rfc0057-acceptance` +- **目标版本**：投产前，分阶段交付 +- **负责人 / 作者**：LLM-EdgeFlow contributors +- **评估基线**：`236e03f`；现有生产工具 Catalog v3 +- **关联决策**：承接 RFC-0006、0007、0045、0051；复用 RFC-0052、0055；新增工具作者操作和已有部署文件关联规则。 + +本文同时保留设计与分阶段验收规格。M1a/M1b/M2/M3 已实现并补充验收修复，已有部署文件关联与 +统一候选配置也已落地；高级模型操作、M5/M6 和真实开发者试用仍按第 10 节推进。第 2 节保留 +评估基线，当前使用方法见 [Studio 指南](../../tools/pipeline_studio/README.md)。 + +## 1. 结论、目标与必要范围 + +当前复用模板调参的负担基本可控，从零组合、改接多分支流程、切换真实模型和重新打开部署 +方案的负担偏高。作者需要维护连接的多种表达、理解工具状态，并在多个配置入口之间核对实际 +生效值。本判断来自源码、Catalog/Validator 实测和静态交互审阅，尚无足够的开发者试用数据， +不能据此宣称具体耗时或改善比例。 + +建议保留显式运行时格式，将机械编排工作集中到工具，使普通任务无需手写内部键、依赖数组 +或部署路径。优先完成下面六项，工程实现与实际试用分别验收。 + +| 优先级 | 工作包 | 开发者获得的变化 | +| --- | --- | --- | +| P0 | M1：统一作者图操作 | 用节点和端口连接；工具安全维护 key、引用和依赖，CLI 与 Studio 行为一致 | +| P0 | M2：连续编辑与可读诊断 | 修改后一次动作校验或运行；错误与修复先用节点、字段和影响说明 | +| P0 | M3：运行前部署预检 | 无需先运行 Demo，即可看清实际模型路径、覆盖来源和部署错误 | +| P1 | M4：关联已有部署文件 | 服务重启后可显式重新关联 JSON 与 `.conf`，继续成套编辑保存 | +| P1 | M5：模型与参数渐进展示 | 优先选择模型实例或资产预设，复杂参数保留类型和缺省语义并减少 JSON 编辑困难 | +| P1 | M6：契约说明与任务验收 | 理解必要的数据关系，通过样例完成编排；不要求先阅读 Core | + +本轮不改变公共 C ABI、Operator、运行时 Pipeline schema、`.conf` schema 或调度语义， +不新增持久化方案清单、图 DSL、Python Builder、自动协议转换、模型下载或算法生成。 +不要求所有用户转用 Web；手写 JSON 继续可用，CLI/Agent 通过同一作者操作生成现行文件。 +不以新增 Node、Model、Backend 或重构整个 Studio 为前置条件。 + +## 2. 现状与真实缺口 + +| 议题 | 已有事实与证据 | 尚需改进的部分 | +| --- | --- | --- | +| 数据绑定与执行依赖 | [connectPorts](../../tools/pipeline_studio/web/workbench.js) 已同时填写 `ports` 与 `depends_on`；[Validator](../../src/core/pipeline_validator.cpp) 检查生产者位于依赖祖先链 | CLI 缺少作者操作；导入无法恢复依赖意图，断线会删除同对节点的依赖，改接会留下旧依赖 | +| 内部字符串 key | [app.js](../../tools/pipeline_studio/web/app.js) 自动生成 ID/key；[AlgContext](../../include/core/alg_context.h) 是请求级、单次发布、有类型检查的数据空间 | 自动命名只检查节点 ID，未完整检查已占用 key；改 ID 后新增节点可能碰撞 | +| 图与合法性 | 可浏览缺资产或尚不合法的文档；[graphDocument](../../tools/pipeline_studio/web/workbench.js) 为布局建立临时连接关系 | 重复 key 的 Map 后写覆盖会掩盖来源歧义；画布连通不代表文件中的依赖已满足 | +| 编辑与诊断 | [editor.js](../../tools/pipeline_studio/web/editor.js)、[app.js](../../tools/pipeline_studio/web/app.js) 已有表单缓冲、继续操作、原因、候选修复、撤销和过期保护 | 合法表单仍需额外确认应用；修复预览偏向 JSON Pointer 与序列化差异 | +| 部署与运行 | [server.py](../../tools/pipeline_studio/server.py) 已有草稿运行、双 revision 成套保存、预检和失败回滚；运行结果已有版本归属 | 预检没有独立任务入口；服务重启后已有 `.conf` 不受当前会话管理 | +| 模型与参数 | 已有资产目录、兼容筛选、散列检查、构建提示和基础字段表单 | Model/Backend/路径同时暴露；object/array 仍是文本 JSON；恢复缺省往往要进入 JSON 页 | +| 来源契约 | Catalog 已含 cardinality/provenance/lifetime；[批次工具](../../include/nodes/traceable_batch_operations.h) 与 [Node 教程](../dev_guide/custom_node_concepts.md) 已封装常用处理 | 连线时缺少数量与来源的直观解释；静态校验与真实对齐保证容易混淆 | + +固定两个回归用例： + +1. A 输出 `a_out`，C 读取它并声明 `depends_on: ["a"]`。当前断线会删除依赖，但文件没有 + 说明该依赖是否还承载作者要求的执行顺序；工具无法无损猜出意图。 +2. 节点 `foo` 改名为 `bar` 后，保留 `foo__out` 是合法的。再次新增 `foo` 时，目前可生成 + 同名 key；断开业务输出时重新分配内部 key 也存在类似风险。 + +已实际查询 `keyword_match_v1` 和 `TextChunkNode`。后者 `chunks` 输出声明为 +`1:N / generate_sub_id`，`chunk_counts` 为 `1:1 / preserve`。这仅是本基线示例；实现时仍 +查询目标工具，不把本文变成第二份节点或字段目录。 + +[RFC-0051](0051-developer-task-experience.md) 的 M1–M5 已交付，M6 试用待完成；本方案扩展其 +诊断和任务闭环，不重建测试生成器、修复引擎或 recipe。历史 RFC 中旧部署写法按当时基线 +阅读；本方案仅消费当前 Resolver 接受的 `data.outputs` 配置。 + +## 3. 开发者需要知道什么 + +| 人员 / 场景 | 必须理解 | 默认由工具或既有 helper 处理 | +| --- | --- | --- | +| 复用已有能力编排 | 完整业务输入输出、节点用途、数据来源、常用参数、校验与运行的区别 | key 分配、依赖补充、模型 ID 引用、命令和配套路径 | +| 拆分、检索、多输入、聚合 | 一条还是多条结果；属于哪个请求/片段；逐条关联还是按请求分组 | 已注册节点的来源实现、静态兼容性检查和关系说明 | +| 编写自定义 Node | C++ 业务函数、逻辑端口、数量与来源、错误处理、请求无状态 | 既有 AuthorNode/Spec、批次工具、生成测试与登记 | +| 实现 Adapter 或部署集成 | 完整 C ABI 契约、外部载体与容量、部署根和输出池 | 原生 Resolver、既有 Adapter/bridge 作者工具 | + +普通编排不要求理解 `AlgContext` 实现、锁、JSON Pointer 编码或批次补齐机制。 +`req_id/sub_id` 表示数据归属，不能随时用循环下标替代；编写自定义批次算法时必须理解。 +基础概念可以学习，但工具已经能够确定的信息不应要求作者重复填写。 + +## 4. M1：统一连接、命名与引用 + +### 4.1 所有者与接口 + +增加工具私有的 `src/tools/pipeline_authoring.h/.cpp`，由生产和测试 `alg_pipeline_tool` 共用， +不链接进 SDK 请求执行路径，不成为 Node/Model 的依赖。它只变换作者文档，从 Catalog 查询 +定义，调用 `PipelineValidator` 判断生成文档；不复制校验规则。 + +```mermaid +flowchart LR + A[Studio / CLI / Agent] --> B[Tooling 作者操作] + B --> C[现行 Pipeline JSON] + C --> D[PipelineValidator] + D --> E[诊断和执行计划] + E --> A + C --> F[现有 Resolver 与 Demo] +``` + +运行时依赖仍为接入适配层 / Integration → 流程编排层 / Orchestration → 能力节点层 / +Capability Nodes → 模型执行层 / Model Execution；运行时不推导缺失的 `depends_on`。 + +拟新增 `alg_pipeline_tool edit --stdin`。输入包含 `schema_version: 1`、完整 `pipeline`，以及 +互斥的单个 `operation` 或有序 `operations` 数组；它只是短生命周期操作请求，不是第二种 +保存格式。首批动作限定为 `add_node`、`remove_node`、`rename_node`、`connect`、`disconnect`、 +`reset_input_binding`、`add_dependency`、`remove_dependency`。 +端点用 `node_id + port`，业务端点沿用 `$ingress/$egress`；参数/模型仍编辑现行字段。 + +| operation.kind | 其余请求字段 | 约定 | +| --- | --- | --- | +| `add_node` | `node_type`，可选 `id`、`config` | 类型查目标 Catalog；省略 ID 时自动分配，显式 ID 冲突时报错；缺参数可留在草稿，归一化默认值不全部回写 | +| `remove_node` | `node_id` | 必须唯一存在；列出断开的引用 | +| `rename_node` | `node_id`、`new_id` | 新 ID 非空且不冲突；同名为无变化操作 | +| `connect` | `source`、`target`，各含 `node_id`、`port` | 定位当前图中的唯一端点；重复同一连接不增加重复依赖 | +| `disconnect` | `source`、`target`，形状同上 | 先检查实际连接仍匹配，避免断开已被改接的输入 | +| `reset_input_binding` | `target`，含 `node_id`、`port` | 删除该输入的显式绑定，按原生默认语义重新解释；展示解析来源或未绑定/错误，不承诺一定接入 ingress | +| `add_dependency` / `remove_dependency` | `node_id`、`depends_on_id` | 分别表示消费者和前置节点，均须存在；已添加/已删除时不重复改变文档 | + +例如操作对象的拟议形状如下；外层 `pipeline` 由调用方放入实际完整文档： + +```json +{ + "kind": "connect", + "source": {"node_id": "chunk", "port": "chunks"}, + "target": {"node_id": "embed", "port": "text"} +} +``` + +严格识别操作字段；未知动作、缺失/重复端点、歧义生产者返回工具诊断，不猜目标。 +操作定位或请求解析失败时 `ok=false`、非零退出，不返回可应用的新文档；输入保持原样。结果包含 +`schema_version`、`ok`、变换后的 `pipeline`、结构化 `changes` 和完整 `validation`。 +`changes` 每项记录动作、稳定节点 ID、相关端口、受影响节点和中文说明，供预览与撤销使用。 +`ok` 表示该请求按指定提交策略完成，`validation.ok` 表示配置合法，退出码随 `ok`。 +顶层可选 `require_valid` 默认为 false，允许交互草稿；true 时最终校验失败则 `ok=false`， +返回诊断但不返回可应用的新文档。生成最终方案的 Agent/脚本使用 `require_valid: true`。 + +新增节点或断开必需端口可产生不完整草稿，允许编辑但必须显示未通过校验。保存为可运行方案 +和运行继续要求严格校验成功。不能放宽运行时规则，也不能要求每个中间编辑状态都合法。 + +作者变换模块和 `edit --stdin` 本身不写文件。Studio 拟增 `/api/v1/authoring/preview` 转发 +单动作或批量请求，复用草稿 revision、工具 fingerprint 与过期响应保护；一次接受整个结果 +形成一个撤销步骤。文件写入由现有 Studio 保存边界或第 4.5 节的显式 CLI 包装负责。 + +### 4.2 数据连接与执行依赖 + +| 作者动作 | 必须产生的行为 | +| --- | --- | +| A 输出连接 B 输入 | 同步实际 key；如 A 尚不是 B 的依赖祖先，加入直接依赖；已有祖先关系不增加冗余边 | +| 改接输入 | 改 key、补新生产者依赖；旧依赖按下述保守规则处理 | +| 断开输入 | 移除可选绑定；必需输入分配不碰撞的未连接占位 key，保持用户明确断开的意图 | +| 恢复默认绑定 | `reset_input_binding` 删除该输入映射；必需输入由原生尝试同名默认来源，可选输入回到不消费状态 | +| 断开/改接后的旧依赖 | 默认保留，显示为纯执行依赖；不依据“没有数据边”自动删除 | +| 删除执行依赖 | 独立动作，展示影响和原生结果；如仍为数据读取所需，草稿显示对应错误 | +| 删除节点 | 列出消费者、业务出口和被移除执行边；清除引用，必需输入转为未连接状态，支持撤销 | +| 打开/导入 | 不补依赖、不改 key、不排序回写；图用于浏览，校验状态单独展示 | + +同一未中断撤销历史内可记录“本次连接动作新增的依赖”，改接后无其他连接使用时，在预览中 +提供清理选项；用户显式添加/保留的依赖不得归为可自动清理。文件重开、JSON 手工编辑或历史 +丢失后一律视为未知意图，恢复保守规则。历史仅作为编辑提示，不写入 Pipeline，不从 ID、 +数组顺序或 key 名猜意图。没有这项辅助清理也可以交付首版，保守规则必须先成立。 + +同一节点对的数据关系与显式依赖都应有可查看、可编辑的入口，不能被图形合并隐藏。 +类型、成环、来源兼容性由原生报告决定；浏览器只做操作形状检查和提示。 +本轮不建设全目录自动接线推荐或搜索规划器。 + +#### 断开与恢复默认的生命周期 + +必需输入省略映射时，现行 Validator 使用逻辑端口名寻找同名祖先生产者，适用时再从 biz +ingress 解析。故意断开时直接删除映射，可能让它悄悄接回默认来源；未连接占位 key 用于阻止 +这种回退，不修改 Catalog 或运行时规则。可选输入省略映射则不消费，不自动读取同名 ingress。 + +重新 `connect` 会覆盖该端口的占位；`reset_input_binding` 删除映射并在预览中显示恢复后的 +来源/诊断；删除节点同时删除其绑定。占位不是独立资源，无引用后不再进入占用集合。不按 +`__unconnected__` 等字符串前缀识别或清理导入内容，因为它仍可能是用户合法使用的数据名。 +恢复默认不等同于连接 ingress，也可能得到同名祖先输出或仍因无来源而报错。 + +#### 扇出共享与重连范围 + +此处扇出指一个输出有多个消费者，与数据 Cardinality 的 `1:N` 是不同维度。`1:1` 输出也可 +扇出，`1:N` 输出也可只有一个消费者；普通扇出不额外复制节点或改变端口数量/来源声明。 + +| 动作（初始为 A.out 同时供给 B.in 和 C.in） | key 与消费者范围 | +| --- | --- | +| 断开 B.in | 只改变 B 的输入绑定；A 的发布 key 和 C 的绑定保持不变 | +| 将 B.in 改接到 D.out | 只改变 B 的输入和所需依赖；A/C 不变，旧顺序依赖沿保守规则 | +| A.out 接到 biz egress | 按契约改 A 的发布 key，同步全部有效消费者，包括省略映射的必需输入 | +| 断开 A.out 与 biz egress | 给 A 分配新内部 key，同步全部有效消费者；B/C 仍连接 A，不能随出口一起断开 | +| 删除 A | 所有消费者分别进入必需未连接或可选未绑定状态，完整预览且一次撤销 | + +出口已被另一生产者占用时，不由 connect 静默替换；用批量 disconnect + connect 明确表达。 +生产者 key 只有在出口连接/断开等明确的输出重映射动作中才改变，普通消费者操作不得改它。 + +### 4.3 key、ID 与歧义 + +1. ID 全图唯一；key 用节点 ID/逻辑端口形成可读前缀，冲突时加确定性后缀。占用集合包括 + 显式绑定、目标 Catalog/原生解析得到的有效默认绑定、未连接占位键和 biz ingress/egress， + 不仅检查节点 ID 或 JSON 中显式出现的 ports。缺 Catalog 导致无法确定默认绑定时，不分配 + 可能冲突的 key，保留浏览/JSON 编辑入口并提示构建工具。 +2. 改 ID 同步 `depends_on`、选择状态与布局关联，保留数据 key；不全文替换字符串。新增 + 节点、断开业务输出、未连接占位键都使用同一分配器。 +3. biz ingress/egress key 属于契约，不任意改名。连接业务输出检查生产者唯一性；连接或断开 + 业务出口导致发布 key 改变时，同步该输出的所有有效消费者，原先省略映射者写入明确绑定。 + 不能遗漏隐式引用；必需消费者已故意断开时不因名字相近而被重新接回。 + 一个输出需要发布到两个不同契约 key 时报告当前操作无法表达， + 不隐式复制节点或伪造输出。 +4. 导入有重复生产者时保留全部候选并显示歧义，不用 Map 后写覆盖伪装唯一来源。可浏览 + 原文、明确修复；选择生产者的操作须先消除歧义。 +5. 不提供全库 key 规范化或打开即迁移；内部 key/依赖数组收进高级详情，JSON 始终可查。 + +### 4.4 有界批量操作 + +首版同时支持单动作和批量，二者调用同一变换实现。批量请求一次读取/解析完整文档，固定 +同一目标 Catalog 快照，按数组顺序作用于私有副本。后续动作引用本批新增节点时使用明确 ID， +不增加结果引用表达式或持久会话协议。 + +- 单次最多 128 个动作，整个输入最多 4 MiB；超限或空数组在变更前拒绝。示例的 5 个新增 + 节点和 8 次连线可放在一个请求，不需要 13 次 CLI 启动或文档往返。 +- 中间允许必需端口尚未连接等不完整状态。逐步只做动作前置检查，不对每个动作完整校验； + 完成后统一执行原生校验，按 `require_valid` 决定是否接受最终候选。 +- 任一步动作失败，返回零起始 `failed_operation_index` 与原因，丢弃整个副本，不暴露可误用的 + 部分成功文档；严格提交的最终校验失败也丢弃副本。原输入与磁盘文件始终不变。 +- 成功返回合并 changes、最终 Pipeline 和报告，Studio 作为一条历史记录接受或撤销。 + 这是内存文档的全有或全无操作，不承诺跨文件、模型资源或 SDK 执行事务。 + +M0a 固定对比输入与测量字段；M1a 实现后记录单动作循环与批量的进程次数、输入/输出字节数、 +校验次数及耗时。批量减少确定的重复工作，但不在测量前宣称 13 次小文档操作已造成严重性能问题。 + +### 4.5 IDE 文件工作流与原地依赖修复 + +增加有界的文件便利命令，复用原生解释与同一作者操作，不将读写文件放入 Validator： + +```bash +# 拟议命令：默认预览差异和最终校验结果，不写文件 +./build/alg_pipeline_tool fix-deps configs/pipeline_example.json +# 明确要求修复并原地保存；此命令仍待实现 +./build/alg_pipeline_tool fix-deps configs/pipeline_example.json --in-place +``` + +`fix-deps` 只处理原生诊断确认的 `producer_not_dependency_ancestor`：定位当前实际 key 的唯一 +生产者，复用有效默认绑定和现有依赖候选，在副本上添加所缺依赖。不猜数据 key，不删除已有 +顺序依赖，不重命名节点，不改参数、biz 或 `.conf`；已存在祖先关系不加冗余边。 + +多生产者/来源歧义、无生产者、成环、类型或来源不兼容、未知配置等不能用补依赖掩盖。读取 +一次当前文档，收集有界的明确候选，用共享操作顺序应用并做最终完整校验；不能通过时返回 +原生诊断、非零退出，原地模式不写入部分修复。无需变化时保持原文件字节不动，重复执行幂等。 + +原地模式只覆盖用户明确指定的普通文件，不跟随符号链接。读入时保存旧字节和文件身份， +在同目录写临时文件、保留原权限，提交前再次核对身份及内容；冲突或预检/写入失败保留原文件， +成功通过单文件原子替换安装。默认预览不写文件；机器报告注明写入与否、目标路径、changes、 +最终 validation，不以退出 0 代替结果检查。跨任意外部编辑器的恶意竞争和多文件事务不在承诺内。 + +这个命令面向已在 IDE 保存到磁盘的配置；未保存的编辑器缓冲不在读取范围。VSCode 可用普通 +任务调用，无需本轮建设专属插件；涉及模型路径或输出池仍走 M3/M4 的配套预检与保存。 + +## 5. M2：连续编辑与可读诊断 + +保留现有表单缓冲和历史机制。存在未应用修改时，“校验”“运行”明确显示“应用修改并校验” +“应用修改并运行”，一次点击提交当前合法表单到草稿并继续，不自动落盘。非法表单保留输入 +和焦点，不调用运行接口；切文档或放弃修改仍沿用明确选择。保存使用相同应用过程，目标由 +服务端返回。 + +将修复的 `window.confirm` 换成页内审阅面板：原生原因、节点/字段、`fix.title/effect`、 +影响的其他节点、剩余错误与验证程度。Pointer、patch、code、diff 放进展开详情。区分 +`pipeline_valid` 与仅解决目标问题的候选；无安全候选时保留原生建议和手工入口。 + +消费既有 remediation，不解析 message 猜原因，不在浏览器重建规则或生成候选。保留草稿 +修改、数组重排、工具重建、切文档使旧候选失效的保护，应用后重验且一次撤销。用户文本用 +文本渲染，不插入 HTML。已有运行快照与旧结果失效机制继续使用，新动作纳入相同版本比较； +不重建运行历史或任务调度系统。 + +## 6. M3 / M4：部署预检与成套编辑 + +### 6.1 M3：独立预检 + +增加“检查运行条件”，复用 staging、`resolve_run_conf` 和原生 `resolve-conf`；临时文件 +结束后清理，不加载权重、不运行 Demo、不写用户文件。草稿与运行设置共同构成预检版本， +修改后摘要标为过期。 + +| 摘要 | 来源与边界 | +| --- | --- | +| 本次 Pipeline、biz、部署根 | 提交快照、首版固定的项目根与 Resolver 结果 | +| 最终模型路径与覆盖来源 | `configuration.model_paths`，不在浏览器推导优先级 | +| 生效参数与输出池容量 | `effective_pipeline`、`output_pools`；布局 `params` 保留文本语义，不伪称已补齐布局默认值 | +| 当前工具、Backend、资产 | 当前 Catalog 与既有资产检查；区分缺失、存在未校验、散列已验证 | +| 下一步 | 配置错误、缺工具、缺资产分阶段指引；保留原始诊断 | + +区分“配置通过”“资产检查结果”“尚未执行”“样例执行结果”；预检通过不能显示模型已加载 +或效果达标。实际加载与样例验收继续走现有路径。 + +### 6.2 M4：显式关联已有配置 + +增加“关联部署配置”：选择受控目录内的 `.conf` 和运行选项；首版部署根固定 `PROJECT_ROOT`， +资产目录 `model_root` 仍可明确选择项目内目录。原生 Resolver 确认该文件解析到当前保存的 +Pipeline，展示两份文件及覆盖后建立本会话关联，记录双 revision。 +关联本身不写文件。重开可提示候选，但不得凭同名或 Profile 自动取得覆盖权限。 + +首批限定现有受控 `configs/` 与项目内路径，复用路径/符号链接约束；外部导入继续另存副本。 +单槽/多槽 `data.outputs` 只要原生接受且工具可原样保存即可关联，不降级输出池布局。 + +路径必须按下表处理；关联旧文件时先保持原来的有效值，不把旧 Pipeline 路径重新解释为所选 +资产目录下的文件。资产目录仅用于用户随后明确选择的新资产。 + +| 作者动作 | Pipeline / conf 的确定变化 | +| --- | --- | +| 常规模型选择中换资产 | 将选中资产相对 `model_root` 的路径写入该模型 `model_path`；同时将 `model_root/该路径` 相对项目根的值写入该 ID 的 conf 覆盖；由 Resolver 验证结果 | +| 只改模型参数 | 只改指定 `model_config/backend_config` 等参数，保留所有路径和覆盖 | +| 高级“只改部署覆盖” | 只更新该 ID 的 conf 覆盖，值相对项目根；Pipeline 保持，摘要显示来自部署配置 | +| 高级原始 JSON 改 model_path，已有覆盖 | 不猜生效意图；保存/运行前提供“保留当前部署覆盖”或“让新资产路径在本部署生效”，明确预览结果。前者保留原覆盖，后者按明确 model_root 重建该条覆盖 | +| 显式模型 ID 改名 | 使用明确 old/new 映射，迁移 Catalog model_dependencies 声明的模型引用及对应覆盖；新 ID 冲突拒绝，不替换任意字符串 | +| 原始 JSON 删除旧 ID 并新增 ID | 作为删除与新增处理，不推断改名；清理旧覆盖，新增项仅在有明确资产选择记录时建立覆盖；否则不创建覆盖，按 Resolver 将 Pipeline 路径相对项目根解析并展示结果 | +| 删除模型 | 删除对应覆盖，保留其他条目；未修复的节点引用继续由 Validator 拒绝 | + +原始 JSON 修改无覆盖模型路径且无明确资产选择记录时，同样不创建覆盖，保持 Resolver 的 +项目根解析语义，展示实际结果；不能把这种编辑默认为相对当前资产目录的选择。 + +例：Pipeline 的 A 为 `a.gguf`、B 为 `b.onnx`，旧 conf 分别覆盖到 `models/a_old.gguf` 和 +`deploy/b_tuned.onnx`。在常规选择中以 `model_root=models` 给 A 换成 `a_new.gguf` 后， +Pipeline A 写 `a_new.gguf`，conf A 写 `models/a_new.gguf`，B 的覆盖仍为 `deploy/b_tuned.onnx`。 +预检、Demo 与保存后重开必须得到相同两条有效路径；仅修改绑定 A 的节点生成参数时，A 的覆盖也不变。 + +1. 扩展当前“本会话生成方案”的管理记录以接受显式关联。模型 ID/路径变更先展示 Pipeline + 与部署覆盖的现值和拟修改值，明确本次写入范围。 +2. 仅修改作者此次改动涉及的模型覆盖；其余覆盖保留。删/改名模型时列出相关条目、更新 + 引用并预检，不从 Profile 重建整份部署文件。 +3. 原样保留无关的合法 `outputs`、`allocator`、`params`、容量。未知或 Resolver 拒绝的配置 + 只读展示并报告原因，不删字段求通过；原样指 JSON 值及参数文本保真，不承诺原缩进。 +4. 复用双 revision、staging、提交前复核、写入失败回滚与备份保留。只成套更新明确关联的 + 文件；外部修改要求重载/审阅，不静默覆盖。 +5. 预检、运行、保存使用同一候选部署配置，仅临时执行替换 `pipe_path`；不能预检保留覆盖， + 运行又按 Profile 重建路径。staging 位于项目根内，Resolver root 与 Demo cwd 均为项目根， + `--config` 必须相对此根；不透传绝对 conf 路径使 Demo 改以配置父目录为根。二进制、数据集、 + 输出位置使用明确路径。支持其他部署根另行设计，不加入首版。 +6. Profile 可提供数据集/运行选项，不覆盖已关联方案；输出含明确 `--config` 的当前命令， + 不回写原 Profile，也不强制创建新 Profile。 + +现有保存提供单服务进程内冲突检测与失败恢复。本轮不承诺任意外部写入竞态或断电时跨文件 +的系统级原子事务；并发编辑、持久事务、跨刷新自动接管另立需求。 + +### 6.3 内部 egress 与输出池的边界 + +`$egress` 是 BizDefinition 声明的内部结果 key；`data.outputs` 是 Operator bridge 声明的逻辑 +输出槽位。二者不要求一一对应，不能根据画布连线增删输出池。 + +本基线 `smart_doc_qa_v1` 有 `llm_answers`、`intent_matches`、`doc_chunk_counts` 三个内部 +出口，Adapter 组装为 DocResult，bridge 只有 `doc_out` 一个槽，conf 对应只有 +`data.outputs.doc_out`。事实分别见 [Adapter](../../src/adapter/biz/doc_qa_adapter.cpp)、 +[bridge](../../src/adapter/biz/doc_qa_operator_bridge.cpp)和[部署配置](../../configs/pipeline_doc_qa_default.conf)。 +可用契约仍需查询目标 Catalog，槽位完整性以原生 Resolver 和 bridge 注册为准。 + +| 变更 | 工具应做什么 | +| --- | --- | +| 同一 biz 内增删/改接 egress 连线 | 仅修改内部数据绑定；outputs/allocator/params/容量保持原值，重跑 Validator 和 Resolver | +| 断开必需 egress | 标为不完整草稿，阻止可运行保存与执行；不能靠删除输出池“修复”缺少业务结果 | +| 某个可选内部结果本次不产出 | 按既有 Adapter 契约处理，不能推断 bridge 槽位可删除；已声明的可选输出槽也要求部署配置 | +| 选择另一 biz / 注册契约变化 | 作为业务契约切换，现有关联失效；选匹配模板或明确重新关联并完整预检，不把旧槽位按端口名猜测迁移 | +| 新增/删除真正的外部输出槽 | 走 Integration 的 Adapter/bridge/输出类型与 conf 变更及契约测试；不是普通 Pipeline 连线操作 | + +因此本工作包补齐的是保留和失配检测规则，不新增自动级联输出池的功能。跨层参考 +[RFC-0049](0049-operator-output-allocation-strategies.md)及[当前输出分配指南](../dev_guide/operator_output_allocation.md)。 + +## 7. M5：模型与参数渐进展示 + +默认先选当前方案中兼容的模型实例，需要新增时再选已有资产组合。继续用目标 Catalog +核对 capability/protocol，用已有资产清单提供路径；不由预设名推断可加载。ID 自动分配， +Model 类型、Backend、路径及两类完整参数留在高级区。 + +预设变更先预览模型、节点绑定和部署覆盖的影响，再形成一次撤销操作。共享实例列出全部 +消费者；默认给单节点换模型仅改变该节点绑定，修改共享实例用独立明确动作。无适用预设 +时保留手动配置，未知或缺 Backend 的导入仍可浏览，不自动下载/更换构建/切到测试注册。 + +参数复用 ConfigFieldDefinition 的类型、默认、枚举、范围、semantic。增加“使用默认值 / +显式设置”，用删除字段恢复缺省；区分缺省、空串、空对象、空数组、零、false。不能因展开 +字段而写默认值，不能修剪提示词空白或破坏多行文本。 + +object/array 首批做通用 JSON 值树：增删键/元素、选择值类型、格式化、切回源码；只保证 +JSON 语法与已声明外层类型。没有子结构元数据不推断必填子字段，不根据示例另造校验。 +业务合法性交给原生语义校验，非法文本缓冲保留，合法内容在树/源码间无损往返。 + +本轮不引入递归 schema 系统或 UI 专属节点表。若试用显示某复杂字段仍是主要阻碍，再提出 +Definition 子结构元数据的独立 RFC，同时迁移 Catalog 消费端；不在浏览器硬编码隐藏 schema。 + +## 8. M6：契约说明与基础概念 + +默认从已有可用模板新建，空图保留为进阶入口。显示已有业务名称、标识、内部输入和必需输出， +明确它们是 Adapter 内部端口；完整外部请求/响应查对应任务样例与[业务接入指南](../dev_guide/business_onboarding.md)。 + +在现有入门/recipe 页面补原始请求/响应示例与变更范围判断，复用 fixture 和 C ABI 契约测试。 +没有完整样例的业务标明尚无样例，不由 TextBatch 或 C 载体推导 payload；业务维护自己的 +样例，不在 Web 建中央业务分支。本轮不新增 Catalog 外部 schema 协议。 + +端口详情展示类型、数量、来源、生命周期，已知值附简短解释，原始值可查；未知值不默认视为 +`1:1/preserve`。可变语义如 `lifetime_config_field` 消费原生规范化/校验结果，没有结果就标为 +待校验，不能展示静态默认却声称已生效。解释是端口声明，不从单个标记推断整个算法行为。 + +不在 UI 重写兼容矩阵：现有 `N:M`、`N:1`、`preserve`、`aggregate` 规则并非字符串相等。 +在既有教程补“两条请求分别拆成三块/两块，排序或过滤后正确关联”的小样例，讲清类型相同 +仍可来自不同请求、逐条关联与按请求聚合不同。1:1 Node 复用现有 Spec，复杂批次复用 +RFC-0055 工具。静态校验只证明声明可连接，真实对齐由 Node/Model/Adapter 运行时检查保证。 + +## 9. 兼容、迁移与替代方案 + +| 对象 | 实施要求 | +| --- | --- | +| Pipeline / `.conf` | 旧合法文件保持合法；不批量重写，显式 `id/depends_on` 不变 | +| Catalog / 诊断 | 复用 Catalog v3、remediation v1；新作者操作单独版本化 | +| CLI / Studio 版本差异 | 不支持 edit 的工具仍可打开、JSON 编辑、原生校验和运行；新图编辑提示重建，不悄悄使用另一套变换实现 | +| SDK / 分层 | 作者变换只进工具目标；不改执行计划消费，不移动平台协议转换 | +| 文件归属 | 普通保存保持原范围；仅显式关联/本会话创建的方案可以成套更新 | +| 作者职责 | 业务语义、来源策略、效果期望由人决定；工具不给不确定行为补成功结果 | + +不采用以下路径： + +- 运行时从 key 隐式补依赖：改变严格契约，也无法恢复纯顺序意图。 +- 所有人只用 Studio：遗漏 CLI/Agent；需共享小型作者操作。 +- 新 DSL、Builder、持久方案 manifest：增加表示和迁移成本，先生成现行文件再评估必要性。 +- 删除来源契约或用数组位置替代归属：保留错配风险，不能保证结果属于正确请求。 +- 只补文档或改布局：不能修复已确认的意图丢失、key 碰撞与部署覆盖缺口。 + +M1 迁移后删除重复 JS 变换，前端保留显示/交互/请求。标准产物可由旧运行时读取；回退工具 +无需迁移文件。M4 回退后仍可用显式 `.conf` 命令执行，不依赖会话记录。 + +## 10. 实施顺序、责任与工作量 + +各包完成指定行为、聚焦测试和指南后交付；估算面向熟悉仓库的单人，不含资产准备、试用 +等待和环境故障，不是完成时限承诺。 + +| 阶段 | 主要内容 / 文件 | 前置 | 估计 | 状态 | +| --- | --- | --- | --- | --- | +| M0a | 工程基线：固定旧提交、工具/环境、任务输入、期望和失败样例；维护者可独立完成 | 采用本 RFC | 1 人日 | 已完成 | +| M0b | 开发者体验基线：安排试用并记录求助/耗时；复用既有试用计划 | 与工程并行，不阻断开发 | 约 1 人日，加人员安排 | 待试用 | +| M1a | 单动作/批量共用作者操作、fix-deps 文件包装；`src/tools/alg_pipeline_tool.cpp`、CMake、测试工具目标 | M0a | 4–7 人日 | 已实施 | +| M1b | Studio 消费操作、歧义图、键隐藏、依赖审阅；`server.py`、`workbench.js`、`app.js`、`graph.js` | M1a | 3–4 人日 | 已实施 | +| M2 | 连续动作和页内修复；`app.js`、`editor.js`、`workflow.js`、`index.html` | M0a，可与 M1a 并行 | 2–3 人日 | 已实施 | +| M3 | 无执行预检、摘要；`server.py`、`app.js`、现有 `verify_selection.py` 辅助 | M0a，可与 M1a 并行 | 2–3 人日 | 已实施 | +| M4 | 已有 conf 关联、局部覆盖更新、预检/运行/保存共用候选 | M3 | 3–5 人日 | 部分实施；高级单独覆盖编辑和显式模型 ID 迁移待实施 | +| M5 | 模型选择分层、共享影响、缺省切换与 JSON 值树 | M1b/M2；部署联动依赖 M4 | 3–5 人日 | 待实施 | +| M6 | 来源说明、既有教程补充、开发者复测 | 说明可提前；复测依赖前述交付 | 2–3 人日，加试用安排 | 待实施 | + +首个可交付增量为 M1a/M1b + M2 + M3；先修正确性和高频动作，再做 M4/M5。仅 M0a 是工程 +前置，M0b 缺参与者不阻断开发、测试或工程交付；维护者走查不能冒充新开发者试用。 + +M0a 固定基线提交、任务夹具、构建/资产版本和环境说明，保留可复现旧版的方式；参与者到位后 +可在隔离环境补做旧版/新版的等价任务，明确延迟采样与学习效应。体验项保持待试用，不因为 +工程通过而宣称用户效率已验证。不得因等待试用停掉已具备工程前置条件的任务，也不顺带重构 +Core、schema 框架或整个可视化 IDE。 + +## 11. 验证与完成条件 + +### 11.1 工程行为矩阵 + +| 编号 | 场景与必须观察到的结果 | 现有责任套件 | +| --- | --- | --- | +| G1 | 同输入的 CLI/Studio 操作得到相同文件与原生报告；草稿不合法不冒充可运行 | `tests/tooling/test_pipeline_studio.py` 的真实工具 roundtrip | +| G2 | 连线维护 key/依赖；间接祖先不加冗余边；改接/断线保留未知顺序意图 | 同上；`studio_graph_test.mjs` | +| G3 | foo 改 bar 再加 foo、断开业务出口、未连接占位键均不碰撞；导入省略绑定时的有效默认 key 也被保留 | 同上；`studio_browser_test.mjs` | +| G4 | 重复生产者/缺依赖/无效 ID 导入不静默修正；歧义图不选任意来源 | 同上；原生 `test_pipeline_catalog_validator.cpp` | +| G5 | 扇出、隐式默认消费者、业务出口、改名/删除/撤销正确；过期结果不覆盖新草稿 | `studio_editor_test.mjs`、真实 handler 测试 | +| G6 | 5 节点/8 连线一次批量请求；中间可不完整；动作失败/strict 最终失败无部分结果；超限拒绝，批量一次撤销 | CLI roundtrip、handler 与历史用例 | +| G7 | 同名 ingress 存在时断线仍不读取；显式 reset 才恢复默认；重连覆盖占位，可选 reset 后不消费 | Catalog/Validator 与 Studio 操作用例 | +| G8 | A 扇出 B/C，改接或断开 B 不改变 A/C；连接/断开业务出口同步含隐式绑定的全部消费者 | 原生 roundtrip、`studio_graph_test.mjs` | +| G9 | fix-deps 预览不写、唯一生产者正确补依赖、幂等；歧义/循环/其他错误不写；文件冲突/写失败保持原文件 | `test_pipeline_studio.py` CLI 与临时文件故障注入 | +| U1 | 合法表单一次动作校验/运行；非法表单无运行请求、原输入保留 | `studio_browser_test.mjs` | +| U2 | 修复面板显示验证程度/影响/剩余错误；保留过期、切文档、工具重建保护 | `studio_fix_workflow_test.mjs`、服务测试 | +| D1 | 预检无 Demo/用户文件写入，临时文件清理；摘要与原生结果逐字段一致 | `test_pipeline_studio.py` 的 resolve-conf/staging 用例 | +| D2 | 重启后显式关联可保存；未关联不接管；错指向/路径/符号链接/revision 拒绝 | 同上的 ownership/conflict 用例 | +| D3 | 两模型旧覆盖与 Pipeline 不同时仅换一个，按动作表保留另一覆盖；多输出保真；预检/运行/保存重开路径一致，Demo 使用项目根和相对 conf | 同上；已有 Operator Resolver 测试 | +| D4 | 第二文件安装失败回滚；回滚失败留备份；冲突不覆盖外部修改 | 同上已有故障注入用例 | +| D5 | 三个内部出口对应一个 doc_out 槽；断开/改接不增删池，缺必需结果仍失败；切 biz/注册槽变更使旧关联失效，缺槽/多槽原生拒绝 | Catalog、Adapter/bridge、Resolver 与 Studio 用例 | +| F1 | 换单节点绑定只影响该节点；编辑共享模型列出消费者；缺资产/构建不伪报就绪 | 现有模型表单、asset hash、build variant 用例 | +| F2 | 缺省/空值/零/false/多行、树/源码往返保真；非法输入保留 | `studio_editor_test.mjs`、浏览器字段用例 | +| C1 | 数量/来源/有效生命周期正确显示；同类型但不兼容仍由原生拒绝 | `test_validated_pipeline_plan.cpp`、Catalog/Validator、Studio | +| C2 | 两请求不等长拆分、排序/过滤正确对齐；错数量/来源被运行时拒绝 | 既有 `test_traceable_batch_operations.cpp`、相应 Node/Adapter 测试 | +| T1 | 生产/测试工具不互换；新操作版本化；旧命令/配置保持行为 | CLI/parity 用例、Catalog 契约套件 | + +只补变更所需差异测试，未修改的来源 helper 不重复写同构测试。不为表格每行创建可执行文件， +按现有 Python/JS/C++ suite 扩展。浏览器验收须执行真实页面动作，不能靠字符串匹配；缺少 +浏览器则明确未覆盖，默认门禁可选浏览器跳过不表示页面验收已完成。 + +每个交付增量完成源码、测试与文档后，按 [CONTRIBUTING](../../CONTRIBUTING.md#6-run-one-canonical-delivery-gate) +执行一次 `./scripts/run_all_tests.sh`，不在前后重复完整构建/测试，不同代理不并发构建同目录。 +工具行为可用确定性 fixture 验证；真实模型质量与目标硬件按现有验收流程单独记录。 + +### 11.2 开发者任务 + +原始记录继续放在[既有试用计划](../plans/solution_developer_acceptance.md)，不建平行台账。 +体验验收目标仍为至少两位未维护 Core 的方案开发者，在相同准备环境做基线/复测； +这不是工程开工条件，暂缺人员按 M0b 记录待试用。交换等价任务顺序，记录 +首次跑通时间、求助原因、人工 key/依赖/路径修改次数、修复尝试、查阅入口数和输出证据； +下载/完整构建等待单列。 + +| 任务 | 通过条件 | +| --- | --- | +| A：关键词模板改规则并运行 | 无 C++ 修改；契约不变、结果随规则改变；不手工维护 key/依赖/路径 | +| B：多节点方案插入、改接、重命名 | CLI/Studio 各覆盖；无 key 碰撞、不丢已有顺序要求；最终校验及样例通过 | +| C：给单节点换兼容模型，重启 Studio 再编辑 | 识别共享影响；关联正确 conf；实际加载与预检一致，缺资源时定位失败阶段 | +| D：修复缺依赖、错端口、复杂字段非法值 | 不阅读 Validator 实现或依靠裸 Pointer，能说明原因、修复并验证 | +| E：解释两请求拆分/聚合与错误来源样例 | 区分类型、逐条对应和同请求归属，不要求复述实现细节 | + +硬性要求：正常图操作人工 key/依赖同步为 0；未知顺序意图不静默删除；已关联方案无需手工 +改路径使修改生效；实际运行与显示版本一致；非法配置与错配不能被简化流程包装成成功。 +时间目标由团队在 M0b 体验基线采集后确定，不用代码行数代替体验改善。求助和反复尝试未下降时， +按记录修正对应工作包，不能仅凭自动化通过关闭体验验收。 + +## 12. 实施记录 + +已完成工程基线 M0a 及首个可交付增量 M1a/M1b + M2 + M3 的实现与自动化回归测试；M0b 体验基线与 M6 复测待真实开发者试用后补充记录。功能实际交付时更新 Changelog，不提前宣称耗时减少。M1–M5 等工程包可在各自 +工程门禁通过后独立交付；RFC 整体关闭仍要求指定体验验收有证据，暂缺人员时保留 M0b/M6 +待试用状态，不阻塞工程交付。未执行的真实模型/平台验收保留边界。公司内部 SDK 与目标硬件 +按 [RFC-0029](0029-external-readiness-and-intranet-sdk-migration.md)的授权环境要求进行。 + +### 2026-09-14 交付增量实现（M0a/M1a/M1b/M2/M3） + +1. **C++ 原生图作者操作库与 CLI**（M1a）： + - 在 `src/tools/pipeline_authoring.h` 和 `src/tools/pipeline_authoring.cpp` 中实现规范操作（`add_node`, `remove_node`, `rename_node`, `connect`, `disconnect`, `reset_input_binding`, `add_dependency`, `remove_dependency`）。 + - 实现最大 128 步有界内存事务与原子回滚，4 MiB 请求上限校验。 + - 实现保留节点 ID 检查（`$ingress`/`$egress`），自动规避间接祖先冗余依赖。 + - 实现隐式/默认端口绑定的解析与断开同步，断开出口时同步所有消费节点（含隐式绑定）。 + - 实现 `fix-deps` 子命令：支持只读预览（stdout/json）与原子安全 `--in-place`（检查短写/刷新/关闭，提交前复核身份和内容，失败保留原文件），对歧义生产者（多生产者提供同名 key）或环路自动拒改（fail-closed）。 +2. **Pipeline Studio 服务与前端交互**（M1b/M2/M3）： + - 在 `tools/pipeline_studio/server.py` 实现 `/api/v1/authoring/preview`、`/api/v1/preflight` 与 `/api/v1/deployment/associate` 路由。 + - `app.js` 的图修改统一调用原生作者操作；`workbench.js` 仅构建只读图。支持端口连接/断开/默认重置、独立依赖编辑、一次应用表单并校验/运行/保存，以及页内修复审阅。 + - 增加运行前无执行部署预检与结构化诊断展示。 +3. **自动化测试套件与门禁保障**： + - 扩充 `tests/tooling/test_pipeline_studio.py`，覆盖原生作者事务、短写故障、请求边界、部署候选与双文件冲突。 + - 覆盖 Node.js/DOM 前端单测（`studio_graph_test.mjs`, `studio_editor_test.mjs`, `studio_fix_workflow_test.mjs`）。 + - 全量通过 `./scripts/run_all_tests.sh` 门禁。 + +### 2026-09-14 实现验收修复 + +- R1/R2/R5/R6/R7：原地写入使用独占临时文件及完整写入/刷新/关闭检查，提交前重验文件；普通扇出保持有效 key；错误端点、歧义 ID/生产者及非法请求拒绝，不返回可应用的部分文档。 +- R3/R10：Studio 图操作统一进入原生工具；添加默认绑定与独立依赖入口，候选应用形成一次撤销；修复改为页内审阅，应用前再次检查草稿和工具版本。 +- R4：预检、运行和保存共享候选部署配置，保留无关覆盖及输出池。常规模型表单换路径记录显式资产动作；原始 JSON 改动已有覆盖的路径时，运行页提供保留覆盖/采用新资产选择。服务拒绝缺少意图的请求。 +- R8/R9：预检绑定方案、文档、关联配置和运行设置快照；改变后标为过期，丢弃延迟响应；工具状态使用服务端同名字段。 +- 聚焦回归及真实 Chromium 流程涵盖原生图请求、默认绑定重置与撤销、页内修复取消/应用、预检过期与延迟响应。最终交付继续执行一次 `scripts/run_all_tests.sh`；浏览器验收需要设置 `STUDIO_PLAYWRIGHT_MODULE`，不能将默认跳过视为页面验收。 + +此次为审查问题修复，RFC 保持 `In Implementation`。M4 尚有高级操作，M5/M6 与真实试用未完成，不以本次修复宣称整个 RFC 或体验指标已验收。 + +### 2026-09-14 评审意见采纳 + +| 意见 | 结论与修订 | +| --- | --- | +| 1. 缺批量事务 | 采纳有界内存批量；原无写文件路径并无磁盘部分提交，“极度低效”待实测，不扩为 SDK/分布式事务 | +| 2. 缺原地同步 | 采纳显式 fix-deps 文件包装；默认预览，只补确定依赖，保留工具变换与文件副作用的责任边界 | +| 3. 占位破坏默认回退 | 占位用于明确断开，机制保留；补 reset 操作、重新连接覆盖和生命周期，纠正“永久阻断”的理解 | +| 4. outputs 级联同步 | 不采纳按内部 egress 增删池；补内部出口与 bridge 槽的区别、保真和重新校验规则 | +| 5. 扇出共享语义 | 采纳更精确操作范围及隐式消费者测试；图扇出不等同数据 Cardinality 1:N | +| 6. 试用阻断工程 | 拆 M0a 工程基线与 M0b 并行体验基线；缺参与者不阻断工程交付，体验结论仍待真实记录 | diff --git a/doc/rfcs/README.md b/doc/rfcs/README.md index fea4e1a5..8ba1ee3f 100644 --- a/doc/rfcs/README.md +++ b/doc/rfcs/README.md @@ -24,6 +24,7 @@ | **RFC-0051** | 开发者任务路径、测试生成与修复诊断 | `In Implementation` | `v10.x` | 流程编排层、能力节点层、模型执行层 / Tooling / Docs | [0051-developer-task-experience.md](0051-developer-task-experience.md) | | **RFC-0052** | 面向基础 C++ 开发者的 Node 作者接口重构 | `In Implementation` | 投产前 / Catalog v3 | 流程编排层、能力节点层 / Tooling / Docs | [0052-function-oriented-node-authoring.md](0052-function-oriented-node-authoring.md) | | **RFC-0054** | Control 作者接口与不可变配置快照 | `In Implementation` | 投产前 | 能力节点层 / Tooling / Docs | [0054-controlled-configuration-snapshots.md](0054-controlled-configuration-snapshots.md) | +| **RFC-0057** | 降低 Pipeline 编排认知负担的实施方案 | `In Implementation` | 投产前 | 流程编排层 / Integration Tooling / Docs | [0057-pipeline-composition-experience.md](0057-pipeline-composition-experience.md) | RFC-0054 是接续 RFC-0052 与已交付 RFC-0053、RFC-0055 的实施规格。`Proposed` 表示设计方案已形成，其中拟议 API、迁移与验证尚未实施，不作为当前 SDK 功能说明。 diff --git a/src/tools/CMakeLists.txt b/src/tools/CMakeLists.txt index 1319756a..63ddacb0 100644 --- a/src/tools/CMakeLists.txt +++ b/src/tools/CMakeLists.txt @@ -2,6 +2,8 @@ add_executable(alg_show alg_show.cpp) target_link_libraries(alg_show PRIVATE edgeflow_internal_headers nlohmann_json::nlohmann_json) -add_executable(alg_pipeline_tool alg_pipeline_tool.cpp) +add_executable(alg_pipeline_tool + alg_pipeline_tool.cpp + pipeline_authoring.cpp) target_link_libraries(alg_pipeline_tool PRIVATE edgeflow_internal_headers llm_edgeflow::internal_runtime) diff --git a/src/tools/alg_pipeline_tool.cpp b/src/tools/alg_pipeline_tool.cpp index aaca4182..1879d75b 100644 --- a/src/tools/alg_pipeline_tool.cpp +++ b/src/tools/alg_pipeline_tool.cpp @@ -11,6 +11,7 @@ #include "core/pipeline_validator.h" #include "edgeflow/operator/interface.h" #include "nlohmann/json.hpp" +#include "tools/pipeline_authoring.h" namespace { @@ -182,7 +183,9 @@ void Usage() { << " alg_pipeline_tool validate FILE|--stdin [--explain]\n" << " alg_pipeline_tool plan FILE|--stdin [--explain]\n"; std::cerr - << " alg_pipeline_tool resolve-conf FILE [--root DIR] [--depth N]\n"; + << " alg_pipeline_tool resolve-conf FILE [--root DIR] [--depth N]\n" + << " alg_pipeline_tool edit --stdin\n" + << " alg_pipeline_tool fix-deps FILE [--in-place]\n"; } } // namespace @@ -359,6 +362,78 @@ int main(int argc, char** argv) { return report.ok ? 0 : 1; } + if (command == "edit") { + if (argc != 3 || std::string(argv[2]) != "--stdin") { + Usage(); + return 2; + } + std::string input_str; + char buffer[65536]; + while (std::cin.read(buffer, sizeof(buffer)) || std::cin.gcount() > 0) { + input_str.append(buffer, std::cin.gcount()); + if (input_str.size() > 4 * 1024 * 1024) { + nlohmann::json err_res = { + {"schema_version", 1}, + {"ok", false}, + {"diagnostics", + {{{"code", "AUTHORING_ERROR"}, + {"path", "/"}, + {"message", "REQUEST_TOO_LARGE: 请求输入大小超过单次上限 4 MiB"}, + {"severity", "error"}}}}}; + std::cout << err_res.dump(2) << std::endl; + return 1; + } + } + nlohmann::json request; + try { + request = nlohmann::json::parse(input_str); + } catch (const std::exception& e) { + std::cout << Error("JSON_READ", e.what()).dump(2) << std::endl; + return 1; + } + try { + auto result = llm_edgeflow::PipelineAuthoring::ApplyRequest(request); + std::cout << result.ToJson().dump(2) << std::endl; + return result.ok ? 0 : 1; + } catch (const std::exception& error) { + std::cout << Error("AUTHORING_ERROR", error.what()).dump(2) << std::endl; + return 1; + } catch (...) { + std::cout << Error("AUTHORING_ERROR", "未知编排工具错误").dump(2) + << std::endl; + return 1; + } + } + + if (command == "fix-deps") { + if (argc < 3 || argc > 4) { + Usage(); + return 2; + } + std::string file = argv[2]; + bool in_place = false; + if (argc == 4) { + if (std::string(argv[3]) == "--in-place") { + in_place = true; + } else { + Usage(); + return 2; + } + } + try { + auto result = llm_edgeflow::PipelineAuthoring::FixDeps(file, in_place); + std::cout << result.ToJson().dump(2) << std::endl; + return result.ok ? 0 : 1; + } catch (const std::exception& error) { + std::cout << Error("AUTHORING_ERROR", error.what()).dump(2) << std::endl; + return 1; + } catch (...) { + std::cout << Error("AUTHORING_ERROR", "未知编排工具错误").dump(2) + << std::endl; + return 1; + } + } + Usage(); return 2; } diff --git a/src/tools/pipeline_authoring.cpp b/src/tools/pipeline_authoring.cpp new file mode 100644 index 00000000..ac64bf22 --- /dev/null +++ b/src/tools/pipeline_authoring.cpp @@ -0,0 +1,1714 @@ +#include "tools/pipeline_authoring.h" + +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "core/pipeline_catalog.h" +#include "core/pipeline_validator.h" + +namespace llm_edgeflow { + +namespace fs = std::filesystem; + +namespace { + +nlohmann::json* FindNodeById(nlohmann::json* pipeline, const std::string& id) { + if (!pipeline || !pipeline->contains("pipeline") || + !(*pipeline)["pipeline"].is_array()) { + return nullptr; + } + nlohmann::json* found = nullptr; + for (auto& node : (*pipeline)["pipeline"]) { + if (node.is_object() && node.value("id", "") == id) { + if (found) throw std::invalid_argument("AMBIGUOUS_NODE_ID: " + id); + found = &node; + } + } + return found; +} + +std::string GetEffectiveOutputKey(const nlohmann::json& node, + const std::string& port_name) { + if (node.contains("ports") && node["ports"].is_object() && + node["ports"].contains("outputs") && + node["ports"]["outputs"].is_object() && + node["ports"]["outputs"].contains(port_name) && + node["ports"]["outputs"][port_name].is_string()) { + return node["ports"]["outputs"][port_name].get(); + } + return port_name; +} + +std::optional GetEffectiveInputKey(const nlohmann::json& node, + const std::string& port_name) { + if (node.contains("ports") && node["ports"].is_object() && + node["ports"].contains("inputs") && node["ports"]["inputs"].is_object() && + node["ports"]["inputs"].contains(port_name) && + node["ports"]["inputs"][port_name].is_string()) { + return node["ports"]["inputs"][port_name].get(); + } + std::string node_type = node.value("node_type", ""); + auto def = PipelineCatalog::FindNode(node_type); + if (def) { + for (const auto& in_p : def->inputs) { + if (in_p.logical_name == port_name) { + if (in_p.required) { + return port_name; + } else { + return std::nullopt; + } + } + } + } + return std::nullopt; +} + +// Validate the transient editing protocol, not the persisted Pipeline schema. +void CheckFields(const nlohmann::json& value, + std::initializer_list allowed) { + if (!value.is_object()) throw std::invalid_argument("字段必须是对象"); + for (const auto& item : value.items()) { + if (std::none_of(allowed.begin(), allowed.end(), + [&](const char* key) { return item.key() == key; })) { + throw std::invalid_argument("UNKNOWN_FIELD: " + item.key()); + } + } +} + +void RequireString(const nlohmann::json& value, const char* key) { + if (!value.contains(key) || !value[key].is_string() || + value[key].get().empty()) { + throw std::invalid_argument(std::string("字段必须是非空字符串: ") + key); + } +} + +void CheckEndpoint(const nlohmann::json& endpoint) { + CheckFields(endpoint, {"node_id", "port"}); + RequireString(endpoint, "node_id"); + RequireString(endpoint, "port"); +} + +void CheckOperation(const nlohmann::json& op) { + RequireString(op, "kind"); + const auto kind = op["kind"].get(); + if (kind == "add_node") { + CheckFields(op, {"kind", "node_type", "id", "config"}); + RequireString(op, "node_type"); + if (op.contains("id")) RequireString(op, "id"); + if (op.contains("config") && !op["config"].is_object()) + throw std::invalid_argument("config 必须是对象"); + } else if (kind == "remove_node" || kind == "rename_node") { + if (kind == "remove_node") + CheckFields(op, {"kind", "node_id"}); + else { + CheckFields(op, {"kind", "node_id", "new_id"}); + RequireString(op, "new_id"); + } + RequireString(op, "node_id"); + } else if (kind == "connect" || kind == "disconnect") { + CheckFields(op, {"kind", "source", "target"}); + CheckEndpoint(op.at("source")); + CheckEndpoint(op.at("target")); + } else if (kind == "reset_input_binding") { + CheckFields(op, {"kind", "target"}); + CheckEndpoint(op.at("target")); + } else if (kind == "add_dependency" || kind == "remove_dependency") { + CheckFields(op, {"kind", "node_id", "depends_on_id"}); + RequireString(op, "node_id"); + RequireString(op, "depends_on_id"); + } else { + throw std::invalid_argument("UNKNOWN_OPERATION_KIND: " + kind); + } +} + +std::string CheckGraphEndpoint(nlohmann::json* pipeline, + const nlohmann::json& endpoint, bool output) { + const auto id = endpoint.at("node_id").get(); + const auto port = endpoint.at("port").get(); + if (id == "$ingress" || id == "$egress") { + if ((output && id != "$ingress") || (!output && id != "$egress")) + throw std::invalid_argument("INVALID_ENDPOINT: 业务端点方向错误"); + const auto biz = PipelineCatalog::FindBiz(pipeline->value("biz_name", "")); + if (biz) { + const auto& ports = output ? biz->ingress : biz->egress; + for (const auto& definition : ports) + if (definition.blackboard_key == port) return port; + } + } else { + auto* node = FindNodeById(pipeline, id); + if (!node) throw std::invalid_argument("NODE_NOT_FOUND: " + id); + const auto definition = + PipelineCatalog::FindNode(node->value("node_type", "")); + if (definition) { + const auto& ports = output ? definition->outputs : definition->inputs; + for (const auto& candidate : ports) { + if (candidate.logical_name == port) + return output ? GetEffectiveOutputKey(*node, port) : port; + } + } + } + throw std::invalid_argument("UNKNOWN_PORT: " + id + "." + port); +} + +void CheckUniqueProducer(const nlohmann::json& pipeline, + const std::string& key) { + size_t count = 0; + auto biz = PipelineCatalog::FindBiz(pipeline.value("biz_name", "")); + if (biz) + for (const auto& port : biz->ingress) + if (port.blackboard_key == key) ++count; + for (const auto& node : pipeline.at("pipeline")) { + auto definition = PipelineCatalog::FindNode(node.value("node_type", "")); + if (definition) + for (const auto& port : definition->outputs) + if (GetEffectiveOutputKey(node, port.logical_name) == key) ++count; + } + if (count != 1) throw std::invalid_argument("AMBIGUOUS_PRODUCER: " + key); +} + +class FileDescriptor { + public: + explicit FileDescriptor(int value) : value_(value) {} + ~FileDescriptor() { + if (value_ >= 0) close(value_); + } + int Get() const { return value_; } + bool Close() { + int value = value_; + value_ = -1; + return close(value) == 0; + } + + private: + int value_; +}; + +std::string ReadRegularFile(const std::string& path, struct stat* identity) { + FileDescriptor descriptor( + open(path.c_str(), O_RDONLY | O_NOFOLLOW | O_NONBLOCK)); + if (descriptor.Get() < 0 || fstat(descriptor.Get(), identity) != 0 || + !S_ISREG(identity->st_mode)) + throw std::runtime_error("CANNOT_READ_FILE: " + path); + std::string contents; + char buffer[65536]; + for (;;) { + const auto size = read(descriptor.Get(), buffer, sizeof(buffer)); + if (size < 0 && errno == EINTR) continue; + if (size < 0) throw std::runtime_error("FILE_READ_FAILED: " + path); + if (size == 0) return contents; + contents.append(buffer, static_cast(size)); + } +} + +void ReplaceFile(const fs::path& path, const std::string& contents, + const std::string& original, const struct stat& identity) { + std::string pattern = + (path.parent_path() / + (".tmp_fix_deps_" + path.filename().string() + "_XXXXXX")) + .string(); + FileDescriptor descriptor(mkstemp(pattern.data())); + if (descriptor.Get() < 0) throw std::runtime_error("CANNOT_WRITE_TEMP_FILE"); + // Cleanup covers write, metadata, conflict and rename failures. + struct Cleanup { + std::string path; + ~Cleanup() { unlink(path.c_str()); } + } cleanup{pattern}; + size_t offset = 0; + while (offset < contents.size()) { + const auto size = write(descriptor.Get(), contents.data() + offset, + contents.size() - offset); + if (size < 0 && errno == EINTR) continue; + if (size <= 0) throw std::runtime_error("FILE_WRITE_FAILED"); + offset += static_cast(size); + } + if (fchmod(descriptor.Get(), identity.st_mode & 07777) != 0 || + fsync(descriptor.Get()) != 0 || !descriptor.Close()) + throw std::runtime_error("FILE_WRITE_FAILED: flush/close/permissions"); + struct stat current { + }, named{}; + const auto bytes = ReadRegularFile(path.string(), ¤t); + if (lstat(path.c_str(), &named) != 0 || !S_ISREG(named.st_mode) || + named.st_dev != current.st_dev || named.st_ino != current.st_ino || + current.st_dev != identity.st_dev || current.st_ino != identity.st_ino || + current.st_mode != identity.st_mode || + current.st_size != identity.st_size || + current.st_mtim.tv_sec != identity.st_mtim.tv_sec || + current.st_mtim.tv_nsec != identity.st_mtim.tv_nsec || bytes != original) + throw std::runtime_error("FILE_CONFLICT: 文件在读取后已被外部修改"); + if (rename(pattern.c_str(), path.c_str()) != 0) + throw std::runtime_error("RENAME_FAILED: " + std::string(strerror(errno))); +} + +} // namespace + +nlohmann::json AuthoringChange::ToJson() const { + nlohmann::json j = {{"action", action}, + {"node_id", node_id}, + {"affected_nodes", affected_nodes}, + {"description", description}}; + if (!port.empty()) { + j["port"] = port; + } + return j; +} + +nlohmann::json AuthoringResult::ToJson() const { + nlohmann::json j = {{"schema_version", schema_version}, {"ok", ok}}; + if (pipeline.has_value()) { + j["pipeline"] = *pipeline; + } + j["changes"] = nlohmann::json::array(); + for (const auto& change : changes) { + j["changes"].push_back(change.ToJson()); + } + if (!validation.empty()) { + j["validation"] = validation; + } + if (!diagnostics.empty()) { + j["diagnostics"] = nlohmann::json::array(); + for (const auto& d : diagnostics) { + j["diagnostics"].push_back({{"code", "AUTHORING_ERROR"}, + {"path", "/"}, + {"message", d}, + {"severity", "error"}}); + } + } + if (failed_operation_index.has_value()) { + j["failed_operation_index"] = *failed_operation_index; + } + return j; +} + +nlohmann::json FixDepsResult::ToJson() const { + nlohmann::json j = {{"schema_version", schema_version}, + {"ok", ok}, + {"target_file", target_file}, + {"written", written}}; + j["changes"] = nlohmann::json::array(); + for (const auto& change : changes) { + j["changes"].push_back(change.ToJson()); + } + if (!validation.empty()) { + j["validation"] = validation; + } + if (!diagnostics.empty()) { + j["diagnostics"] = nlohmann::json::array(); + for (const auto& d : diagnostics) { + j["diagnostics"].push_back({{"code", "FIX_DEPS_ERROR"}, + {"path", "/"}, + {"message", d}, + {"severity", "error"}}); + } + } + return j; +} + +std::unordered_map> +PipelineAuthoring::BuildDependencyGraph(const nlohmann::json& pipeline) { + std::unordered_map> graph; + if (!pipeline.is_object() || !pipeline.contains("pipeline") || + !pipeline["pipeline"].is_array()) { + return graph; + } + for (const auto& node : pipeline["pipeline"]) { + if (!node.is_object() || !node.contains("id") || !node["id"].is_string()) { + continue; + } + std::string id = node["id"].get(); + std::vector deps; + if (node.contains("depends_on") && node["depends_on"].is_array()) { + for (const auto& dep : node["depends_on"]) { + if (dep.is_string()) { + deps.push_back(dep.get()); + } + } + } + graph[id] = std::move(deps); + } + return graph; +} + +bool PipelineAuthoring::IsAncestor( + const std::string& potential_ancestor, const std::string& node_id, + const std::unordered_map>& + dep_graph) { + if (potential_ancestor.empty() || node_id.empty()) return false; + if (potential_ancestor == node_id) return true; + std::unordered_set visited; + std::vector stack; + auto it = dep_graph.find(node_id); + if (it != dep_graph.end()) { + for (const auto& dep : it->second) { + stack.push_back(dep); + } + } + while (!stack.empty()) { + std::string curr = stack.back(); + stack.pop_back(); + if (curr == potential_ancestor) return true; + if (!visited.insert(curr).second) continue; + auto curr_it = dep_graph.find(curr); + if (curr_it != dep_graph.end()) { + for (const auto& dep : curr_it->second) { + if (visited.find(dep) == visited.end()) { + stack.push_back(dep); + } + } + } + } + return false; +} + +std::unordered_set PipelineAuthoring::GetOccupiedKeys( + const nlohmann::json& pipeline) { + std::unordered_set occupied; + if (!pipeline.is_object()) return occupied; + + std::string biz_name = pipeline.value("biz_name", ""); + if (!biz_name.empty()) { + auto biz = PipelineCatalog::FindBiz(biz_name); + if (biz) { + for (const auto& in : biz->ingress) { + occupied.insert(in.blackboard_key); + } + for (const auto& out : biz->egress) { + occupied.insert(out.blackboard_key); + } + } + } + + if (pipeline.contains("pipeline") && pipeline["pipeline"].is_array()) { + for (const auto& node : pipeline["pipeline"]) { + if (!node.is_object()) continue; + std::string node_type = node.value("node_type", ""); + auto node_def = PipelineCatalog::FindNode(node_type); + + if (node.contains("ports") && node["ports"].is_object()) { + const auto& ports = node["ports"]; + if (ports.contains("inputs") && ports["inputs"].is_object()) { + for (const auto& [k, v] : ports["inputs"].items()) { + if (v.is_string()) { + occupied.insert(v.get()); + } + } + } + if (ports.contains("outputs") && ports["outputs"].is_object()) { + for (const auto& [k, v] : ports["outputs"].items()) { + if (v.is_string()) { + occupied.insert(v.get()); + } + } + } + } + + if (node_def) { + for (const auto& in : node_def->inputs) { + if (in.required) { + bool has_explicit = false; + if (node.contains("ports") && node["ports"].is_object() && + node["ports"].contains("inputs") && + node["ports"]["inputs"].is_object() && + node["ports"]["inputs"].contains(in.logical_name)) { + has_explicit = true; + } + if (!has_explicit) { + occupied.insert(in.logical_name); + } + } + } + for (const auto& out : node_def->outputs) { + bool has_explicit = false; + if (node.contains("ports") && node["ports"].is_object() && + node["ports"].contains("outputs") && + node["ports"]["outputs"].is_object() && + node["ports"]["outputs"].contains(out.logical_name)) { + has_explicit = true; + } + if (!has_explicit) { + occupied.insert(out.logical_name); + } + } + } + } + } + return occupied; +} + +std::string PipelineAuthoring::AllocateKey( + const std::string& base, const std::unordered_set& occupied) { + if (occupied.find(base) == occupied.end()) { + return base; + } + int suffix = 1; + while (true) { + std::string cand = base + "_" + std::to_string(suffix); + if (occupied.find(cand) == occupied.end()) { + return cand; + } + ++suffix; + } +} + +bool PipelineAuthoring::ApplyOperation(nlohmann::json* pipeline, + const nlohmann::json& operation, + std::vector* changes, + std::string* error) { + if (!pipeline || !pipeline->is_object() || !operation.is_object()) { + if (error) *error = "无效的 pipeline 或 operation 对象"; + return false; + } + CheckOperation(operation); + std::string kind = operation.value("kind", ""); + if (kind == "connect" || kind == "disconnect") { + auto key = CheckGraphEndpoint(pipeline, operation.at("source"), true); + CheckGraphEndpoint(pipeline, operation.at("target"), false); + CheckUniqueProducer(*pipeline, key); + } + if (kind == "reset_input_binding") { + CheckGraphEndpoint(pipeline, operation.at("target"), false); + } + if (kind.empty()) { + if (error) *error = "operation 缺少 kind 字段"; + return false; + } + + std::string biz_name = pipeline->value("biz_name", ""); + auto biz = PipelineCatalog::FindBiz(biz_name); + + if (kind == "add_node") { + std::string node_type = operation.value("node_type", ""); + if (node_type.empty()) { + if (error) *error = "add_node 缺少 node_type 字段"; + return false; + } + auto node_def = PipelineCatalog::FindNode(node_type); + if (!node_def) { + if (error) *error = "UNKNOWN_NODE_TYPE: " + node_type; + return false; + } + + std::string node_id; + if (operation.contains("id")) { + if (!operation["id"].is_string()) { + if (error) *error = "node id 必须是字符串"; + return false; + } + node_id = operation["id"].get(); + if (node_id.empty()) { + if (error) *error = "node id 不能为空"; + return false; + } + if (node_id == "$ingress" || node_id == "$egress") { + if (error) + *error = "RESERVED_NODE_ID: 节点 ID 不能使用保留名 " + node_id; + return false; + } + if (FindNodeById(pipeline, node_id) != nullptr) { + if (error) *error = "DUPLICATE_NODE_ID: " + node_id; + return false; + } + } else { + size_t index = (*pipeline)["pipeline"].size(); + while (true) { + std::string cand = "node_" + std::to_string(index) + "_" + node_type; + if (FindNodeById(pipeline, cand) == nullptr) { + node_id = cand; + break; + } + ++index; + } + } + + nlohmann::json config = operation.value("config", nlohmann::json::object()); + if (!config.is_object()) { + if (error) *error = "config 必须是 JSON 对象"; + return false; + } + + nlohmann::json new_node; + new_node["id"] = node_id; + new_node["node_type"] = node_type; + new_node["depends_on"] = nlohmann::json::array(); + new_node["ports"] = {{"inputs", nlohmann::json::object()}, + {"outputs", nlohmann::json::object()}}; + new_node["config"] = config; + + auto occupied = GetOccupiedKeys(*pipeline); + for (const auto& out : node_def->outputs) { + std::string base = node_id + "__" + out.logical_name; + std::string out_key = AllocateKey(base, occupied); + occupied.insert(out_key); + new_node["ports"]["outputs"][out.logical_name] = out_key; + } + + (*pipeline)["pipeline"].push_back(std::move(new_node)); + if (changes) { + changes->push_back({"add_node", + node_id, + "", + {node_id}, + "添加节点 " + node_id + " (" + node_type + ")"}); + } + return true; + } + + if (kind == "remove_node") { + std::string node_id = operation.value("node_id", ""); + if (node_id.empty()) { + if (error) *error = "remove_node 缺少 node_id"; + return false; + } + auto* target_node = FindNodeById(pipeline, node_id); + if (!target_node) { + if (error) *error = "NODE_NOT_FOUND: " + node_id; + return false; + } + + std::unordered_set out_keys; + if (target_node->contains("ports") && + (*target_node)["ports"].contains("outputs") && + (*target_node)["ports"]["outputs"].is_object()) { + for (const auto& [k, v] : (*target_node)["ports"]["outputs"].items()) { + if (v.is_string()) { + out_keys.insert(v.get()); + } + } + } + auto node_def = + PipelineCatalog::FindNode(target_node->value("node_type", "")); + if (node_def) { + for (const auto& out : node_def->outputs) { + if (!target_node->contains("ports") || + !(*target_node)["ports"].contains("outputs") || + !(*target_node)["ports"]["outputs"].contains(out.logical_name)) { + out_keys.insert(out.logical_name); + } + } + } + + bool connects_to_egress = false; + if (biz) { + for (const auto& out_k : out_keys) { + for (const auto& eg : biz->egress) { + if (eg.blackboard_key == out_k) { + connects_to_egress = true; + break; + } + } + if (connects_to_egress) break; + } + } + + std::vector affected = {node_id}; + auto occupied = GetOccupiedKeys(*pipeline); + + for (auto& other : (*pipeline)["pipeline"]) { + if (!other.is_object() || other.value("id", "") == node_id) continue; + std::string other_id = other.value("id", ""); + auto other_def = PipelineCatalog::FindNode(other.value("node_type", "")); + bool other_affected = false; + + if (other.contains("ports") && other["ports"].is_object() && + other["ports"].contains("inputs") && + other["ports"]["inputs"].is_object()) { + std::vector keys_to_erase; + for (auto& [in_k, in_v] : other["ports"]["inputs"].items()) { + if (!in_v.is_string()) continue; + std::string val = in_v.get(); + if (out_keys.find(val) != out_keys.end()) { + bool req = true; + if (other_def) { + for (const auto& in_p : other_def->inputs) { + if (in_p.logical_name == in_k) { + req = in_p.required; + break; + } + } + } + if (req) { + std::string placeholder = + AllocateKey(other_id + "__unconnected__" + in_k, occupied); + occupied.insert(placeholder); + in_v = placeholder; + } else { + keys_to_erase.push_back(in_k); + } + other_affected = true; + } + } + for (const auto& k : keys_to_erase) { + other["ports"]["inputs"].erase(k); + } + } + + if (other_def) { + for (const auto& in_p : other_def->inputs) { + if (in_p.required && + out_keys.find(in_p.logical_name) != out_keys.end()) { + bool has_explicit = false; + if (other.contains("ports") && other["ports"].is_object() && + other["ports"].contains("inputs") && + other["ports"]["inputs"].is_object() && + other["ports"]["inputs"].contains(in_p.logical_name)) { + has_explicit = true; + } + if (!has_explicit) { + if (!other.contains("ports") || !other["ports"].is_object()) { + other["ports"] = nlohmann::json::object(); + } + if (!other["ports"].contains("inputs") || + !other["ports"]["inputs"].is_object()) { + other["ports"]["inputs"] = nlohmann::json::object(); + } + std::string placeholder = AllocateKey( + other_id + "__unconnected__" + in_p.logical_name, occupied); + occupied.insert(placeholder); + other["ports"]["inputs"][in_p.logical_name] = placeholder; + other_affected = true; + } + } + } + } + + if (other.contains("depends_on") && other["depends_on"].is_array()) { + auto& deps = other["depends_on"]; + for (auto it = deps.begin(); it != deps.end();) { + if (it->is_string() && it->get() == node_id) { + it = deps.erase(it); + other_affected = true; + } else { + ++it; + } + } + } + + if (other_affected) { + affected.push_back(other_id); + } + } + + if (connects_to_egress) { + affected.push_back("$egress"); + } + + auto& pipe_arr = (*pipeline)["pipeline"]; + for (auto it = pipe_arr.begin(); it != pipe_arr.end(); ++it) { + if (it->is_object() && it->value("id", "") == node_id) { + pipe_arr.erase(it); + break; + } + } + + if (changes) { + std::string desc = "删除节点 " + node_id; + if (connects_to_egress) { + desc += "（断开业务出口）"; + } + changes->push_back({"remove_node", node_id, "", affected, desc}); + } + return true; + } + + if (kind == "rename_node") { + std::string node_id = operation.value("node_id", ""); + if (!operation.contains("new_id") || !operation["new_id"].is_string()) { + if (error) *error = "rename_node 缺少 new_id 字符串"; + return false; + } + std::string new_id = operation["new_id"].get(); + if (node_id.empty() || new_id.empty()) { + if (error) *error = "rename_node 缺少 node_id 或 new_id"; + return false; + } + if (new_id == "$ingress" || new_id == "$egress") { + if (error) *error = "RESERVED_NODE_ID: 节点 ID 不能使用保留名 " + new_id; + return false; + } + auto* target_node = FindNodeById(pipeline, node_id); + if (!target_node) { + if (error) *error = "NODE_NOT_FOUND: " + node_id; + return false; + } + if (node_id == new_id) { + if (changes) { + changes->push_back({"rename_node", + node_id, + "", + {node_id}, + "重命名节点 " + node_id + "（无变化）"}); + } + return true; + } + if (FindNodeById(pipeline, new_id) != nullptr) { + if (error) *error = "DUPLICATE_NODE_ID: " + new_id; + return false; + } + + (*target_node)["id"] = new_id; + std::vector affected = {new_id}; + for (auto& other : (*pipeline)["pipeline"]) { + if (!other.is_object() || other.value("id", "") == new_id) continue; + if (other.contains("depends_on") && other["depends_on"].is_array()) { + bool updated = false; + for (auto& dep : other["depends_on"]) { + if (dep.is_string() && dep.get() == node_id) { + dep = new_id; + updated = true; + } + } + if (updated) { + affected.push_back(other.value("id", "")); + } + } + } + + if (changes) { + changes->push_back({"rename_node", new_id, "", affected, + "重命名节点 " + node_id + " 为 " + new_id}); + } + return true; + } + + if (kind == "connect") { + if (!operation.contains("source") || !operation["source"].is_object() || + !operation.contains("target") || !operation["target"].is_object()) { + if (error) *error = "connect 缺少 source 或 target 对象"; + return false; + } + std::string src_id = operation["source"].value("node_id", ""); + std::string src_port = operation["source"].value("port", ""); + std::string tgt_id = operation["target"].value("node_id", ""); + std::string tgt_port = operation["target"].value("port", ""); + if (src_id.empty() || src_port.empty() || tgt_id.empty() || + tgt_port.empty()) { + if (error) *error = "connect 端点 node_id 或 port 不能为空"; + return false; + } + + if (src_id == "$ingress") { + if (!biz) { + if (error) *error = "UNKNOWN_BIZ: " + biz_name; + return false; + } + auto in_it = std::find_if(biz->ingress.begin(), biz->ingress.end(), + [&](const BizPortDefinition& p) { + return p.blackboard_key == src_port; + }); + if (in_it == biz->ingress.end()) { + if (error) *error = "业务输入端口不存在: " + src_port; + return false; + } + if (tgt_id == "$ingress" || tgt_id == "$egress") { + if (error) *error = "目标节点不能是业务端口"; + return false; + } + auto* tgt_node = FindNodeById(pipeline, tgt_id); + if (!tgt_node) { + if (error) *error = "NODE_NOT_FOUND: " + tgt_id; + return false; + } + auto tgt_def = + PipelineCatalog::FindNode(tgt_node->value("node_type", "")); + if (!tgt_def) { + if (error) + *error = "UNKNOWN_NODE_TYPE: " + tgt_node->value("node_type", ""); + return false; + } + auto tgt_p_it = + std::find_if(tgt_def->inputs.begin(), tgt_def->inputs.end(), + [&](const NodePortDefinition& p) { + return p.logical_name == tgt_port; + }); + if (tgt_p_it == tgt_def->inputs.end()) { + if (error) *error = "目标输入端口不存在: " + tgt_port; + return false; + } + if (in_it->type_id != tgt_p_it->type_id) { + if (error) { + *error = "端口不存在或数据类型不兼容: " + in_it->type_id + + " != " + tgt_p_it->type_id; + } + return false; + } + + if (!tgt_node->contains("ports")) + (*tgt_node)["ports"] = nlohmann::json::object(); + if (!(*tgt_node)["ports"].contains("inputs")) { + (*tgt_node)["ports"]["inputs"] = nlohmann::json::object(); + } + (*tgt_node)["ports"]["inputs"][tgt_port] = src_port; + + if (changes) { + changes->push_back( + {"connect", + tgt_id, + tgt_port, + {tgt_id}, + "连接业务输入 " + src_port + " 到 " + tgt_id + "." + tgt_port}); + } + return true; + } + + if (tgt_id == "$egress") { + if (!biz) { + if (error) *error = "UNKNOWN_BIZ: " + biz_name; + return false; + } + auto out_it = std::find_if(biz->egress.begin(), biz->egress.end(), + [&](const BizPortDefinition& p) { + return p.blackboard_key == tgt_port; + }); + if (out_it == biz->egress.end()) { + if (error) *error = "业务输出端口不存在: " + tgt_port; + return false; + } + auto* src_node = FindNodeById(pipeline, src_id); + if (!src_node) { + if (error) *error = "NODE_NOT_FOUND: " + src_id; + return false; + } + auto src_def = + PipelineCatalog::FindNode(src_node->value("node_type", "")); + if (!src_def) { + if (error) + *error = "UNKNOWN_NODE_TYPE: " + src_node->value("node_type", ""); + return false; + } + auto src_p_it = + std::find_if(src_def->outputs.begin(), src_def->outputs.end(), + [&](const NodePortDefinition& p) { + return p.logical_name == src_port; + }); + if (src_p_it == src_def->outputs.end()) { + if (error) *error = "源输出端口不存在: " + src_port; + return false; + } + if (src_p_it->type_id != out_it->type_id) { + if (error) { + *error = "端口不存在或数据类型不兼容: " + src_p_it->type_id + + " != " + out_it->type_id; + } + return false; + } + + // Check conflict: does any other node produce tgt_port? + for (const auto& other : (*pipeline)["pipeline"]) { + if (!other.is_object()) continue; + std::string o_id = other.value("id", ""); + auto o_def = PipelineCatalog::FindNode(other.value("node_type", "")); + if (o_id != src_id) { + if (o_def) { + for (const auto& out : o_def->outputs) { + if (GetEffectiveOutputKey(other, out.logical_name) == tgt_port) { + if (error) *error = "业务输出已有生产者，请先断开原连线"; + return false; + } + } + } + } else { + if (o_def) { + for (const auto& out : o_def->outputs) { + if (out.logical_name != src_port && + GetEffectiveOutputKey(other, out.logical_name) == tgt_port) { + if (error) *error = "业务输出已有生产者，请先断开原连线"; + return false; + } + } + } + } + } + + std::string current_key = src_port; + if (src_node->contains("ports") && + (*src_node)["ports"].contains("outputs") && + (*src_node)["ports"]["outputs"].contains(src_port) && + (*src_node)["ports"]["outputs"][src_port].is_string()) { + current_key = + (*src_node)["ports"]["outputs"][src_port].get(); + } + + // Check if current_key is already mapped to another egress port + for (const auto& eg : biz->egress) { + if (eg.blackboard_key != tgt_port && eg.blackboard_key == current_key) { + if (error) { + *error = "当前输出已连接到另一业务出口，无法发布到两个不同契约 key"; + } + return false; + } + } + + if (current_key == tgt_port) { + return true; + } + + if (!src_node->contains("ports")) + (*src_node)["ports"] = nlohmann::json::object(); + if (!(*src_node)["ports"].contains("outputs")) { + (*src_node)["ports"]["outputs"] = nlohmann::json::object(); + } + (*src_node)["ports"]["outputs"][src_port] = tgt_port; + + std::vector affected = {src_id}; + for (auto& consumer : (*pipeline)["pipeline"]) { + if (!consumer.is_object()) continue; + std::string c_id = consumer.value("id", ""); + if (c_id == src_id) continue; + bool c_affected = false; + if (consumer.contains("ports") && + consumer["ports"].contains("inputs") && + consumer["ports"]["inputs"].is_object()) { + for (auto& [in_k, in_v] : consumer["ports"]["inputs"].items()) { + if (in_v.is_string() && in_v.get() == current_key) { + in_v = tgt_port; + c_affected = true; + } + } + } + auto c_def = PipelineCatalog::FindNode(consumer.value("node_type", "")); + if (c_def) { + for (const auto& in_p : c_def->inputs) { + if (in_p.required && in_p.logical_name == current_key) { + if (!consumer.contains("ports") || + !consumer["ports"].contains("inputs") || + !consumer["ports"]["inputs"].contains(in_p.logical_name)) { + if (!consumer.contains("ports")) { + consumer["ports"] = nlohmann::json::object(); + } + if (!consumer["ports"].contains("inputs")) { + consumer["ports"]["inputs"] = nlohmann::json::object(); + } + consumer["ports"]["inputs"][in_p.logical_name] = tgt_port; + c_affected = true; + } + } + } + } + if (c_affected) affected.push_back(c_id); + } + + if (changes) { + changes->push_back( + {"connect", src_id, src_port, affected, + "连接 " + src_id + "." + src_port + " 到业务输出 " + tgt_port}); + } + return true; + } + + // Node to Node + if (src_id == tgt_id) { + if (error) *error = "连线会形成环"; + return false; + } + auto* src_node = FindNodeById(pipeline, src_id); + auto* tgt_node = FindNodeById(pipeline, tgt_id); + if (!src_node) { + if (error) *error = "NODE_NOT_FOUND: " + src_id; + return false; + } + if (!tgt_node) { + if (error) *error = "NODE_NOT_FOUND: " + tgt_id; + return false; + } + + auto dep_graph = BuildDependencyGraph(*pipeline); + if (IsAncestor(tgt_id, src_id, dep_graph)) { + if (error) *error = "连线会形成环"; + return false; + } + + auto src_def = PipelineCatalog::FindNode(src_node->value("node_type", "")); + auto tgt_def = PipelineCatalog::FindNode(tgt_node->value("node_type", "")); + if (!src_def) { + if (error) + *error = "UNKNOWN_NODE_TYPE: " + src_node->value("node_type", ""); + return false; + } + if (!tgt_def) { + if (error) + *error = "UNKNOWN_NODE_TYPE: " + tgt_node->value("node_type", ""); + return false; + } + + auto src_p_it = + std::find_if(src_def->outputs.begin(), src_def->outputs.end(), + [&](const NodePortDefinition& p) { + return p.logical_name == src_port; + }); + auto tgt_p_it = std::find_if(tgt_def->inputs.begin(), tgt_def->inputs.end(), + [&](const NodePortDefinition& p) { + return p.logical_name == tgt_port; + }); + if (src_p_it == src_def->outputs.end()) { + if (error) *error = "源输出端口不存在: " + src_port; + return false; + } + if (tgt_p_it == tgt_def->inputs.end()) { + if (error) *error = "目标输入端口不存在: " + tgt_port; + return false; + } + if (src_p_it->type_id != tgt_p_it->type_id) { + if (error) { + *error = "端口不存在或数据类型不兼容: " + src_p_it->type_id + + " != " + tgt_p_it->type_id; + } + return false; + } + + // Ordinary fan-out never renames the producer's existing effective key. + std::string key = GetEffectiveOutputKey(*src_node, src_port); + + if (!tgt_node->contains("ports")) + (*tgt_node)["ports"] = nlohmann::json::object(); + if (!(*tgt_node)["ports"].contains("inputs")) { + (*tgt_node)["ports"]["inputs"] = nlohmann::json::object(); + } + (*tgt_node)["ports"]["inputs"][tgt_port] = key; + + if (!IsAncestor(src_id, tgt_id, dep_graph)) { + if (!tgt_node->contains("depends_on") || + !(*tgt_node)["depends_on"].is_array()) { + (*tgt_node)["depends_on"] = nlohmann::json::array(); + } + (*tgt_node)["depends_on"].push_back(src_id); + } + + if (changes) { + changes->push_back({"connect", + tgt_id, + tgt_port, + {src_id, tgt_id}, + "连接 " + src_id + "." + src_port + " 到 " + tgt_id + + "." + tgt_port}); + } + return true; + } + + if (kind == "disconnect") { + if (!operation.contains("source") || !operation["source"].is_object() || + !operation.contains("target") || !operation["target"].is_object()) { + if (error) *error = "disconnect 缺少 source 或 target 对象"; + return false; + } + std::string src_id = operation["source"].value("node_id", ""); + std::string src_port = operation["source"].value("port", ""); + std::string tgt_id = operation["target"].value("node_id", ""); + std::string tgt_port = operation["target"].value("port", ""); + + if (tgt_id == "$egress") { + auto* src_node = FindNodeById(pipeline, src_id); + if (!src_node) { + if (error) *error = "NODE_NOT_FOUND: " + src_id; + return false; + } + std::string current_key = GetEffectiveOutputKey(*src_node, src_port); + if (current_key != tgt_port) { + if (error) *error = "连线不存在或已被修改"; + return false; + } + + auto occupied = GetOccupiedKeys(*pipeline); + std::string new_key = AllocateKey(src_id + "__" + src_port, occupied); + occupied.insert(new_key); + if (!src_node->contains("ports") || !(*src_node)["ports"].is_object()) { + (*src_node)["ports"] = nlohmann::json::object(); + } + if (!(*src_node)["ports"].contains("outputs") || + !(*src_node)["ports"]["outputs"].is_object()) { + (*src_node)["ports"]["outputs"] = nlohmann::json::object(); + } + (*src_node)["ports"]["outputs"][src_port] = new_key; + + std::vector affected = {src_id}; + for (auto& consumer : (*pipeline)["pipeline"]) { + if (!consumer.is_object() || consumer.value("id", "") == src_id) + continue; + bool c_affected = false; + if (consumer.contains("ports") && + consumer["ports"].contains("inputs") && + consumer["ports"]["inputs"].is_object()) { + for (auto& [in_k, in_v] : consumer["ports"]["inputs"].items()) { + if (in_v.is_string() && in_v.get() == tgt_port) { + in_v = new_key; + c_affected = true; + } + } + } + auto c_def = PipelineCatalog::FindNode(consumer.value("node_type", "")); + if (c_def) { + for (const auto& in_p : c_def->inputs) { + if (in_p.required && in_p.logical_name == tgt_port) { + if (!consumer.contains("ports") || + !consumer["ports"].is_object() || + !consumer["ports"].contains("inputs") || + !consumer["ports"]["inputs"].is_object() || + !consumer["ports"]["inputs"].contains(in_p.logical_name)) { + if (!consumer.contains("ports") || + !consumer["ports"].is_object()) { + consumer["ports"] = nlohmann::json::object(); + } + if (!consumer["ports"].contains("inputs") || + !consumer["ports"]["inputs"].is_object()) { + consumer["ports"]["inputs"] = nlohmann::json::object(); + } + consumer["ports"]["inputs"][in_p.logical_name] = new_key; + c_affected = true; + } + } + } + } + if (c_affected) affected.push_back(consumer.value("id", "")); + } + + if (changes) { + changes->push_back( + {"disconnect", src_id, src_port, affected, + "断开 " + src_id + "." + src_port + " 与业务输出 " + tgt_port}); + } + return true; + } + + if (src_id == "$ingress") { + auto* tgt_node = FindNodeById(pipeline, tgt_id); + if (!tgt_node) { + if (error) *error = "NODE_NOT_FOUND: " + tgt_id; + return false; + } + auto in_key_opt = GetEffectiveInputKey(*tgt_node, tgt_port); + if (!in_key_opt.has_value() || *in_key_opt != src_port) { + if (error) *error = "连线不存在或已被修改"; + return false; + } + + auto tgt_def = + PipelineCatalog::FindNode(tgt_node->value("node_type", "")); + bool req = true; + if (tgt_def) { + for (const auto& in_p : tgt_def->inputs) { + if (in_p.logical_name == tgt_port) { + req = in_p.required; + break; + } + } + } + + if (!tgt_node->contains("ports") || !(*tgt_node)["ports"].is_object()) { + (*tgt_node)["ports"] = nlohmann::json::object(); + } + if (!(*tgt_node)["ports"].contains("inputs") || + !(*tgt_node)["ports"]["inputs"].is_object()) { + (*tgt_node)["ports"]["inputs"] = nlohmann::json::object(); + } + + if (req) { + auto occupied = GetOccupiedKeys(*pipeline); + std::string placeholder = + AllocateKey(tgt_id + "__unconnected__" + tgt_port, occupied); + (*tgt_node)["ports"]["inputs"][tgt_port] = placeholder; + } else { + (*tgt_node)["ports"]["inputs"].erase(tgt_port); + } + + if (changes) { + changes->push_back( + {"disconnect", + tgt_id, + tgt_port, + {tgt_id}, + "断开业务输入 " + src_port + " 到 " + tgt_id + "." + tgt_port}); + } + return true; + } + + // Normal node to Normal node + auto* src_node = FindNodeById(pipeline, src_id); + auto* tgt_node = FindNodeById(pipeline, tgt_id); + if (!src_node) { + if (error) *error = "NODE_NOT_FOUND: " + src_id; + return false; + } + if (!tgt_node) { + if (error) *error = "NODE_NOT_FOUND: " + tgt_id; + return false; + } + + std::string src_out_key = GetEffectiveOutputKey(*src_node, src_port); + auto in_key_opt = GetEffectiveInputKey(*tgt_node, tgt_port); + if (!in_key_opt.has_value() || *in_key_opt != src_out_key) { + if (error) *error = "连线不存在或已被修改"; + return false; + } + + auto tgt_def = PipelineCatalog::FindNode(tgt_node->value("node_type", "")); + bool req = true; + if (tgt_def) { + for (const auto& in_p : tgt_def->inputs) { + if (in_p.logical_name == tgt_port) { + req = in_p.required; + break; + } + } + } + + if (!tgt_node->contains("ports") || !(*tgt_node)["ports"].is_object()) { + (*tgt_node)["ports"] = nlohmann::json::object(); + } + if (!(*tgt_node)["ports"].contains("inputs") || + !(*tgt_node)["ports"]["inputs"].is_object()) { + (*tgt_node)["ports"]["inputs"] = nlohmann::json::object(); + } + + if (req) { + auto occupied = GetOccupiedKeys(*pipeline); + std::string placeholder = + AllocateKey(tgt_id + "__unconnected__" + tgt_port, occupied); + (*tgt_node)["ports"]["inputs"][tgt_port] = placeholder; + } else { + (*tgt_node)["ports"]["inputs"].erase(tgt_port); + } + // Execution dependency is preserved per RFC-0057 conservative rule! + + if (changes) { + changes->push_back({"disconnect", + tgt_id, + tgt_port, + {src_id, tgt_id}, + "断开 " + src_id + "." + src_port + " 到 " + tgt_id + + "." + tgt_port}); + } + return true; + } + + if (kind == "reset_input_binding") { + if (!operation.contains("target") || !operation["target"].is_object()) { + if (error) *error = "reset_input_binding 缺少 target 对象"; + return false; + } + std::string tgt_id = operation["target"].value("node_id", ""); + std::string tgt_port = operation["target"].value("port", ""); + if (tgt_id.empty() || tgt_port.empty()) { + if (error) *error = "reset_input_binding 端点 node_id 或 port 不能为空"; + return false; + } + + auto* tgt_node = FindNodeById(pipeline, tgt_id); + if (!tgt_node) { + if (error) *error = "NODE_NOT_FOUND: " + tgt_id; + return false; + } + auto tgt_def = PipelineCatalog::FindNode(tgt_node->value("node_type", "")); + if (!tgt_def) { + if (error) + *error = "UNKNOWN_NODE_TYPE: " + tgt_node->value("node_type", ""); + return false; + } + auto in_it = std::find_if(tgt_def->inputs.begin(), tgt_def->inputs.end(), + [&](const NodePortDefinition& p) { + return p.logical_name == tgt_port; + }); + if (in_it == tgt_def->inputs.end()) { + if (error) *error = "目标输入端口不存在: " + tgt_port; + return false; + } + + if (tgt_node->contains("ports") && + (*tgt_node)["ports"].contains("inputs") && + (*tgt_node)["ports"]["inputs"].is_object() && + (*tgt_node)["ports"]["inputs"].contains(tgt_port)) { + (*tgt_node)["ports"]["inputs"].erase(tgt_port); + } + + if (changes) { + changes->push_back({"reset_input_binding", + tgt_id, + tgt_port, + {tgt_id}, + "恢复 " + tgt_id + "." + tgt_port + " 为默认绑定"}); + } + return true; + } + + if (kind == "add_dependency") { + std::string node_id = operation.value("node_id", ""); + std::string dep_id = operation.value("depends_on_id", ""); + if (node_id.empty() || dep_id.empty()) { + if (error) *error = "add_dependency 缺少 node_id 或 depends_on_id"; + return false; + } + if (node_id == dep_id) { + if (error) *error = "添加依赖会形成环"; + return false; + } + auto* node = FindNodeById(pipeline, node_id); + auto* dep_node = FindNodeById(pipeline, dep_id); + if (!node) { + if (error) *error = "NODE_NOT_FOUND: " + node_id; + return false; + } + if (!dep_node) { + if (error) *error = "NODE_NOT_FOUND: " + dep_id; + return false; + } + + auto dep_graph = BuildDependencyGraph(*pipeline); + if (IsAncestor(node_id, dep_id, dep_graph)) { + if (error) *error = "添加依赖会形成环"; + return false; + } + + if (IsAncestor(dep_id, node_id, dep_graph)) { + // Already an ancestor; do not add redundant edge (RFC-0057 Section 4.5) + if (changes) { + changes->push_back({"add_dependency", + node_id, + "", + {node_id, dep_id}, + "添加 " + node_id + " 对 " + dep_id + + " 的执行依赖（已存在祖先关系，无变化）"}); + } + return true; + } + + if (!node->contains("depends_on") || !(*node)["depends_on"].is_array()) { + (*node)["depends_on"] = nlohmann::json::array(); + } + bool already = false; + for (const auto& d : (*node)["depends_on"]) { + if (d.is_string() && d.get() == dep_id) { + already = true; + break; + } + } + if (!already) { + (*node)["depends_on"].push_back(dep_id); + } + + if (changes) { + changes->push_back({"add_dependency", + node_id, + "", + {node_id, dep_id}, + "添加 " + node_id + " 对 " + dep_id + " 的执行依赖"}); + } + return true; + } + + if (kind == "remove_dependency") { + std::string node_id = operation.value("node_id", ""); + std::string dep_id = operation.value("depends_on_id", ""); + if (node_id.empty() || dep_id.empty()) { + if (error) *error = "remove_dependency 缺少 node_id 或 depends_on_id"; + return false; + } + auto* node = FindNodeById(pipeline, node_id); + if (!node) { + if (error) *error = "NODE_NOT_FOUND: " + node_id; + return false; + } + + if (node->contains("depends_on") && (*node)["depends_on"].is_array()) { + auto& deps = (*node)["depends_on"]; + for (auto it = deps.begin(); it != deps.end();) { + if (it->is_string() && it->get() == dep_id) { + it = deps.erase(it); + } else { + ++it; + } + } + } + + if (changes) { + changes->push_back({"remove_dependency", + node_id, + "", + {node_id, dep_id}, + "删除 " + node_id + " 对 " + dep_id + " 的执行依赖"}); + } + return true; + } + + if (error) *error = "UNKNOWN_OPERATION_KIND: " + kind; + return false; +} + +AuthoringResult PipelineAuthoring::ApplyRequest(const nlohmann::json& request) { + AuthoringResult result; + try { + if (!request.is_object()) { + result.ok = false; + result.diagnostics.push_back("请求必须是 JSON 对象"); + return result; + } + + // Check 4 MiB payload limit (RFC-0057 Section 4.4) + if (request.dump().size() > 4 * 1024 * 1024) { + result.ok = false; + result.diagnostics.push_back( + "REQUEST_TOO_LARGE: 请求输入大小超过单次上限 4 MiB"); + return result; + } + + CheckFields(request, {"schema_version", "pipeline", "operation", + "operations", "require_valid"}); + if (request.contains("require_valid") && + !request["require_valid"].is_boolean()) + throw std::invalid_argument("require_valid 必须是布尔值"); + if (!request.contains("schema_version") || + !request["schema_version"].is_number_integer() || + request["schema_version"] != 1) { + result.ok = false; + result.diagnostics.push_back("schema_version 必须为 1"); + return result; + } + if (!request.contains("pipeline") || !request["pipeline"].is_object()) { + result.ok = false; + result.diagnostics.push_back("缺少 pipeline 对象"); + return result; + } + const auto& orig_pipeline = request["pipeline"]; + if (!orig_pipeline.contains("pipeline") || + !orig_pipeline["pipeline"].is_array()) { + result.ok = false; + result.diagnostics.push_back("pipeline 必须包含 pipeline 节点数组"); + return result; + } + + bool has_op = request.contains("operation"); + bool has_ops = request.contains("operations"); + if ((has_op && has_ops) || (!has_op && !has_ops)) { + result.ok = false; + result.diagnostics.push_back( + "请求必须包含互斥的 operation 或 operations"); + return result; + } + + if ((has_op && !request["operation"].is_object()) || + (has_ops && !request["operations"].is_array())) + throw std::invalid_argument( + "operation 必须是对象，operations 必须是数组"); + std::vector operations; + if (has_op) { + operations.push_back(request["operation"]); + } else { + operations = request["operations"].get>(); + } + + if (operations.empty()) { + result.ok = false; + result.diagnostics.push_back("operations 数组不能为空"); + return result; + } + if (operations.size() > 128) { + result.ok = false; + result.diagnostics.push_back("operations 超过单次上限 128"); + return result; + } + + nlohmann::json working_pipeline = orig_pipeline; + for (size_t i = 0; i < operations.size(); ++i) { + result.failed_operation_index = i; + std::string err; + if (!ApplyOperation(&working_pipeline, operations[i], &result.changes, + &err)) { + result.ok = false; + result.failed_operation_index = i; + result.diagnostics.push_back(err); + result.pipeline = std::nullopt; + return result; + } + } + + result.failed_operation_index.reset(); + auto report = PipelineValidator::Explain(working_pipeline); + result.validation = report.ToJson(); + + bool require_valid = request.value("require_valid", false); + if (require_valid && !report.ok) { + result.ok = false; + result.pipeline = std::nullopt; + return result; + } + + result.ok = true; + result.pipeline = std::move(working_pipeline); + return result; + } catch (const std::exception& error) { + result.ok = false; + result.pipeline.reset(); + result.changes.clear(); + result.diagnostics.push_back(error.what()); + return result; + } +} + +FixDepsResult PipelineAuthoring::FixDeps(const std::string& file_path, + bool in_place) { + FixDepsResult result; + result.target_file = file_path; + try { + fs::path path(file_path); + + if (fs::is_symlink(path)) { + result.ok = false; + result.written = false; + result.diagnostics.push_back("SYMLINK_REJECTED: 拒绝修复符号链接方案"); + return result; + } + if (!fs::exists(path) || !fs::is_regular_file(path)) { + result.ok = false; + result.written = false; + result.diagnostics.push_back("FILE_NOT_FOUND: 文件不存在或不是普通文件"); + return result; + } + + struct stat old_stat {}; + const std::string old_bytes = ReadRegularFile(file_path, &old_stat); + + nlohmann::json root; + try { + root = nlohmann::json::parse(old_bytes); + } catch (const std::exception& e) { + result.ok = false; + result.written = false; + result.diagnostics.push_back(std::string("JSON_READ: ") + e.what()); + return result; + } + + auto report = PipelineValidator::Explain(root); + if (report.ok) { + result.ok = true; + result.written = false; + result.validation = report.ToJson(); + return result; + } + + nlohmann::json working = root; + auto dep_graph = BuildDependencyGraph(working); + bool found_fix = false; + + // Build map of producers for each key in working pipeline + // to detect ambiguity (RFC-0057 Section 4.5 & G9) + std::unordered_map> key_producers; + std::string biz_name = working.value("biz_name", ""); + auto biz = PipelineCatalog::FindBiz(biz_name); + if (biz) { + for (const auto& in_p : biz->ingress) { + key_producers[in_p.blackboard_key].push_back("$ingress"); + } + } + if (working.contains("pipeline") && working["pipeline"].is_array()) { + for (const auto& node : working["pipeline"]) { + if (!node.is_object()) continue; + std::string n_id = node.value("id", ""); + std::string n_type = node.value("node_type", ""); + auto n_def = PipelineCatalog::FindNode(n_type); + if (n_def) { + for (const auto& out : n_def->outputs) { + std::string actual_key = + GetEffectiveOutputKey(node, out.logical_name); + key_producers[actual_key].push_back(n_id); + } + } + } + } + + for (const auto& diag : report.diagnostics) { + if (diag.remediation.has_value() && + diag.remediation->cause == "producer_not_dependency_ancestor") { + std::string producer_id = + diag.remediation->facts.value("producer_id", ""); + std::string bound_key = diag.remediation->facts.value("bound_key", ""); + std::string consumer_id = diag.node_id; + + // Check ambiguity: is producer unique? + auto it_p = key_producers.find(bound_key); + if (it_p != key_producers.end() && it_p->second.size() > 1) { + result.ok = false; + result.written = false; + result.validation = report.ToJson(); + result.diagnostics.push_back( + "AMBIGUOUS_PRODUCER: 数据键 '" + bound_key + + "' 存在多个生产者，来源存在歧义，不能自动修复"); + return result; + } + + if (!producer_id.empty() && !consumer_id.empty()) { + auto* c_node = FindNodeById(&working, consumer_id); + if (c_node) { + // Check cycle + if (IsAncestor(consumer_id, producer_id, dep_graph)) { + result.ok = false; + result.written = false; + result.validation = report.ToJson(); + result.diagnostics.push_back("CYCLE_DETECTED: 添加 " + + consumer_id + " 对 " + producer_id + + " 的依赖会形成环"); + return result; + } + if (!IsAncestor(producer_id, consumer_id, dep_graph)) { + if (!c_node->contains("depends_on") || + !(*c_node)["depends_on"].is_array()) { + (*c_node)["depends_on"] = nlohmann::json::array(); + } + (*c_node)["depends_on"].push_back(producer_id); + dep_graph[consumer_id].push_back(producer_id); + found_fix = true; + result.changes.push_back({"add_dependency", + consumer_id, + "", + {consumer_id, producer_id}, + "添加 " + consumer_id + " 对 " + + producer_id + " 的执行依赖"}); + } + } + } + } + } + + if (!found_fix) { + result.ok = false; + result.written = false; + result.validation = report.ToJson(); + result.diagnostics.push_back( + "NO_FIXABLE_DEPENDENCY: 未发现可安全自动修复的生产者依赖"); + return result; + } + + auto final_report = PipelineValidator::Explain(working); + result.validation = final_report.ToJson(); + if (!final_report.ok) { + result.ok = false; + result.written = false; + for (const auto& d : final_report.diagnostics) { + result.diagnostics.push_back(std::string(DiagnosticCodeName(d.code)) + + " " + d.path + ": " + d.message); + } + return result; + } + + result.ok = true; + if (in_place) { + ReplaceFile(path, working.dump(2) + "\n", old_bytes, old_stat); + result.written = true; + } else { + result.written = false; + } + return result; + } catch (const std::exception& error) { + result.ok = false; + result.written = false; + result.diagnostics.push_back(error.what()); + return result; + } +} + +} // namespace llm_edgeflow diff --git a/src/tools/pipeline_authoring.h b/src/tools/pipeline_authoring.h new file mode 100644 index 00000000..01251023 --- /dev/null +++ b/src/tools/pipeline_authoring.h @@ -0,0 +1,84 @@ +#pragma once + +#include +#include +#include +#include +#include +#include + +namespace llm_edgeflow { + +struct AuthoringChange { + std::string action; + std::string node_id; + std::string port; + std::vector affected_nodes; + std::string description; + + nlohmann::json ToJson() const; +}; + +struct AuthoringResult { + int schema_version = 1; + bool ok = false; + std::optional pipeline; + std::vector changes; + nlohmann::json validation = nlohmann::json::object(); + std::vector diagnostics; + std::optional failed_operation_index; + + nlohmann::json ToJson() const; +}; + +struct FixDepsResult { + int schema_version = 1; + bool ok = false; + std::string target_file; + bool written = false; + std::vector changes; + nlohmann::json validation = nlohmann::json::object(); + std::vector diagnostics; + + nlohmann::json ToJson() const; +}; + +class PipelineAuthoring { + public: + // Applies a single authoring request containing {schema_version, pipeline, + // operation/operations, require_valid}. + static AuthoringResult ApplyRequest(const nlohmann::json& request); + + // Applies one authoring operation to an in-memory pipeline document. + static bool ApplyOperation(nlohmann::json* pipeline, + const nlohmann::json& operation, + std::vector* changes, + std::string* error); + + // Analyzes missing producer dependencies, previews or applies fixes in-place. + static FixDepsResult FixDeps(const std::string& file_path, bool in_place); + + // Collects all occupied blackboard keys in document (explicit, default, + // placeholders, ingress, egress). + static std::unordered_set GetOccupiedKeys( + const nlohmann::json& pipeline); + + // Allocates a unique key based on base name without colliding with occupied + // keys. + static std::string AllocateKey( + const std::string& base, const std::unordered_set& occupied); + + // Checks whether potential_ancestor is a direct or transitive dependency of + // node_id. + static bool IsAncestor( + const std::string& potential_ancestor, const std::string& node_id, + const std::unordered_map>& + dep_graph); + + // Builds a depends_on adjacency graph mapping node_id to its listed + // dependencies. + static std::unordered_map> + BuildDependencyGraph(const nlohmann::json& pipeline); +}; + +} // namespace llm_edgeflow diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index e5e537e9..08d9e924 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -26,6 +26,7 @@ endif() # production CLI or shared SDK. add_executable(alg_pipeline_tool_test ${PROJECT_SOURCE_DIR}/src/tools/alg_pipeline_tool.cpp + ${PROJECT_SOURCE_DIR}/src/tools/pipeline_authoring.cpp $ $) target_link_libraries(alg_pipeline_tool_test PRIVATE llm_edgeflow::internal_runtime) diff --git a/tests/tooling/studio_browser_test.mjs b/tests/tooling/studio_browser_test.mjs index 1ae86aba..2f447ec9 100644 --- a/tests/tooling/studio_browser_test.mjs +++ b/tests/tooling/studio_browser_test.mjs @@ -16,6 +16,10 @@ if (screenshotRoot) mkdirSync(screenshotRoot, { recursive: true }); const page = await browser.newPage({ viewport: { width: 1366, height: 768 } }); page.setDefaultTimeout(10000); const errors = []; +const authoringRequests = []; +page.on("request", request => { + if (request.url().endsWith("/api/v1/authoring/preview")) authoringRequests.push(request.postDataJSON()); +}); page.on("pageerror", error => errors.push(error.message)); const screenshot = async name => { if (screenshotRoot) await page.screenshot({ path: join(screenshotRoot, `${name}.png`) }); }; const open = async filename => { @@ -76,14 +80,38 @@ try { assert.match(await page.locator("#runSummary").textContent(), /成功 2 条/); assert.equal(await page.locator(".run-sample").count(), 2); const originalResult = await page.locator("#runResult").textContent(); + await page.click("#preflightButton"); + await page.waitForFunction(() => !document.querySelector("#preflightSummary").classList.contains("loading")); + assert.match(await page.locator("#preflightSummary").textContent(), /alg_pipeline_tool=✓.*alg_demo=✓/, + "Real server field names must render both tools as present"); + await page.locator(".run-settings > summary").click(); + const modelRootBefore = await page.locator("#runModelRoot").inputValue(); + await page.locator("#runModelRoot").fill("changed-model-root"); + assert.match(await page.locator("#preflightSummary").textContent(), /过期|重新预检/); + await page.locator("#runModelRoot").fill(modelRootBefore); + let releasePreflight, sawPreflight; + const heldPreflight = new Promise(resolve => { releasePreflight = resolve; }); + const preflightStarted = new Promise(resolve => { sawPreflight = resolve; }); + await page.route("**/api/v1/preflight", async route => { + sawPreflight(); await heldPreflight; + await route.fulfill({ json: { ok: true, summary: { status: "ready", next_step: "STALE_PREFLIGHT_SENTINEL" } } }); + }); + await page.click("#preflightButton"); await preflightStarted; + await page.locator("#runModelRoot").fill("changed-during-preflight"); + releasePreflight(); + await page.waitForFunction(() => !document.querySelector("#preflightSummary").classList.contains("loading")); + assert.doesNotMatch(await page.locator("#preflightSummary").textContent(), /STALE_PREFLIGHT_SENTINEL/); + await page.unroute("**/api/v1/preflight"); + await page.locator("#runModelRoot").fill(modelRootBefore); + await screenshot("run-results"); await page.click("#editModeButton"); await rule().click(); await categories().fill("{invalid"); - await page.click("#quickValidateButton"); await page.click("#applyContinue"); + await page.click("#quickValidateButton"); assert.equal(await categories().inputValue(), "{invalid", "invalid input survives continuation"); - assert.equal(await page.locator("#draftActions").isVisible(), true); + assert.equal(await page.locator("#draftActions").isVisible(), false, "one-click validation must not require a second continuation"); assert.match(await page.locator("#operationFeedback").textContent(), /categories/); - await categories().fill('{"BROWSER":["VIP"]}'); await page.click("#applyContinue"); + await categories().fill('{"BROWSER":["VIP"]}'); await page.click("#quickValidateButton"); await page.waitForFunction(() => document.querySelector("#validationOutput").textContent.includes("校验通过")); await page.click("#openRunButton"); assert.equal(await page.locator("#runFreshness").isVisible(), true); @@ -92,7 +120,7 @@ try { assert.equal(await page.locator("#runFreshness").isVisible(), false); await page.click("#redoButton"); assert.equal(await page.locator("#runFreshness").isVisible(), true); await rule().click(); await categories().fill('{"SAVED_BROWSER":["VIP"]}'); - await page.click("#saveButton"); await page.click("#applyContinue"); + await page.click("#saveButton"); await page.waitForFunction(() => document.querySelector("#operationFeedback").textContent.includes("已保存")); assert.deepEqual(JSON.parse(readFileSync(join(configRoot, "pipeline_browser.json"))).pipeline[0].config.categories, { SAVED_BROWSER: ["VIP"] }); assert.match(await page.locator("#saveScope").textContent(), /pipeline_browser.json/); @@ -111,12 +139,12 @@ try { await open("pipeline_browser_other.json"); await open("pipeline_browser_pair.json"); assert.match(await page.locator("#saveScope").textContent(), /pipeline_browser_pair.conf/); await rule().click(); await categories().fill('{"PAIR_UPDATE":["VIP"]}'); - await page.click("#saveButton"); await page.click("#applyContinue"); + await page.click("#saveButton"); await page.waitForFunction(() => document.querySelector("#operationFeedback").textContent.includes("已保存")); assert.deepEqual(JSON.parse(readFileSync(join(configRoot, "pipeline_browser_pair.json"))).pipeline[0].config.categories, { PAIR_UPDATE: ["VIP"] }); const savedPair = readFileSync(join(configRoot, "pipeline_browser_pair.json"), "utf8"); writeFileSync(confPath, readFileSync(confPath, "utf8") + "\n"); - await categories().fill('{"CONFLICT":["VIP"]}'); await page.click("#saveButton"); await page.click("#applyContinue"); + await categories().fill('{"CONFLICT":["VIP"]}'); await page.click("#saveButton"); await page.waitForFunction(() => document.querySelector("#operationFeedback").textContent.includes("其他编辑器修改")); assert.equal(readFileSync(join(configRoot, "pipeline_browser_pair.json"), "utf8"), savedPair); page.once("dialog", dialog => dialog.accept()); @@ -126,29 +154,33 @@ try { await page.locator('#nodeId').fill('browser_renamed'); await page.locator(`.node[data-node-id="${ids[1]}"]`).click(); await page.click('#applyContinue'); + await page.waitForFunction(id => document.querySelector('#nodeId').value === id, ids[1]); assert.equal(await page.locator('#nodeId').inputValue(), ids[1]); assert.ok((await json()).pipeline.some(n => n.id === 'browser_renamed')); + assert.ok(authoringRequests.some(request => request.operation?.kind === 'rename_node' || + request.operations?.some(operation => operation.kind === 'rename_node')), + 'Actual node form renames must pass through the shared authoring endpoint'); await page.locator('#nodeId').fill('discarded_name'); await page.locator('.node[data-node-id="browser_renamed"]').click(); await page.click('#discardContinue'); assert.equal(await page.locator('#nodeId').inputValue(), 'browser_renamed'); assert.ok(!(await json()).pipeline.some(n => n.id === 'discarded_name')); await page.click('#undoButton'); assert.ok(!(await json()).pipeline.some(n => n.id === 'browser_renamed')); - // Model and raw JSON buffers use the same explicit continuation path. + // Validation applies model and raw JSON buffers in the same action. const originalMulti = await json(); await page.click('[data-tab="models"]'); await page.selectOption('#modelSelect', originalMulti.models[0].model_id); await page.locator('#modelId').fill('browser_model'); - await page.click('#quickValidateButton'); await page.click('#applyContinue'); + await page.click('#quickValidateButton'); await page.waitForFunction(() => document.querySelector('#validationOutput').textContent.includes('校验通过')); assert.ok((await json()).models.some(model => model.model_id === 'browser_model')); await page.click('#undoButton'); assert.deepEqual(await json(), originalMulti); await page.click('[data-tab="json"]'); await page.locator('#rawJson').fill('{ broken'); - await page.click('#quickValidateButton'); await page.click('#applyContinue'); + await page.click('#quickValidateButton'); assert.equal(await page.locator('#rawJson').inputValue(), '{ broken'); await page.locator('#rawJson').fill(JSON.stringify({ ...originalMulti, comment: 'browser JSON' }, null, 2)); - await page.click('#applyContinue'); + await page.click('#quickValidateButton'); await page.waitForFunction(() => document.querySelector('#validationOutput').textContent.includes('校验通过')); assert.equal((await json()).comment, 'browser JSON'); await page.click('#undoButton'); assert.deepEqual(await json(), originalMulti); @@ -176,6 +208,74 @@ try { await page.click('#runButton'); await page.waitForFunction(() => document.querySelector('#runSummary').textContent.includes('运行已完成')); assert.equal(await page.locator('.run-sample').count(), 2); + // A real port action must use native authoring and remain a single undo step. + await rule().click(); + const beforeReset = await json(); + await page.locator("#nodeForm details > summary").click(); + await page.locator("#nodeBindings button").filter({ hasText: "恢复默认绑定" }).first().click(); + await page.waitForFunction(() => !JSON.parse(document.querySelector("#rawJson").value).pipeline[0].ports?.inputs?.text); + assert.ok(authoringRequests.some(request => request.operation?.kind === "reset_input_binding")); + await page.click("#undoButton"); + assert.deepEqual(await json(), beforeReset); + + // Dependency controls are graph actions, not node-property draft buffers. + // Add an independent node so the new ordering is observable and acyclic. + const beforeAdd = await json(); + if (!(await page.locator("#operatorSearch").isVisible())) await page.click("#operatorsToggle"); + await page.locator("#operatorSearch").fill("TextTemplateNode"); + await page.locator("#operatorList button").filter({ hasText: "TextTemplateNode" }).click(); + await page.waitForFunction(count => JSON.parse(document.querySelector("#rawJson").value).pipeline.length === count + 1, + beforeAdd.pipeline.length); + const afterAdd = await json(); + const addedNode = afterAdd.pipeline.find(node => !beforeAdd.pipeline.some(previous => previous.id === node.id)); + assert.ok(authoringRequests.some(request => request.operation?.kind === "add_node")); + await page.locator(`.node[data-node-id="${addedNode.id}"]`).click(); + if (!(await page.locator("#dependencySource").isVisible())) await page.locator("#nodeForm details > summary").click(); + await page.selectOption("#dependencySource", beforeAdd.pipeline[0].id); + assert.equal(await page.locator("#nodeDraftHint").isVisible(), false, + "Selecting an execution dependency must not create a pending parameter draft"); + await page.click("#addDependencyButton"); + await page.waitForFunction(({ id, dependency }) => JSON.parse(document.querySelector("#rawJson").value) + .pipeline.find(node => node.id === id).depends_on.includes(dependency), + { id: addedNode.id, dependency: beforeAdd.pipeline[0].id }); + assert.ok(authoringRequests.some(request => request.operation?.kind === "add_dependency" && + request.operation.node_id === addedNode.id)); + await page.click("#undoButton"); + assert.deepEqual(await json(), afterAdd, "One undo removes only the added ordering"); + await page.click("#undoButton"); + assert.deepEqual(await json(), beforeAdd, "The preceding undo removes the added node"); + await page.locator("#operatorSearch").fill(""); + + // Native missing-dependency remedies are reviewed on the page, never in a dialog. + await open("pipeline_browser_multi.json"); + const repairOriginal = await json(); + const missingDependencies = structuredClone(repairOriginal); + for (const node of missingDependencies.pipeline) node.depends_on = []; + await page.click('[data-tab="json"]'); + await page.locator("#rawJson").fill(JSON.stringify(missingDependencies)); + await page.click("#quickValidateButton"); + await page.waitForSelector("#validationOutput .fix-apply-btn"); + let repairDialogs = 0; + const rejectRepairDialog = async dialog => { repairDialogs++; await dialog.dismiss(); }; + page.on("dialog", rejectRepairDialog); + await page.locator("#validationOutput .fix-apply-btn").first().click(); + await page.waitForSelector("#applyReviewedFix"); + assert.deepEqual(await json(), missingDependencies, "Review must leave draft unchanged"); + assert.equal(await page.locator("#fixReviewPanel details").getAttribute("open"), null); + await page.click("#cancelReviewedFix"); + assert.equal(await page.locator("#fixReviewPanel").isVisible(), false); + await page.locator("#validationOutput .fix-apply-btn").first().click(); + await page.waitForSelector("#applyReviewedFix"); + await page.click("#applyReviewedFix"); + await page.waitForFunction(() => document.querySelector("#fixReviewPanel").hidden); + assert.notDeepEqual(await json(), missingDependencies); + assert.equal(repairDialogs, 0); + page.off("dialog", rejectRepairDialog); + await page.click("#undoButton"); + assert.deepEqual(await json(), missingDependencies, "One undo restores the entire reviewed draft"); + await page.click("#undoButton"); + assert.deepEqual(await json(), repairOriginal); + await open("pipeline_browser_other.json"); // Native Validator reports use an object-valued error.message. await page.click('[data-tab="json"]'); const invalidPipeline = await json(); invalidPipeline.pipeline[0].node_type = 'MissingNode'; diff --git a/tests/tooling/studio_fix_workflow_test.mjs b/tests/tooling/studio_fix_workflow_test.mjs index 75c81330..df724a03 100644 --- a/tests/tooling/studio_fix_workflow_test.mjs +++ b/tests/tooling/studio_fix_workflow_test.mjs @@ -31,6 +31,8 @@ const document = { }; let previewResponse, previewCalls = 0, confirmations = 0, validationCalls = 0; let catalogVersion = 3; +let authoringResponse, authoringCalls = [], preflightResponse; +const requests = []; const context = vm.createContext({ document, structuredClone, console, URL, URLSearchParams, setTimeout() {}, clearTimeout() {}, requestAnimationFrame() {}, @@ -45,9 +47,12 @@ const context = vm.createContext({ if (path === "/assets") return { selections: [], variants: [] }; throw new Error(`Unexpected API request: ${path}`); }, - async testWrite(path) { + async testWrite(path, method, body) { + requests.push({ path, body: structuredClone(body) }); + if (path === "/authoring/preview") { authoringCalls.push(structuredClone(body)); return await authoringResponse; } + if (path === "/preflight") return await preflightResponse; if (path === "/fixes/preview") { previewCalls++; return await previewResponse; } - if (path === "/validate") { validationCalls++; return { ok: true, diagnostics: [], plan: { topological_order: [], layers: [] } }; } + if (path === "/validate") { validationCalls++; return { ok: true, revision: "revision-1", tool_fingerprint: "tool-1", diagnostics: [], plan: { topological_order: [], layers: [] } }; } throw new Error(`Unexpected API request: ${path}`); }, }); @@ -62,7 +67,7 @@ async function load(name) { const startup = code.indexOf("\ntry {\n await refreshLists();"); assert.notEqual(startup, -1, "Locate and omit browser startup, without extracting the handler"); code = code.slice(0, startup); - code += "\nrenderAll = () => {}; updateEditorStatus = () => {};\nexport { state, history, drafts, handleApplyFix, restoreHistory, refreshLists, loadCatalog, validate };"; + code += "\nrenderAll = () => {}; updateEditorStatus = () => {};\nexport { state, history, drafts, handleApplyFix, handleApplyReviewedFix, applyAuthoring, runPreflight, renderPreflightFreshness, markPipelineChanged, restoreHistory, refreshLists, loadCatalog, validate };"; } const module = new vm.SourceTextModule(code, { context, identifier: name }); modules.set(name, module); @@ -79,21 +84,26 @@ const fix = { id: "repair-1", title: "Repair", effect: "Replace comment", verifi function reset() { Object.assign(state, { pipeline: structuredClone(initial), selected: "", loading: false, editing: true, catalogReady: true, pipelineVersion: 1, documentVersion: 1, - savedPipeline: JSON.stringify(initial), dirty: false, + savedPipeline: JSON.stringify(initial), dirty: false, deployment: null, modelPathActions: {}, preflight: null, validationReport: { revision: "revision-1", tool_fingerprint: "tool-1", diagnostics: [ { code: "TEST", remediation: { schema_version: 1, fixes: [fix] } }, ] }, }); drafts.clear(); history.reset({ pipeline: state.pipeline, selected: "" }); - previewCalls = 0; confirmations = 0; + previewCalls = 0; confirmations = 0; authoringCalls = []; requests.length = 0; previewResponse = Promise.resolve({ ok: true, patched: structuredClone(patched), report: { ok: true } }); } const plain = value => JSON.parse(JSON.stringify(value)); reset(); await handleApplyFix(fix); +assert.deepEqual(plain(state.pipeline), initial, "Preview never changes the draft before page approval"); +assert.equal(history.canUndo, false); +assert.equal(confirmations, 0, "Repair review must not invoke a blocking browser dialog"); +assert.equal(elements.get("#fixReviewPanel").hidden, false); +await app.namespace.handleApplyReviewedFix(); assert.deepEqual(plain(state.pipeline), patched, "Apply replaces the current draft"); assert.equal(state.dirty, true, "Applying a repair marks the document unsaved"); -assert.equal(previewCalls, 1); +assert.equal(previewCalls, 2, "Application revalidates the reviewed patch and tool fingerprint"); await restoreHistory("undo"); assert.deepEqual(plain(state.pipeline), initial, "Undo restores a complete application snapshot"); await restoreHistory("redo"); @@ -122,6 +132,91 @@ drafts.set("json", "{ unfinished"); await handleApplyFix(fix); assert.equal(previewCalls, 0, "Do not preview a repair against unapplied form buffers"); + +// The application must route graph changes through the native endpoint and treat +// its complete candidate as one undoable operation, including retained ordering. +reset(); +const graphInitial = { ...initial, pipeline: [{ id: "a" }, { id: "b", depends_on: ["a"], inputs: { text: "shared" } }] }; +state.pipeline = structuredClone(graphInitial); +history.reset({ pipeline: state.pipeline, selected: "" }); +const graphCandidate = structuredClone(graphInitial); +graphCandidate.pipeline[1].inputs.text = "unconnected_1"; +authoringResponse = { ok: true, pipeline: graphCandidate, validation: { ok: false, diagnostics: [] } }; +const disconnect = { kind: "disconnect", source: { node_id: "a", port: "text" }, target: { node_id: "b", port: "text" } }; +await app.namespace.applyAuthoring(disconnect); +assert.equal(authoringCalls.length, 1); +assert.deepEqual(authoringCalls[0].operation, disconnect); +assert.deepEqual(authoringCalls[0].pipeline, graphInitial); +assert.equal(authoringCalls[0].revision, "revision-1"); +assert.equal(authoringCalls[0].tool_fingerprint, "tool-1"); +assert.deepEqual(plain(state.pipeline), graphCandidate, "Use the complete native candidate without a second graph transform"); +await restoreHistory("undo"); +assert.deepEqual(plain(state.pipeline), graphInitial); +assert.equal(history.canUndo, false, "One action records exactly one history transaction"); + +for (const change of [ + () => { state.pipeline.comment = "newer"; state.pipelineVersion++; }, + () => { state.documentVersion++; }, + () => { drafts.set("json", "unfinished"); }, +]) { + reset(); + let finish; + authoringResponse = new Promise(resolve => { finish = resolve; }); + const action = app.namespace.applyAuthoring(disconnect); + await new Promise(resolve => setImmediate(resolve)); + assert.equal(authoringCalls.length, 1); + change(); + const expected = structuredClone(state.pipeline); + finish({ ok: true, pipeline: patched, validation: { ok: true, diagnostics: [] } }); + await action; + assert.deepEqual(plain(state.pipeline), expected, "Late graph operations cannot replace newer work"); + assert.equal(history.canUndo, false); +} + +// Page review can last arbitrarily long: reject draft edits and changed tools +// before committing the reviewed candidate. +reset(); +await handleApplyFix(fix); +state.pipeline.comment = "edited while reviewing"; +state.pipelineVersion++; +await app.namespace.handleApplyReviewedFix(); +assert.equal(state.pipeline.comment, "edited while reviewing"); +assert.equal(history.canUndo, false); +reset(); +await handleApplyFix(fix); +previewResponse = Promise.reject(new Error("TOOL_FINGERPRINT_MISMATCH")); +await app.namespace.handleApplyReviewedFix(); +assert.deepEqual(plain(state.pipeline), initial); +assert.equal(history.canUndo, false); +assert.equal(confirmations, 0); + +const ready = { ok: true, summary: { status: "ready", tools: { alg_pipeline_tool: true, alg_demo: true }, assets: [] } }; +const elementText = node => [node.textContent || "", ...node.children.map(elementText)].join("\n"); +reset(); +preflightResponse = ready; +await app.namespace.runPreflight(); +assert.match(elementText(elements.get("#preflightSummary")), /alg_pipeline_tool=✓.*alg_demo=✓/); +elements.get("#runModelRoot").value = "other-models"; +app.namespace.renderPreflightFreshness(); +assert.match(elementText(elements.get("#preflightSummary")), /过期|重新预检/); +for (const change of [ + () => { state.pipelineVersion++; state.pipeline.comment = "newer"; }, + () => { state.documentVersion++; }, + () => { elements.get("#runProfile").value = "different-profile"; }, + () => { state.deployment = { conf_name: "different.conf", conf_revision: "new" }; }, +]) { + reset(); + let finish; + preflightResponse = new Promise(resolve => { finish = resolve; }); + const action = app.namespace.runPreflight(); + change(); + finish({ ...ready, summary: { ...ready.summary, next_step: "STALE_RESPONSE_SENTINEL" } }); + await action; + assert.doesNotMatch(elementText(elements.get("#preflightSummary")), /STALE_RESPONSE_SENTINEL/, + "Delayed preflight must not display a result for another draft or deployment"); +} +console.log("Studio native authoring, page approval and preflight snapshot regressions passed"); + // Render the actual diagnostic card for unsupported wire-format versions. const { appendDiagnostic } = modules.get("editor.js").namespace; for (const schema_version of [0, 2, 999]) { diff --git a/tests/tooling/test_pipeline_studio.py b/tests/tooling/test_pipeline_studio.py index 27c6e6be..7d7d100c 100644 --- a/tests/tooling/test_pipeline_studio.py +++ b/tests/tooling/test_pipeline_studio.py @@ -1,9 +1,11 @@ #!/usr/bin/env python3 """API and filesystem boundary tests for the local Pipeline Studio.""" +import copy import importlib.util import json import os +import sys from pathlib import Path import shutil import shlex @@ -377,7 +379,11 @@ def test_ordinary_save_updates_managed_model_paths_and_node_parameters(self): pipeline["models"][0]["model_id"] = "replacement_model" pipeline["pipeline"][1]["config"]["bind_model"] = "replacement_model" pipeline["pipeline"][1]["config"]["max_tokens"] = 17 - updated = self.service.save_pipeline(saved["filename"], pipeline, saved["revision"]) + updated = self.service.save_pipeline( + saved["filename"], pipeline, saved["revision"], + model_path_actions={"replacement_model": { + "path": "replacement.gguf", "action": "select_asset"}}, + ) self.assertEqual(updated["command"], saved["command"]) self.assertEqual(updated["conf"]["data"]["model_paths"], {"replacement_model": "models/replacement.gguf"}) self.assertEqual(json.loads((self.configs / saved["filename"]).read_text()), pipeline) @@ -739,6 +745,34 @@ def post(self): ) return urllib.request.urlopen(request, timeout=5) + def test_authoring_http_rejects_unknown_fields_and_conflicting_revisions(self): + pipeline = json.loads((ROOT / "configs" / "pipeline_keyword_match_rules.json").read_text()) + body = { + "pipeline": pipeline, + "operation": {"kind": "add_node", "node_type": "TextTemplateNode"}, + "revision": SHOW.revision_for(json.dumps(pipeline, sort_keys=True).encode()), + "tool_fingerprint": self.service.get_tool_fingerprint(), + } + for fields, status, code in [ + ({"requrie_valid": True}, 400, "INVALID_AUTHORING_REQUEST"), + ({"expected_revision": "different_revision"}, 409, "REVISION_CONFLICT"), + ]: + with self.subTest(fields=fields): + request = urllib.request.Request( + self.base + "/authoring/preview", data=json.dumps(dict(body, **fields)).encode(), + method="POST", headers={"Content-Type": "application/json"}, + ) + with mock.patch.object(self.service, "preview_authoring") as authoring: + with self.assertRaises(urllib.error.HTTPError) as ctx: + urllib.request.urlopen(request, timeout=5) + self.assertEqual(ctx.exception.code, status) + with ctx.exception as response: + result = json.load(response) + self.assertFalse(result["ok"]) + self.assertEqual(result["error"]["code"], code) + self.assertNotIn("pipeline", result) + authoring.assert_not_called() + def test_local_development_api_and_static_modules(self): origin = self.base.removesuffix("/api/v1") with urllib.request.urlopen(origin + "/index.html", timeout=5) as response: @@ -973,7 +1007,7 @@ def test_actual_fix_handler_history_and_stale_response_protection(self): self.assertEqual(process.returncode, 0, process.stdout + process.stderr) @unittest.skipUnless(shutil.which("node"), "Node.js is required for Web module tests") - def test_port_editing_roundtrips_to_native_validator_and_model_forms(self): + def test_readonly_graph_matches_native_validator_and_model_forms(self): catalog = json.loads(subprocess.check_output([str(PIPELINE_TOOL), "catalog"], text=True)) script = """ import assert from 'node:assert/strict'; @@ -982,21 +1016,15 @@ def test_port_editing_roundtrips_to_native_validator_and_model_forms(self): const w = await import(`data:text/javascript;base64,${Buffer.from(code).toString('base64')}`); const catalog = JSON.parse(readFileSync(0, 'utf8')); const pipeline = {biz_name:'keyword_match_v1', models:[], pipeline:[ - {id:'template', node_type:'TextTemplateNode', depends_on:[], config:{template:'{{primary}}'}, ports:{outputs:{text:'template_text'}}}, - {id:'rule', node_type:'TextRuleMatchNode', depends_on:[], config:{}, ports:{outputs:{matches:'rule_result'}}} + {id:'template', node_type:'TextTemplateNode', depends_on:[], config:{template:'{{primary}}'}, ports:{inputs:{primary:'input_sentences'},outputs:{text:'template_text'}}}, + {id:'rule', node_type:'TextRuleMatchNode', depends_on:['template'], config:{}, ports:{inputs:{text:'template_text'},outputs:{matches:'rule_matches'}}} ]}; -w.connectPorts(pipeline,catalog,w.INGRESS,'input_sentences','template','primary'); -w.connectPorts(pipeline,catalog,'template','text','rule','text'); -w.connectPorts(pipeline,catalog,'rule','matches',w.EGRESS,'rule_matches'); -assert.equal(pipeline.pipeline[1].ports.inputs.text,'template_text'); -assert.deepEqual(pipeline.pipeline[1].depends_on,['template']); -assert.throws(()=>w.connectPorts(pipeline,catalog,'rule','matches','template','primary'), /类型/); -assert.throws(()=>w.connectPorts(pipeline,catalog,'template','text','template','primary'), /环/); -const detached = structuredClone(pipeline); -w.disconnectPorts(detached,catalog,w.graphDocument(detached,catalog).edges.find(e=>e.target==='rule' && e.targetPort==='text')); -assert.ok(!w.graphDocument(detached,catalog).edges.some(e=>e.source==='template' && e.target==='rule')); -w.removeNode(detached,catalog,'template'); -assert.deepEqual(detached.pipeline[0].depends_on,[]); +const snapshot = structuredClone(pipeline); +const graph = w.graphDocument(pipeline,catalog); +assert.ok(graph.edges.some(e=>e.source===w.INGRESS && e.target==='template' && e.targetPort==='primary')); +assert.ok(graph.edges.some(e=>e.source==='template' && e.sourcePort==='text' && e.target==='rule' && e.targetPort==='text')); +assert.ok(graph.edges.some(e=>e.source==='rule' && e.target===w.EGRESS && e.targetPort==='rule_matches')); +assert.deepEqual(pipeline,snapshot,'graph rendering must not mutate the document'); const modelDef = catalog.models.find(m=>m.model_type==='bge_embedding'); const backend = w.compatibleBackends(catalog.backends,modelDef).find(b=>b.backend_type==='onnxruntime'); if (backend) { @@ -1140,5 +1168,1104 @@ def test_native_build_variant_and_real_effects_are_bound_to_selection(self): records[0]["output"]["is_hit"] = False self.assertEqual(selection.compare_samples(records, selection.read_json(spec))["status"], "failed") + +class Rfc0057AuthoringAndDeploymentTest(unittest.TestCase): + def setUp(self): + self.temporary = tempfile.TemporaryDirectory(prefix="rfc0057-test-", dir=ROOT / "build") + self.root = Path(self.temporary.name) + self.configs = self.root / "configs" + self.configs.mkdir(parents=True, exist_ok=True) + self.service = SHOW.WorkbenchService(self.configs) + self.keyword = json.loads( + (ROOT / "configs" / "pipeline_keyword_match_rules.json").read_text() + ) + + def tearDown(self): + self.temporary.cleanup() + + def run_edit(self, request: dict) -> tuple[int, dict]: + proc = subprocess.run( + [str(PIPELINE_TOOL), "edit", "--stdin"], + input=json.dumps(request), + text=True, + capture_output=True, + cwd=ROOT, + check=False, + ) + try: + payload = json.loads(proc.stdout) + except json.JSONDecodeError: + payload = {"raw": proc.stdout, "stderr": proc.stderr} + return proc.returncode, payload + + def assert_edit_rejected(self, request, failed_index=None): + code, result = self.run_edit(request) + self.assertEqual(code, 1, result) + self.assertIs(result.get("ok"), False, result) + self.assertTrue(result.get("diagnostics"), result) + self.assertNotIn("pipeline", result) + if failed_index is not None: + self.assertEqual(result.get("failed_operation_index"), failed_index, result) + + def test_authoring_connect_preserves_valid_default_output_fanout(self): + pipe = copy.deepcopy(self.keyword) + existing = pipe["pipeline"][0] + existing["id"] = "b" + existing["depends_on"] = ["a"] + existing["ports"]["inputs"] = {} # Required text defaults to producer A.text. + source = { + "id": "a", "node_type": "TextTemplateNode", "depends_on": [], + "ports": {"inputs": {"primary": "input_sentences"}, "outputs": {}}, + "config": {"template": "{{primary}}"}, + } + consumer = copy.deepcopy(existing) + consumer.update(id="c", depends_on=[]) + consumer["ports"] = {"inputs": {"text": "input_sentences"}, + "outputs": {"matches": "other_matches"}} + pipe["pipeline"] = [source, existing, consumer] + self.assertTrue(self.service.validate(pipe)["ok"]) + code, result = self.run_edit({ + "schema_version": 1, "pipeline": pipe, "require_valid": True, + "operation": {"kind": "connect", + "source": {"node_id": "a", "port": "text"}, + "target": {"node_id": "c", "port": "text"}}, + }) + self.assertEqual(code, 0, result) + self.assertTrue(result["validation"]["ok"]) + a, b, c = result["pipeline"]["pipeline"] + self.assertEqual(a.get("ports", {}).get("outputs", {}).get("text", "text"), "text") + self.assertEqual(b, existing) + self.assertEqual(c["ports"]["inputs"]["text"], "text") + self.assertIn("a", c["depends_on"]) + + def test_authoring_disconnect_rejects_unknown_and_reversed_endpoints(self): + pipe = copy.deepcopy(self.keyword) + pipe["pipeline"][0]["id"] = "c" + pipe["pipeline"].insert(0, { + "id": "a", "node_type": "TextTemplateNode", "depends_on": [], + "ports": {"inputs": {"primary": "input_sentences"}, "outputs": {}}, + "config": {"template": "{{primary}}"}, + }) + cases = [ + ({"node_id": "a", "port": "input_sentences"}, {"node_id": "c", "port": "text"}), + ({"node_id": "$ingress", "port": "missing"}, {"node_id": "c", "port": "text"}), + ({"node_id": "$egress", "port": "rule_matches"}, {"node_id": "c", "port": "text"}), + ({"node_id": "c", "port": "matches"}, {"node_id": "$egress", "port": "missing"}), + ({"node_id": "c", "port": "matches"}, {"node_id": "$ingress", "port": "input_sentences"}), + ({"node_id": "$ingress", "port": "input_sentences"}, {"node_id": "c", "port": "missing"}), + ] + for source, target in cases: + with self.subTest(source=source, target=target): + self.assert_edit_rejected({ + "schema_version": 1, "pipeline": pipe, + "operation": {"kind": "disconnect", "source": source, "target": target}, + }, 0) + + def test_authoring_rejects_ambiguous_duplicate_node_targets(self): + pipe = copy.deepcopy(self.keyword) + pipe["pipeline"][0]["id"] = "b" + pipe["pipeline"].append(copy.deepcopy(pipe["pipeline"][0])) + operations = [ + {"kind": "rename_node", "node_id": "b", "new_id": "bb"}, + {"kind": "remove_node", "node_id": "b"}, + {"kind": "connect", "source": {"node_id": "$ingress", "port": "input_sentences"}, + "target": {"node_id": "b", "port": "text"}}, + {"kind": "connect", "source": {"node_id": "b", "port": "matches"}, + "target": {"node_id": "$egress", "port": "rule_matches"}}, + ] + for operation in operations: + with self.subTest(operation=operation): + self.assert_edit_rejected({"schema_version": 1, "pipeline": pipe, + "operation": operation}, 0) + + def test_authoring_rejects_invalid_request_fields_without_crashing(self): + base = {"schema_version": 1, "pipeline": self.keyword, + "operation": {"kind": "add_node", "node_type": "TextTemplateNode"}} + requests = [ + dict(base, requrie_valid=True), dict(base, require_valid="true"), + dict(base, schema_version=True), dict(base, schema_version=1.0), + dict(base, operations=None), dict(base, operation=None, operations=[base["operation"]]), + dict(base, operations={}), dict(base, pipeline=[]), + ] + for request in requests: + with self.subTest(request=request): + self.assert_edit_rejected(request) + + def test_authoring_batch_schema_errors_report_index_and_no_candidate(self): + invalid_operations = [ + {"kind": 42}, {"kind": "remove_node", "node_id": 42}, + {"kind": "add_node", "node_type": "TextTemplateNode", "config": []}, + {"kind": "add_node", "node_type": "TextTemplateNode", "unexpected": True}, + {"kind": "connect", "source": [], "target": {}}, + {"kind": "connect", "source": {"node_id": "$ingress", "port": "input_sentences", "unexpected": 1}, + "target": {"node_id": "added", "port": "primary"}}, + {"kind": "connect", "source": {"node_id": "$ingress", "port": 42}, + "target": {"node_id": "added", "port": "primary"}}, + None, [], + ] + for operation in invalid_operations: + with self.subTest(operation=operation): + self.assert_edit_rejected({ + "schema_version": 1, "pipeline": self.keyword, + "operations": [ + {"kind": "add_node", "node_type": "TextTemplateNode", "id": "added"}, + operation, + ], + }, 1) + + @unittest.skipUnless(sys.platform.startswith("linux"), "requires RLIMIT_FSIZE") + def test_fix_deps_short_write_preserves_original_and_cleans_temporary(self): + pipe = json.loads((ROOT / "configs" / "pipeline_doc_qa_default.json").read_text()) + pipe["pipeline"][2]["depends_on"] = [] + path = self.configs / "short-write.json" + original = json.dumps(pipe, indent=2).encode() + path.write_bytes(original) + path.chmod(0o640) + # A separate launcher avoids preexec_fn in this threaded service test process. + launcher = ( + "import os, resource, signal, sys; " + "signal.signal(signal.SIGXFSZ, signal.SIG_IGN); " + "resource.setrlimit(resource.RLIMIT_FSIZE, (128, 128)); " + "os.execv(sys.argv[1], sys.argv[1:])" + ) + before = set(self.configs.iterdir()) + proc = subprocess.run( + [sys.executable, "-c", launcher, str(PIPELINE_TOOL), "fix-deps", str(path), "--in-place"], + capture_output=True, text=True, cwd=ROOT, check=False, + ) + self.assertEqual(path.read_bytes(), original, "failed write must never replace the source") + self.assertEqual(path.stat().st_mode & 0o777, 0o640) + self.assertEqual(set(self.configs.iterdir()), before, "failed write must clean staging files") + self.assertEqual(proc.returncode, 1, proc.stderr) + result = json.loads(proc.stdout) + self.assertFalse(result["ok"]) + self.assertFalse(result.get("written", False)) + self.assertTrue(result.get("diagnostics")) + + def test_authoring_add_node_explicit_auto_id_and_collision(self): + req = { + "schema_version": 1, + "pipeline": {"biz_name": "keyword_match_v1", "models": [], "pipeline": []}, + "operation": { + "kind": "add_node", + "node_type": "TextRuleMatchNode", + "id": "custom_rule", + }, + } + code, res = self.run_edit(req) + self.assertEqual(code, 0) + self.assertTrue(res["ok"]) + self.assertEqual(res["pipeline"]["pipeline"][0]["id"], "custom_rule") + + # Duplicate ID rejection + req2 = { + "schema_version": 1, + "pipeline": res["pipeline"], + "operation": { + "kind": "add_node", + "node_type": "TextRuleMatchNode", + "id": "custom_rule", + }, + } + code2, res2 = self.run_edit(req2) + self.assertEqual(code2, 1) + self.assertFalse(res2["ok"]) + self.assertIn("DUPLICATE_NODE_ID", res2["diagnostics"][0]["message"]) + + # Auto ID allocation + req3 = { + "schema_version": 1, + "pipeline": res["pipeline"], + "operation": { + "kind": "add_node", + "node_type": "TextRuleMatchNode", + }, + } + code3, res3 = self.run_edit(req3) + self.assertEqual(code3, 0) + self.assertTrue(res3["ok"]) + self.assertNotEqual(res3["pipeline"]["pipeline"][1]["id"], "custom_rule") + + # Unknown node type + req4 = { + "schema_version": 1, + "pipeline": res["pipeline"], + "operation": {"kind": "add_node", "node_type": "NonExistentNode"}, + } + code4, res4 = self.run_edit(req4) + self.assertEqual(code4, 1) + self.assertFalse(res4["ok"]) + self.assertIn("UNKNOWN_NODE_TYPE", res4["diagnostics"][0]["message"]) + + def test_authoring_remove_node_detaches_inputs_and_dependencies(self): + pipe = { + "biz_name": "keyword_match_v1", + "models": [], + "pipeline": [ + { + "id": "template", + "node_type": "TextTemplateNode", + "depends_on": [], + "ports": {"inputs": {}, "outputs": {"text": "tpl_text"}}, + "config": {"template": "{{primary}}"}, + }, + { + "id": "rule", + "node_type": "TextRuleMatchNode", + "depends_on": ["template"], + "ports": {"inputs": {"text": "tpl_text"}, "outputs": {}}, + "config": {}, + }, + ], + } + req = { + "schema_version": 1, + "pipeline": pipe, + "operation": {"kind": "remove_node", "node_id": "template"}, + } + code, res = self.run_edit(req) + self.assertEqual(code, 0) + self.assertTrue(res["ok"]) + self.assertEqual(len(res["pipeline"]["pipeline"]), 1) + rule_node = res["pipeline"]["pipeline"][0] + self.assertEqual(rule_node["id"], "rule") + # depends_on detached + self.assertEqual(rule_node["depends_on"], []) + # Required input text assigned unconnected placeholder + self.assertIn("__unconnected__text", rule_node["ports"]["inputs"]["text"]) + + def test_authoring_rename_node_syncs_dependencies_and_preserves_keys(self): + pipe = { + "biz_name": "keyword_match_v1", + "models": [], + "pipeline": [ + { + "id": "template", + "node_type": "TextTemplateNode", + "depends_on": [], + "ports": {"inputs": {}, "outputs": {"text": "tpl_text"}}, + "config": {"template": "{{primary}}"}, + }, + { + "id": "rule", + "node_type": "TextRuleMatchNode", + "depends_on": ["template"], + "ports": {"inputs": {"text": "tpl_text"}, "outputs": {}}, + "config": {}, + }, + ], + } + req = { + "schema_version": 1, + "pipeline": pipe, + "operation": { + "kind": "rename_node", + "node_id": "template", + "new_id": "new_template", + }, + } + code, res = self.run_edit(req) + self.assertEqual(code, 0) + self.assertTrue(res["ok"]) + tpl = res["pipeline"]["pipeline"][0] + rule = res["pipeline"]["pipeline"][1] + self.assertEqual(tpl["id"], "new_template") + # Data keys are preserved + self.assertEqual(tpl["ports"]["outputs"]["text"], "tpl_text") + # depends_on updated + self.assertEqual(rule["depends_on"], ["new_template"]) + + # Duplicate ID rejection + req_dup = { + "schema_version": 1, + "pipeline": pipe, + "operation": {"kind": "rename_node", "node_id": "template", "new_id": "rule"}, + } + code_dup, res_dup = self.run_edit(req_dup) + self.assertEqual(code_dup, 1) + self.assertIn("DUPLICATE_NODE_ID", res_dup["diagnostics"][0]["message"]) + + def test_authoring_connect_and_cycle_and_redundant_dependencies(self): + pipe = { + "biz_name": "keyword_match_v1", + "models": [], + "pipeline": [ + { + "id": "template", + "node_type": "TextTemplateNode", + "depends_on": [], + "ports": {"inputs": {}, "outputs": {}}, + "config": {"template": "{{primary}}"}, + }, + { + "id": "rule", + "node_type": "TextRuleMatchNode", + "depends_on": [], + "ports": {"inputs": {}, "outputs": {}}, + "config": {}, + }, + ], + } + # Ingress to node + req1 = { + "schema_version": 1, + "pipeline": pipe, + "operation": { + "kind": "connect", + "source": {"node_id": "$ingress", "port": "input_sentences"}, + "target": {"node_id": "template", "port": "primary"}, + }, + } + code1, res1 = self.run_edit(req1) + self.assertEqual(code1, 0) + self.assertEqual( + res1["pipeline"]["pipeline"][0]["ports"]["inputs"]["primary"], + "input_sentences", + ) + self.assertEqual(res1["pipeline"]["pipeline"][0]["depends_on"], []) + + # Node to node + req2 = { + "schema_version": 1, + "pipeline": res1["pipeline"], + "operation": { + "kind": "connect", + "source": {"node_id": "template", "port": "text"}, + "target": {"node_id": "rule", "port": "text"}, + }, + } + code2, res2 = self.run_edit(req2) + self.assertEqual(code2, 0) + rule = res2["pipeline"]["pipeline"][1] + tpl_key = res2["pipeline"]["pipeline"][0]["ports"]["outputs"].get("text", "text") + self.assertEqual(tpl_key, "text", "connecting preserves the effective default output key") + self.assertEqual(rule["ports"]["inputs"]["text"], tpl_key) + self.assertEqual(rule["depends_on"], ["template"]) + + # Cycle detection: connecting rule to template creates cycle + req_cycle = { + "schema_version": 1, + "pipeline": res2["pipeline"], + "operation": { + "kind": "connect", + "source": {"node_id": "rule", "port": "matches"}, + "target": {"node_id": "template", "port": "primary"}, + }, + } + code_cycle, res_cycle = self.run_edit(req_cycle) + self.assertEqual(code_cycle, 1) + self.assertFalse(res_cycle["ok"]) + + def test_authoring_connect_and_disconnect_biz_egress(self): + pipe = { + "biz_name": "keyword_match_v1", + "models": [], + "pipeline": [ + { + "id": "rule", + "node_type": "TextRuleMatchNode", + "depends_on": [], + "ports": { + "inputs": {"text": "input_sentences"}, + "outputs": {"matches": "rule_internal_out"}, + }, + "config": {}, + }, + { + "id": "audit", + "node_type": "TextRuleMatchNode", + "depends_on": ["rule"], + "ports": {"inputs": {"text": "rule_internal_out"}, "outputs": {}}, + "config": {}, + }, + ], + } + # Connect rule.matches to $egress rule_matches + req_egress = { + "schema_version": 1, + "pipeline": pipe, + "operation": { + "kind": "connect", + "source": {"node_id": "rule", "port": "matches"}, + "target": {"node_id": "$egress", "port": "rule_matches"}, + }, + } + code_egress, res_egress = self.run_edit(req_egress) + self.assertEqual(code_egress, 0) + rule_out = res_egress["pipeline"]["pipeline"][0]["ports"]["outputs"]["matches"] + audit_in = res_egress["pipeline"]["pipeline"][1]["ports"]["inputs"]["text"] + self.assertEqual(rule_out, "rule_matches") + self.assertEqual(audit_in, "rule_matches") + + # Disconnect rule.matches from $egress rule_matches + req_disc = { + "schema_version": 1, + "pipeline": res_egress["pipeline"], + "operation": { + "kind": "disconnect", + "source": {"node_id": "rule", "port": "matches"}, + "target": {"node_id": "$egress", "port": "rule_matches"}, + }, + } + code_disc, res_disc = self.run_edit(req_disc) + self.assertEqual(code_disc, 0) + rule_out2 = res_disc["pipeline"]["pipeline"][0]["ports"]["outputs"]["matches"] + audit_in2 = res_disc["pipeline"]["pipeline"][1]["ports"]["inputs"]["text"] + self.assertNotEqual(rule_out2, "rule_matches") + self.assertEqual(rule_out2, audit_in2) + + def test_authoring_disconnect_preserves_execution_dependency_and_placeholders(self): + pipe = { + "biz_name": "keyword_match_v1", + "models": [], + "pipeline": [ + { + "id": "tpl", + "node_type": "TextTemplateNode", + "depends_on": [], + "ports": {"inputs": {}, "outputs": {"text": "tpl_out"}}, + "config": {"template": "{{primary}}"}, + }, + { + "id": "rule", + "node_type": "TextRuleMatchNode", + "depends_on": ["tpl"], + "ports": {"inputs": {"text": "tpl_out"}, "outputs": {}}, + "config": {}, + }, + ], + } + # Disconnecting data port preserves execution dependency (Regression Case 1) + req = { + "schema_version": 1, + "pipeline": pipe, + "operation": { + "kind": "disconnect", + "source": {"node_id": "tpl", "port": "text"}, + "target": {"node_id": "rule", "port": "text"}, + }, + } + code, res = self.run_edit(req) + self.assertEqual(code, 0) + rule = res["pipeline"]["pipeline"][1] + self.assertEqual(rule["depends_on"], ["tpl"]) + self.assertIn("__unconnected__text", rule["ports"]["inputs"]["text"]) + + # reset_input_binding removes the placeholder + req_reset = { + "schema_version": 1, + "pipeline": res["pipeline"], + "operation": { + "kind": "reset_input_binding", + "target": {"node_id": "rule", "port": "text"}, + }, + } + code_reset, res_reset = self.run_edit(req_reset) + self.assertEqual(code_reset, 0) + self.assertNotIn("text", res_reset["pipeline"]["pipeline"][1]["ports"]["inputs"]) + + # remove_dependency removes execution dependency + req_rm_dep = { + "schema_version": 1, + "pipeline": res_reset["pipeline"], + "operation": { + "kind": "remove_dependency", + "node_id": "rule", + "depends_on_id": "tpl", + }, + } + code_rm_dep, res_rm_dep = self.run_edit(req_rm_dep) + self.assertEqual(code_rm_dep, 0) + self.assertEqual(res_rm_dep["pipeline"]["pipeline"][1]["depends_on"], []) + + def test_authoring_batch_operations_and_atomic_rollback(self): + pipe = {"biz_name": "keyword_match_v1", "models": [], "pipeline": []} + req_valid = { + "schema_version": 1, + "pipeline": pipe, + "operations": [ + {"kind": "add_node", "node_type": "TextTemplateNode", "id": "t1", "config": {"template": "{{primary}}"}}, + {"kind": "add_node", "node_type": "TextRuleMatchNode", "id": "r1", "config": {"categories": {"K": ["v"]}}}, + {"kind": "connect", "source": {"node_id": "$ingress", "port": "input_sentences"}, "target": {"node_id": "t1", "port": "primary"}}, + {"kind": "connect", "source": {"node_id": "t1", "port": "text"}, "target": {"node_id": "r1", "port": "text"}}, + {"kind": "connect", "source": {"node_id": "r1", "port": "matches"}, "target": {"node_id": "$egress", "port": "rule_matches"}}, + ], + "require_valid": True, + } + code_valid, res_valid = self.run_edit(req_valid) + self.assertEqual(code_valid, 0) + self.assertTrue(res_valid["ok"]) + self.assertTrue(res_valid["validation"]["ok"]) + self.assertEqual(len(res_valid["pipeline"]["pipeline"]), 2) + + # Failure mid-batch discards changes + req_fail = { + "schema_version": 1, + "pipeline": pipe, + "operations": [ + {"kind": "add_node", "node_type": "TextTemplateNode", "id": "t1"}, + {"kind": "add_node", "node_type": "InvalidNodeType"}, + {"kind": "add_node", "node_type": "TextRuleMatchNode", "id": "r1"}, + ], + } + code_fail, res_fail = self.run_edit(req_fail) + self.assertEqual(code_fail, 1) + self.assertFalse(res_fail["ok"]) + self.assertEqual(res_fail["failed_operation_index"], 1) + self.assertNotIn("pipeline", res_fail) + + # Batch limit > 128 + req_oversize = { + "schema_version": 1, + "pipeline": pipe, + "operations": [{"kind": "add_node", "node_type": "TextTemplateNode"}] * 129, + } + code_oversize, res_oversize = self.run_edit(req_oversize) + self.assertEqual(code_oversize, 1) + self.assertFalse(res_oversize["ok"]) + + def test_authoring_regression_case_2_key_allocation_no_collision(self): + pipe = {"biz_name": "keyword_match_v1", "models": [], "pipeline": []} + req = { + "schema_version": 1, + "pipeline": pipe, + "operations": [ + {"kind": "add_node", "node_type": "TextRuleMatchNode", "id": "foo"}, + {"kind": "rename_node", "node_id": "foo", "new_id": "bar"}, + {"kind": "add_node", "node_type": "TextRuleMatchNode", "id": "foo"}, + ], + } + code, res = self.run_edit(req) + self.assertEqual(code, 0) + self.assertTrue(res["ok"]) + bar_node = res["pipeline"]["pipeline"][0] + foo_node = res["pipeline"]["pipeline"][1] + self.assertEqual(bar_node["id"], "bar") + self.assertEqual(foo_node["id"], "foo") + bar_key = bar_node["ports"]["outputs"]["matches"] + foo_key = foo_node["ports"]["outputs"]["matches"] + self.assertNotEqual(bar_key, foo_key, "Allocated keys must not collide with retained keys") + + def test_fix_deps_cli_preview_and_inplace_and_idempotent(self): + pipe = json.loads((ROOT / "configs" / "pipeline_doc_qa_default.json").read_text()) + # Remove dependency from node 2 + pipe["pipeline"][2]["depends_on"] = [] + with tempfile.TemporaryDirectory() as td: + fpath = Path(td) / "pipeline_missing_dep.json" + fpath.write_text(json.dumps(pipe, indent=2)) + + # Preview mode: written is false, file unchanged + proc_prev = subprocess.run( + [str(PIPELINE_TOOL), "fix-deps", str(fpath)], + capture_output=True, + text=True, + cwd=ROOT, + check=False, + ) + self.assertEqual(proc_prev.returncode, 0) + res_prev = json.loads(proc_prev.stdout) + self.assertTrue(res_prev["ok"]) + self.assertFalse(res_prev["written"]) + self.assertEqual(len(res_prev["changes"]), 1) + self.assertEqual(res_prev["changes"][0]["action"], "add_dependency") + self.assertEqual( + json.loads(fpath.read_text())["pipeline"][2]["depends_on"], [] + ) + + # In-place mode: written is true, file updated + proc_fix = subprocess.run( + [str(PIPELINE_TOOL), "fix-deps", str(fpath), "--in-place"], + capture_output=True, + text=True, + cwd=ROOT, + check=False, + ) + self.assertEqual(proc_fix.returncode, 0) + res_fix = json.loads(proc_fix.stdout) + self.assertTrue(res_fix["ok"]) + self.assertTrue(res_fix["written"]) + fixed_pipe = json.loads(fpath.read_text()) + self.assertEqual( + fixed_pipe["pipeline"][2]["depends_on"], ["node_0_TextChunkNode"] + ) + + # Repeated in-place mode: idempotent, written is false, changes empty + proc_idem = subprocess.run( + [str(PIPELINE_TOOL), "fix-deps", str(fpath), "--in-place"], + capture_output=True, + text=True, + cwd=ROOT, + check=False, + ) + self.assertEqual(proc_idem.returncode, 0) + res_idem = json.loads(proc_idem.stdout) + self.assertTrue(res_idem["ok"]) + self.assertFalse(res_idem["written"]) + self.assertEqual(len(res_idem["changes"]), 0) + + # Symlink rejected + link_path = Path(td) / "symlink.json" + link_path.symlink_to(fpath) + proc_link = subprocess.run( + [str(PIPELINE_TOOL), "fix-deps", str(link_path)], + capture_output=True, + text=True, + cwd=ROOT, + check=False, + ) + self.assertEqual(proc_link.returncode, 1) + res_link = json.loads(proc_link.stdout) + self.assertFalse(res_link["ok"]) + self.assertIn("SYMLINK_REJECTED", res_link["diagnostics"][0]["message"]) + + def test_studio_authoring_preview_endpoint(self): + pipe = {"biz_name": "keyword_match_v1", "models": [], "pipeline": []} + res = self.service.preview_authoring( + pipe, + operation={"kind": "add_node", "node_type": "TextRuleMatchNode", "id": "n1"}, + expected_revision=SHOW.revision_for(json.dumps(pipe, sort_keys=True).encode()), + tool_fingerprint=self.service.get_tool_fingerprint(), + ) + self.assertTrue(res["ok"]) + self.assertIn("revision", res) + self.assertIn("tool_fingerprint", res) + self.assertEqual(res["pipeline"]["pipeline"][0]["id"], "n1") + + # Stale revision conflict + with self.assertRaises(SHOW.StudioError) as ctx: + self.service.preview_authoring( + pipe, + operation={"kind": "add_node", "node_type": "TextRuleMatchNode"}, + expected_revision="wrong_rev", + tool_fingerprint=self.service.get_tool_fingerprint(), + ) + self.assertEqual(ctx.exception.code, "REVISION_CONFLICT") + + def test_studio_authoring_requires_current_revision_and_tool(self): + revision = SHOW.revision_for(json.dumps(self.keyword, sort_keys=True).encode()) + fingerprint = self.service.get_tool_fingerprint() + for expected, tool, error in [ + (None, fingerprint, "REVISION_CONFLICT"), + (revision, None, "TOOL_OUTDATED"), + (revision, "outdated", "TOOL_OUTDATED"), + ]: + with self.subTest(revision=expected, fingerprint=tool): + with mock.patch.object(self.service, "invoke_tool") as invoke: + with self.assertRaises(SHOW.StudioError) as ctx: + self.service.preview_authoring( + self.keyword, operation={"kind": "add_node", "node_type": "TextTemplateNode"}, + expected_revision=expected, tool_fingerprint=tool, + ) + self.assertEqual(ctx.exception.code, error) + invoke.assert_not_called() + + def test_studio_preflight_independent_summary(self): + preflight_res = self.service.preflight( + self.keyword, + profile_name="keyword_match_rules", + ) + self.assertTrue(preflight_res["ok"]) + summary = preflight_res["summary"] + self.assertEqual(summary["biz_name"], "keyword_match_v1") + self.assertIn("tools", summary) + self.assertIn("pipeline_snapshot", summary) + self.assertIn("status", summary) + # Verify no user file was written + self.assertFalse((self.configs / "pipeline_preflight_leak.json").exists()) + + def associated_doc_qa(self): + pipeline = json.loads((ROOT / "configs" / "pipeline_doc_qa_cpu.json").read_text()) + conf = json.loads((ROOT / "configs" / "pipeline_doc_qa_cpu.conf").read_text()) + pipeline_path = self.configs / "pipeline_associated.json" + conf_path = self.configs / "pipeline_associated.conf" + conf["data"]["pipe_path"] = str(pipeline_path.relative_to(ROOT)) + for model_id in conf["data"]["model_paths"]: + conf["data"]["model_paths"][model_id] = "models/deployed_" + model_id + conf["data"]["outputs"]["doc_out"]["capacities"]["answer_text"] = 2047 + pipeline_path.write_text(json.dumps(pipeline)) + conf_path.write_text(json.dumps(conf)) + self.service.associate_deployment(pipeline_path.name, conf_path.name) + return pipeline, conf, pipeline_path, conf_path + + def test_associated_preflight_run_and_save_share_candidate(self): + pipeline, original_conf, path, conf_path = self.associated_doc_qa() + pipeline["models"][0]["model_path"] = "selected_A.onnx" + model_id = pipeline["models"][0]["model_id"] + actions = {model_id: {"path": "selected_A.onnx", "action": "select_asset"}} + expected = copy.deepcopy(original_conf) + expected["data"]["model_paths"][model_id] = "models/selected_A.onnx" + # Spy on real native resolution so this checks exactly what preflight resolves. + resolved_candidates = [] + resolve = self.service.resolve_run_conf + + def capture_resolve(conf_file, profile): + resolved_candidates.append(json.loads(Path(conf_file).read_text())) + return resolve(conf_file, profile) + + with mock.patch.object(self.service, "resolve_run_conf", side_effect=capture_resolve): + preview = self.service.preflight( + pipeline, filename=path.name, model_path_actions=actions, + ) + self.assertTrue(preview["ok"], preview) + self.assertEqual(len(resolved_candidates), 1) + with mock.patch.object(SHOW.threading, "Thread") as thread: + self.service.start_run(pipeline, "doc_qa_cpu", filename=path.name, + model_path_actions=actions) + args = thread.call_args.kwargs["args"] + run_profile, run_conf = args[2], args[3] + self.assertEqual(run_profile["biz"], "doc_qa") + self.service.save_pipeline( + path.name, pipeline, SHOW.revision_for(path.read_bytes()), + model_path_actions=actions, + ) + saved = json.loads(conf_path.read_text()) + for candidate in [resolved_candidates[0], run_conf, saved]: + with self.subTest(candidate=candidate): + candidate = copy.deepcopy(candidate) + candidate["data"]["pipe_path"] = expected["data"]["pipe_path"] + self.assertEqual(candidate, expected) + + def test_associated_raw_model_edit_requires_explicit_override_intent(self): + pipeline, conf, path, _ = self.associated_doc_qa() + model = pipeline["models"][0] + model["model_path"] = "raw_edit.onnx" + with self.assertRaises(SHOW.StudioError) as ctx: + self.service.deployment_candidate(pipeline, filename=path.name) + self.assertEqual(ctx.exception.code, "DEPLOYMENT_PATH_INTENT_REQUIRED") + _, candidate = self.service.deployment_candidate( + pipeline, filename=path.name, + model_path_actions={model["model_id"]: {"path": model["model_path"], + "action": "preserve_override"}}, + ) + self.assertEqual(candidate["data"]["model_paths"], conf["data"]["model_paths"]) + self.assertEqual(candidate["data"]["outputs"], conf["data"]["outputs"]) + + def test_associated_new_model_does_not_invent_deployment_override(self): + pipeline, conf, path, _ = self.associated_doc_qa() + new_model = copy.deepcopy(pipeline["models"][0]) + new_model.update(model_id="new_model", model_path="new.onnx") + pipeline["models"].append(new_model) + _, candidate = self.service.deployment_candidate(pipeline, filename=path.name) + self.assertEqual(candidate["data"]["model_paths"], conf["data"]["model_paths"]) + + def test_associated_external_file_changes_block_candidate_and_save(self): + for changed_name in ["pipeline", "conf"]: + with self.subTest(changed=changed_name): + pipeline, _, path, conf_path = self.associated_doc_qa() + revision = SHOW.revision_for(path.read_bytes()) + changed = path if changed_name == "pipeline" else conf_path + changed.write_bytes(changed.read_bytes() + b"\n") + originals = (path.read_bytes(), conf_path.read_bytes()) + with self.assertRaises(SHOW.StudioError) as ctx: + self.service.deployment_candidate(pipeline, filename=path.name) + self.assertEqual(ctx.exception.code, "REVISION_CONFLICT") + with self.assertRaises(SHOW.StudioError) as ctx: + self.service.save_pipeline(path.name, pipeline, revision) + self.assertEqual(ctx.exception.code, "REVISION_CONFLICT") + self.assertEqual((path.read_bytes(), conf_path.read_bytes()), originals) + + def test_studio_deployment_associate_and_partial_override_update(self): + # Create pipeline in configs + pipe_path = self.configs / "pipeline_doc_qa_assoc.json" + doc_qa_pipe = json.loads((ROOT / "configs" / "pipeline_doc_qa_cpu.json").read_text()) + pipe_path.write_text(json.dumps(doc_qa_pipe, indent=2)) + + # Create conf in configs pointing to this pipeline + conf_path = self.configs / "pipeline_doc_qa_assoc.conf" + doc_qa_conf = json.loads((ROOT / "configs" / "pipeline_doc_qa_cpu.conf").read_text()) + doc_qa_conf["data"]["pipe_path"] = str(pipe_path.relative_to(ROOT)) + conf_path.write_text(json.dumps(doc_qa_conf, indent=2)) + + # Associate + assoc_res = self.service.associate_deployment( + "pipeline_doc_qa_assoc.json", "pipeline_doc_qa_assoc.conf" + ) + self.assertTrue(assoc_res["ok"]) + self.assertEqual(assoc_res["conf_name"], "pipeline_doc_qa_assoc.conf") + self.assertIn("pipeline_doc_qa_assoc.conf", self.service.save_targets(pipe_path)) + + # Now update pipeline models and save + modified_pipe = copy.deepcopy(doc_qa_pipe) + # Update model_path of first model + modified_pipe["models"][0]["model_path"] = "new_embed_model.onnx" + pipe_raw = pipe_path.read_bytes() + pipe_rev = SHOW.revision_for(pipe_raw) + + save_res = self.service.save_pipeline( + "pipeline_doc_qa_assoc.json", modified_pipe, pipe_rev, save_as=False, + model_path_actions={modified_pipe["models"][0]["model_id"]: { + "path": "new_embed_model.onnx", "action": "select_asset"}}, + ) + self.assertTrue(save_res["ok"]) + + # Verify conf was updated with modified model_path override + updated_conf = json.loads(conf_path.read_text()) + self.assertEqual( + updated_conf["data"]["model_paths"][modified_pipe["models"][0]["model_id"]], + "models/new_embed_model.onnx", + ) + # Verify non-model conf settings preserved + self.assertIn("doc_out", updated_conf["data"]["outputs"]) + + def test_authoring_oversized_payload_rejection_4mib(self): + pipe = {"biz_name": "keyword_match_v1", "models": [], "pipeline": []} + large_comment = "x" * (4 * 1024 * 1024 + 100) + req = { + "schema_version": 1, + "pipeline": pipe, + "operation": {"kind": "add_node", "node_type": "TextRuleMatchNode"}, + "comment": large_comment, + } + code, res = self.run_edit(req) + self.assertEqual(code, 1) + self.assertFalse(res["ok"]) + self.assertIn("REQUEST_TOO_LARGE", res["diagnostics"][0]["message"]) + + def test_authoring_reserved_node_ids_rejected(self): + pipe = {"biz_name": "keyword_match_v1", "models": [], "pipeline": []} + req_add_ingress = { + "schema_version": 1, + "pipeline": pipe, + "operation": {"kind": "add_node", "node_type": "TextRuleMatchNode", "id": "$ingress"}, + } + code1, res1 = self.run_edit(req_add_ingress) + self.assertEqual(code1, 1) + self.assertFalse(res1["ok"]) + self.assertIn("RESERVED_NODE_ID", res1["diagnostics"][0]["message"]) + + pipe2 = { + "biz_name": "keyword_match_v1", + "models": [], + "pipeline": [ + {"id": "node_a", "node_type": "TextRuleMatchNode", "depends_on": [], "ports": {"inputs": {}, "outputs": {}}, "config": {}} + ], + } + req_rename_egress = { + "schema_version": 1, + "pipeline": pipe2, + "operation": {"kind": "rename_node", "node_id": "node_a", "new_id": "$egress"}, + } + code2, res2 = self.run_edit(req_rename_egress) + self.assertEqual(code2, 1) + self.assertFalse(res2["ok"]) + self.assertIn("RESERVED_NODE_ID", res2["diagnostics"][0]["message"]) + + def test_authoring_disconnect_implicit_bindings(self): + # Implicit node-to-node binding: rule requires text, omitted in ports.inputs + pipe = { + "biz_name": "keyword_match_v1", + "models": [], + "pipeline": [ + { + "id": "tpl", + "node_type": "TextTemplateNode", + "depends_on": [], + "ports": {"inputs": {}, "outputs": {"text": "text"}}, + "config": {"template": "{{primary}}"}, + }, + { + "id": "rule", + "node_type": "TextRuleMatchNode", + "depends_on": ["tpl"], + "ports": {"inputs": {}, "outputs": {}}, + "config": {}, + }, + ], + } + req_disc = { + "schema_version": 1, + "pipeline": pipe, + "operation": { + "kind": "disconnect", + "source": {"node_id": "tpl", "port": "text"}, + "target": {"node_id": "rule", "port": "text"}, + }, + } + code, res = self.run_edit(req_disc) + self.assertEqual(code, 0) + self.assertTrue(res["ok"]) + rule = res["pipeline"]["pipeline"][1] + self.assertEqual(rule["depends_on"], ["tpl"]) + self.assertIn("__unconnected__text", rule["ports"]["inputs"]["text"]) + + # Ingress disconnect + pipe_ing = { + "biz_name": "keyword_match_v1", + "models": [], + "pipeline": [ + { + "id": "rule", + "node_type": "TextRuleMatchNode", + "depends_on": [], + "ports": {"inputs": {"text": "input_sentences"}, "outputs": {}}, + "config": {}, + } + ], + } + req_ing_disc = { + "schema_version": 1, + "pipeline": pipe_ing, + "operation": { + "kind": "disconnect", + "source": {"node_id": "$ingress", "port": "input_sentences"}, + "target": {"node_id": "rule", "port": "text"}, + }, + } + code_ing, res_ing = self.run_edit(req_ing_disc) + self.assertEqual(code_ing, 0) + self.assertTrue(res_ing["ok"]) + self.assertIn("__unconnected__text", res_ing["pipeline"]["pipeline"][0]["ports"]["inputs"]["text"]) + + def test_authoring_disconnect_egress_syncs_implicit_consumers(self): + pipe = { + "biz_name": "keyword_match_v1", + "models": [], + "pipeline": [ + { + "id": "rule", + "node_type": "TextRuleMatchNode", + "depends_on": [], + "ports": { + "inputs": {"text": "input_sentences"}, + "outputs": {"matches": "rule_matches"}, + }, + "config": {}, + }, + { + "id": "consumer", + "node_type": "TextRuleMatchNode", + "depends_on": ["rule"], + "ports": { + "inputs": {"text": "rule_matches"}, + "outputs": {}, + }, + "config": {}, + }, + ], + } + req = { + "schema_version": 1, + "pipeline": pipe, + "operation": { + "kind": "disconnect", + "source": {"node_id": "rule", "port": "matches"}, + "target": {"node_id": "$egress", "port": "rule_matches"}, + }, + } + code, res = self.run_edit(req) + self.assertEqual(code, 0) + self.assertTrue(res["ok"]) + rule = res["pipeline"]["pipeline"][0] + consumer = res["pipeline"]["pipeline"][1] + new_key = rule["ports"]["outputs"]["matches"] + self.assertNotEqual(new_key, "rule_matches") + self.assertEqual(consumer["ports"]["inputs"]["text"], new_key) + + def test_authoring_remove_node_handles_implicit_consumers_and_egress(self): + pipe = { + "biz_name": "keyword_match_v1", + "models": [], + "pipeline": [ + { + "id": "tpl", + "node_type": "TextTemplateNode", + "depends_on": [], + "ports": {"inputs": {}, "outputs": {"text": "tpl_out"}}, + "config": {"template": "{{primary}}"}, + }, + { + "id": "rule", + "node_type": "TextRuleMatchNode", + "depends_on": ["tpl"], + "ports": {"inputs": {"text": "tpl_out"}, "outputs": {"matches": "rule_matches"}}, + "config": {}, + }, + ], + } + req = { + "schema_version": 1, + "pipeline": pipe, + "operation": {"kind": "remove_node", "node_id": "rule"}, + } + code, res = self.run_edit(req) + self.assertEqual(code, 0) + self.assertTrue(res["ok"]) + change = res["changes"][0] + self.assertIn("$egress", change["affected_nodes"]) + + def test_authoring_add_dependency_avoids_redundant_indirect_ancestor(self): + pipe = { + "biz_name": "keyword_match_v1", + "models": [], + "pipeline": [ + {"id": "a", "node_type": "TextRuleMatchNode", "depends_on": [], "ports": {"inputs": {}, "outputs": {}}, "config": {}}, + {"id": "b", "node_type": "TextRuleMatchNode", "depends_on": ["a"], "ports": {"inputs": {}, "outputs": {}}, "config": {}}, + {"id": "c", "node_type": "TextRuleMatchNode", "depends_on": ["b"], "ports": {"inputs": {}, "outputs": {}}, "config": {}}, + ], + } + # Adding c depends on a: a is already an indirect ancestor of c via b + req = { + "schema_version": 1, + "pipeline": pipe, + "operation": {"kind": "add_dependency", "node_id": "c", "depends_on_id": "a"}, + } + code, res = self.run_edit(req) + self.assertEqual(code, 0) + self.assertTrue(res["ok"]) + c_node = res["pipeline"]["pipeline"][2] + self.assertEqual(c_node["depends_on"], ["b"]) + + def test_fix_deps_fails_closed_on_ambiguity_and_cycles(self): + with tempfile.TemporaryDirectory() as td: + # Ambiguity: two nodes output the same key 'tpl_out' + ambig_pipe = { + "biz_name": "keyword_match_v1", + "models": [], + "pipeline": [ + {"id": "a1", "node_type": "TextTemplateNode", "depends_on": [], "ports": {"inputs": {}, "outputs": {"text": "tpl_out"}}, "config": {"template": "1"}}, + {"id": "a2", "node_type": "TextTemplateNode", "depends_on": [], "ports": {"inputs": {}, "outputs": {"text": "tpl_out"}}, "config": {"template": "2"}}, + {"id": "b", "node_type": "TextRuleMatchNode", "depends_on": [], "ports": {"inputs": {"text": "tpl_out"}, "outputs": {}}, "config": {}}, + ], + } + f_ambig = Path(td) / "ambig.json" + f_ambig.write_text(json.dumps(ambig_pipe, indent=2)) + proc_ambig = subprocess.run( + [str(PIPELINE_TOOL), "fix-deps", str(f_ambig), "--in-place"], + capture_output=True, + text=True, + cwd=ROOT, + check=False, + ) + self.assertEqual(proc_ambig.returncode, 1) + res_ambig = json.loads(proc_ambig.stdout) + self.assertFalse(res_ambig["ok"]) + self.assertIn("AMBIGUOUS_PRODUCER", res_ambig["diagnostics"][0]["message"]) + self.assertEqual(json.loads(f_ambig.read_text()), ambig_pipe) + + # Cycle: a depends on b, b needs a's output + cycle_pipe = { + "biz_name": "keyword_match_v1", + "models": [], + "pipeline": [ + {"id": "a", "node_type": "TextTemplateNode", "depends_on": ["b"], "ports": {"inputs": {}, "outputs": {"text": "tpl_out"}}, "config": {"template": "1"}}, + {"id": "b", "node_type": "TextRuleMatchNode", "depends_on": [], "ports": {"inputs": {"text": "tpl_out"}, "outputs": {}}, "config": {}}, + ], + } + f_cycle = Path(td) / "cycle.json" + f_cycle.write_text(json.dumps(cycle_pipe, indent=2)) + proc_cycle = subprocess.run( + [str(PIPELINE_TOOL), "fix-deps", str(f_cycle), "--in-place"], + capture_output=True, + text=True, + cwd=ROOT, + check=False, + ) + self.assertEqual(proc_cycle.returncode, 1) + res_cycle = json.loads(proc_cycle.stdout) + self.assertFalse(res_cycle["ok"]) + self.assertIn("CYCLE_DETECTED", res_cycle["diagnostics"][0]["message"]) + self.assertEqual(json.loads(f_cycle.read_text()), cycle_pipe) + + if __name__ == "__main__": unittest.main() diff --git a/tools/pipeline_studio/README.md b/tools/pipeline_studio/README.md index cecc1d03..63408d37 100644 --- a/tools/pipeline_studio/README.md +++ b/tools/pipeline_studio/README.md @@ -81,9 +81,14 @@ C++ 查看工具，展示更多声明信息： `Ctrl` / `⌘` + `Shift` + `Z` 或 `Ctrl` / `⌘` + `Y` 重做。 在文本输入控件中这些编辑快捷键保留给文本自身。 - 节点、模型或 JSON 表单修改后显示“有未应用修改”。点击“应用”更新方案，或点击 - “放弃修改”恢复表单。保存、校验、运行、切换节点或返回浏览遇到未应用修改时， - 可在详情面板选择“应用并继续”“放弃并继续”或“留在当前”。非法字段保留原输入并 - 定位错误，修正后可再次继续。一次编辑一个表单；应用后仍需保存到文件。 + “放弃修改”恢复表单。校验、运行和保存会一次应用当前合法表单并继续；非法字段保留原输入并 + 定位错误，不启动运行或保存。切换节点或返回浏览时，可选择“应用并继续”“放弃并继续”或 + “留在当前”。一次编辑一个表单；应用后仍需保存到文件。 +- 图上的新增、删节点、重命名和连接操作统一使用原生工具。断开数据线保留执行依赖；节点属性的 + “输入绑定与执行依赖”可单独增删依赖、恢复默认绑定。恢复默认由 Catalog/Validator 解释， + 不保证一定连上业务输入；不完整草稿仍可编辑，运行和保存要求校验成功。 +- 校验诊断的修复按钮打开页内候选审阅，展开详情检查改动，选择应用或取消。草稿或工具变化会 + 使旧候选失效；一次应用只产生一个撤销步骤。 - 自由字符串支持多行编辑；原样应用保留换行、空白、已有空串和可选字段的缺省状态。 修改后保存实际字符串，清空已有文本会保存 `""`。删除字段恢复缺省、精确设置转义字符， 或在原本缺省且默认为空时显式设置 `""`，使用已有 JSON 页编辑。 @@ -128,6 +133,12 @@ C++ 查看工具，展示更多声明信息： ./build/alg_pipeline_tool plan configs/pipeline_doc_qa_default.json ``` +可将版本化编辑请求交给 `alg_pipeline_tool edit --stdin`，返回候选 Pipeline、变更和校验报告； +单次最多 128 个动作、4 MiB，`require_valid: true` 要求最终候选合法。具体请求见 +[RFC-0057 的操作契约](../../doc/rfcs/0057-pipeline-composition-experience.md#41-所有者与接口)。 +手写 ports 后可先运行 `alg_pipeline_tool fix-deps pipeline.json` 预览确定的依赖补充，确认后 +加 `--in-place` 写回；歧义、环路、其他校验错误或文件写入失败均不覆盖原文件。 + 编排或修改 Pipeline 时，应先查询 Catalog 与节点 Definition，再执行 validate 和 plan。完整开发流程参见项目的 `pipeline-composer` skill 与[开发者指南](../../doc/developer_guide.md)。 ### 校验工具选择 @@ -164,15 +175,22 @@ Pipeline JSON 描述算法连线；`.conf` 描述部署路径和输出容量；P `demo/fixtures/...` 的测试方案填 `.`。模型路径按当前 Pipeline 重建，避免旧 Profile 的路径覆盖刚选择的权重。运行与已保存方案各自的展开区显示原生部署解析结果，已保存方案同时提供完整命令。 -本次服务会话通过“另存为可运行方案”创建的文件，后续点击“保存”会同步 JSON 和 `.conf`， -复用创建时选择的模型目录、Profile 和容量；两份文件都会检查修改冲突，预检失败不会写入。 +运行页的“检查运行条件”独立调用原生部署解析，不加载模型、不执行 Demo；摘要区分配置检查与 +文件存在性，不能代替实际加载和效果验收。方案、部署关联或运行设置改变后，旧摘要标为过期。 + +本次服务会话创建的配套文件，以及运行页“关联部署配置”明确关联的已有文件，后续点击“保存” +会同步 JSON 和 `.conf`。关联本身不写文件，并校验该 conf 确实指向当前保存的 Pipeline。 +预检、运行和保存共用候选配置，Profile 只提供数据集及运行选项，不替换关联文件的模型覆盖和 +输出池。模型表单中换资产只改对应覆盖；原始 JSON 改动已有部署覆盖的路径时，在运行页明确 +选择“保留当前部署覆盖”或“将新路径作为所选模型目录下的资产”。未明确选择时服务拒绝运行和保存。 +两份文件都会检查修改冲突，预检失败不会写入。 更新后的运行命令和解析结果会一起刷新。详情面板的“保存目标”来自服务端，打开或 保存后列明本次普通保存会写入的文件；按钮悬停也可查看。保存成功显示实际文件名， 保存或运行遇到 Validator 错误时展示可定位诊断；文件冲突以持续可见的提示展示。 其他方案的普通“保存”/“另存”只写 JSON。服务重启后也不会自动接管已有 `.conf`；若其 -`model_paths` 会覆盖本次模型路径或 ID 修改，保存会明确提示核对部署文件。此时在编辑器中 -同步修改两份文件并用 `resolve-conf` 检查，或从 Studio 另存可运行副本。 +`model_paths` 会覆盖本次模型路径或 ID 修改，保存会明确提示核对部署文件。可在运行页明确 +关联该文件后继续成套编辑，也可在外部同步并用 `resolve-conf` 检查，或另存可运行副本。 若手动使用只写 JSON 的路径，需自行配套 `.conf`： @@ -208,7 +226,7 @@ CLI 的 `--config` 覆盖 Profile 原配置，因此不需要新增 Profile。 命令必须在当前 Pipeline 的节点 Definition 中声明。Profile 可设置 `control_cmd`，CLI 优先。详见[第一个 Control](../../doc/dev_guide/first_control.md)。 -Studio 为草稿生成项目内的临时 JSON 和 `.conf`，复用 Profile 输出池容量，按“模型目录” +Studio 为草稿生成项目内的临时 JSON 和 `.conf`。未关联部署时复用 Profile 输出池容量，按“模型目录” 和当前 Pipeline 重建模型路径，并使用原生解析器预检；运行结束清理临时文件。草稿运行 使用配置初值，Control 练习通过 CLI 显式下发。解析成功说明部署配置可接受，不代表模型 已加载；日志和样本结果用于确认实际执行。 diff --git a/tools/pipeline_studio/server.py b/tools/pipeline_studio/server.py index 31d9e7e6..b0cf16c7 100755 --- a/tools/pipeline_studio/server.py +++ b/tools/pipeline_studio/server.py @@ -48,6 +48,7 @@ MANAGED_NAME = re.compile(r"^pipeline_[a-z0-9_]+\.json$") MAX_LOG_BYTES = 2 * 1024 * 1024 MAX_DOCUMENT_BYTES = 4 * 1024 * 1024 +_AUTHORING_UNSET = object() class StudioError(RuntimeError): @@ -249,10 +250,14 @@ def pipelines(self) -> dict[str, Any]: "revision": revision_for(raw), } ) - return json_result(True, pipelines=items) + confs = [p.name for p in sorted(self.config_root.glob("*.conf")) if p.is_file()] + return json_result(True, pipelines=items, deployments=confs) def save_targets(self, path: Path) -> list[str]: - return [path.name, path.with_suffix(".conf").name] if path.name in self.generated_solutions else [path.name] + if path.name in self.generated_solutions: + conf_name = self.generated_solutions[path.name].get("conf_name", path.with_suffix(".conf").name) + return [path.name, conf_name] + return [path.name] def open_pipeline(self, requested: str) -> dict[str, Any]: path = self.managed_path(requested, must_exist=True) @@ -267,8 +272,17 @@ def open_pipeline(self, requested: str) -> dict[str, Any]: revision=revision_for(raw), save_targets=self.save_targets(path), pipeline=pipeline, + deployment=self.deployment_info(path.name), ) + def deployment_info(self, filename: str) -> dict[str, Any] | None: + managed = self.generated_solutions.get(filename) + if not managed: + return None + return {"conf_name": managed["conf_path"].name, + "conf_revision": managed["conf_revision"], + "model_root": managed["model_root"]} + def invoke_tool( self, command: list[str], pipeline: Any | None = None ) -> dict[str, Any]: @@ -356,6 +370,8 @@ def preview_fix( "PATCH_APPLICATION_FAILED", f"补丁应用失败: {error}" ) from error report = self.validate(patched, explain=True) + if current_fp != self.get_tool_fingerprint(): + raise StudioError("TOOL_OUTDATED", "修复预览期间工具已更新，请重新校验", 409) return json_result( True, patched=patched, @@ -366,6 +382,46 @@ def preview_fix( tool_fingerprint=current_fp, ) + def preview_authoring( + self, + pipeline: Any, + operation: Any = _AUTHORING_UNSET, + operations: Any = _AUTHORING_UNSET, + require_valid: bool = False, + expected_revision: str | None = None, + tool_fingerprint: str | None = None, + ) -> dict[str, Any]: + if not expected_revision: + raise StudioError("REVISION_CONFLICT", "必须指定预期修订版本，草稿可能已变更", 409) + raw = json.dumps(pipeline, sort_keys=True).encode("utf-8") + if revision_for(raw) != expected_revision: + raise StudioError("REVISION_CONFLICT", "草稿已变更，操作失效", 409) + current_fp = self.get_tool_fingerprint() + if not tool_fingerprint or current_fp != tool_fingerprint: + raise StudioError("TOOL_OUTDATED", "操作期间工具已更新，请重新加载", 409) + + req: dict[str, Any] = { + "schema_version": 1, + "pipeline": pipeline, + "require_valid": require_valid, + } + if operations is not _AUTHORING_UNSET: + req["operations"] = operations + if operation is not _AUTHORING_UNSET: + req["operation"] = operation + + res = self.invoke_tool(["edit", "--stdin"], req) + if self.get_tool_fingerprint() != current_fp: + raise StudioError("TOOL_OUTDATED", "操作期间工具已更新，请重新加载", 409) + if isinstance(res, dict): + res["tool_fingerprint"] = current_fp + if res.get("pipeline"): + res["revision"] = revision_for( + json.dumps(res["pipeline"], sort_keys=True).encode("utf-8") + ) + return res + + def init_pipeline( self, biz: str, profile: str = "", empty: bool = False ) -> dict[str, Any]: @@ -382,13 +438,16 @@ def save_pipeline( pipeline: Any, expected_revision: str | None, save_as: bool = False, + profile_name: str = "", + model_root: str = "models", + model_path_actions: Any = None, ) -> dict[str, Any]: report = self.validate(pipeline) if not report.get("ok"): raise StudioError("VALIDATION_FAILED", json.dumps(report, ensure_ascii=False)) path = self.managed_path(requested) if not save_as and path.name in self.generated_solutions: - return self.update_solution(path, pipeline, expected_revision) + return self.update_solution(path, pipeline, expected_revision, profile_name, model_root, model_path_actions) if path.exists() and not save_as: current = revision_for(path.read_bytes()) if not expected_revision or current != expected_revision: @@ -420,6 +479,7 @@ def save_pipeline( revision=revision_for(encoded), save_targets=self.save_targets(path), pipeline=pipeline, + deployment=self.deployment_info(path.name), ) def check_unmanaged_deployment(self, path: Path, pipeline: Any) -> None: @@ -450,10 +510,12 @@ def model_paths(document: Any) -> dict: 409, ) - def update_solution(self, path: Path, pipeline: Any, expected_revision: str | None) -> dict[str, Any]: + def update_solution(self, path: Path, pipeline: Any, expected_revision: str | None, + profile_name: str = "", model_root: str = "models", + model_path_actions: Any = None) -> dict[str, Any]: with self.solution_lock: managed = self.generated_solutions[path.name] - conf_path = path.with_suffix(".conf") + conf_path = managed.get("conf_path") or path.with_suffix(".conf") def check_revisions() -> tuple[bytes, bytes]: if path.is_symlink() or conf_path.is_symlink() or not path.is_file() or not conf_path.is_file(): @@ -464,9 +526,10 @@ def check_revisions() -> tuple[bytes, bytes]: return raw, conf_raw old_json, _ = check_revisions() - profile = managed["profile"] + profile, conf = self.deployment_candidate( + pipeline, profile_name, model_root, path.name, + model_path_actions=model_path_actions) encoded = (json.dumps(pipeline, ensure_ascii=False, indent=2) + "\n").encode() - conf = self.run_conf(pipeline, managed["outputs"], path, managed["model_root"]) conf_encoded = (json.dumps(conf, ensure_ascii=False, indent=2) + "\n").encode() staging = Path(tempfile.mkdtemp(prefix=".studio-save-", dir=self.config_root)) preserve_backup = False @@ -475,8 +538,11 @@ def check_revisions() -> tuple[bytes, bytes]: staged_conf = staging / "pipeline.conf" backup_json = staging / "previous.json" staged_json.write_bytes(encoded) - staged_conf.write_text(json.dumps(self.run_conf(pipeline, managed["outputs"], staged_json, managed["model_root"]))) + staged_conf_data = copy.deepcopy(conf) + staged_conf_data["data"]["pipe_path"] = str(staged_json.relative_to(PROJECT_ROOT)) + staged_conf.write_text(json.dumps(staged_conf_data, ensure_ascii=False, indent=2)) configuration = self.resolve_run_conf(staged_conf, profile) + # Native validation used the staged JSON; installed paths have # the same model mappings and normalized node configuration. configuration["conf_path"] = str(conf_path) @@ -495,6 +561,9 @@ def check_revisions() -> tuple[bytes, bytes]: raise StudioError("SAVE_ROLLBACK_FAILED", f"保存失败，旧 JSON 备份保留在 {backup_json}: {rollback_error}", 500) from rollback_error raise managed["conf_revision"] = revision_for(conf_encoded) + managed["pipeline_revision"] = revision_for(encoded) + managed["profile"] = profile + managed["model_root"] = model_root return self.solution_result(path, pipeline, conf, encoded, profile, managed["model_root"], configuration) except StudioError: raise @@ -543,6 +612,71 @@ def run_conf(self, pipeline: Any, outputs: Any, pipe_path: Path, model_root: str except (ValueError, TypeError, KeyError) as error: raise StudioError("INVALID_DEPLOYMENT_PATH", str(error)) from error + def deployment_candidate( + self, pipeline: Any, profile_name: str = "", model_root: str = "models", + filename: str = "", conf_name: str = "", model_path_actions: Any = None, + ) -> tuple[dict[str, Any], dict[str, Any]]: + """Build one deployment snapshot shared by preview, run and save.""" + models = {m["model_id"]: m for m in pipeline.get("models", [])} + actions = {} if model_path_actions is None else model_path_actions + if not isinstance(actions, dict): + raise StudioError("INVALID_MODEL_PATH_ACTION", "模型路径选择必须是对象") + for mid, choice in actions.items(): + if (mid not in models or not isinstance(choice, dict) + or set(choice) != {"path", "action"} + or not isinstance(choice["path"], str) + or choice["path"] != models[mid].get("model_path") + or choice["action"] not in ("select_asset", "preserve_override")): + raise StudioError("INVALID_MODEL_PATH_ACTION", f"模型 {mid} 的路径选择无效或已过期") + if conf_name: + requested_conf = self.managed_conf_path(conf_name, must_exist=True) + if not filename: + matches = [name for name, info in self.generated_solutions.items() + if info["conf_path"] == requested_conf] + if len(matches) != 1: + raise StudioError("DEPLOYMENT_NOT_ASSOCIATED", "请先明确关联当前方案与部署配置") + filename = matches[0] + managed = None + path = None + if filename: + path = self.managed_path(filename) + managed = self.generated_solutions.get(path.name) + if conf_name and (not managed or managed["conf_path"] != requested_conf): + raise StudioError("DEPLOYMENT_NOT_ASSOCIATED", "部署配置未关联到当前方案") + if not managed: + profile, outputs = self.profile_inputs(pipeline, profile_name) + return profile, self.run_conf(pipeline, outputs, path or PROJECT_ROOT / "build/pipeline.json", model_root) + conf_path = managed["conf_path"] + if (path.is_symlink() or conf_path.is_symlink() + or not path.is_file() or not conf_path.is_file()): + raise StudioError("REVISION_CONFLICT", "配套 JSON 或 .conf 已被替换或删除，请重新关联", 409) + raw, conf_raw = path.read_bytes(), conf_path.read_bytes() + if (revision_for(raw) != managed["pipeline_revision"] + or revision_for(conf_raw) != managed["conf_revision"]): + raise StudioError("REVISION_CONFLICT", "配套 JSON 或 .conf 已被其他编辑器修改，请重新关联", 409) + original = json.loads(raw) + if original.get("biz_name") != pipeline.get("biz_name"): + raise StudioError("DEPLOYMENT_MISMATCH", "已关联方案不能改变业务契约，请另存方案") + profile = self.profile_inputs(pipeline, profile_name)[0] if profile_name else copy.deepcopy(managed["profile"]) + conf = json.loads(conf_raw) + overrides = copy.deepcopy(conf["data"].get("model_paths", {})) + old_models = {m["model_id"]: m for m in original.get("models", [])} + for mid in list(overrides): + if mid not in models: + del overrides[mid] + for mid, model in models.items(): + choice = actions.get(mid) + if choice and choice["action"] == "select_asset": + selected = self.run_conf({"models": [model]}, {}, path, model_root) + overrides[mid] = selected["data"]["model_paths"][mid] + elif (not choice and mid in overrides and mid in old_models + and model.get("model_path") != old_models[mid].get("model_path")): + raise StudioError("DEPLOYMENT_PATH_INTENT_REQUIRED", + f"模型 {mid} 的路径已改变，请明确选择保留部署覆盖或采用新资产路径", 409) + if "model_paths" in conf["data"] or overrides: + conf["data"]["model_paths"] = overrides + return profile, conf + def resolve_run_conf(self, conf_path: Path, profile: dict[str, Any]) -> dict[str, Any]: depth = max(int(profile.get("batch_size", 1)), int(profile.get("depth", 1))) report = self.invoke_tool(["resolve-conf", str(conf_path.relative_to(PROJECT_ROOT)), "--root", str(PROJECT_ROOT), "--depth", str(depth)]) @@ -571,8 +705,8 @@ def save_solution(self, requested: str, pipeline: Any, profile_name: str, model_ raise StudioError("SYMLINK_REJECTED", "拒绝写入符号链接方案") if target.exists(): raise StudioError("FILE_EXISTS", f"另存目标已存在：{target.name}", 409) - profile, outputs = self.profile_inputs(pipeline, profile_name) - conf = self.run_conf(pipeline, outputs, path, model_root) + profile, conf = self.deployment_candidate(pipeline, profile_name, model_root, path.name) + outputs = conf["data"]["outputs"] encoded = (json.dumps(pipeline, ensure_ascii=False, indent=2) + "\n").encode() conf_encoded = (json.dumps(conf, ensure_ascii=False, indent=2) + "\n").encode() created = [] @@ -596,28 +730,243 @@ def save_solution(self, requested: str, pipeline: Any, profile_name: str, model_ raise StudioError("SAVE_FAILED", str(error), 500) from error self.generated_solutions[path.name] = { "profile": profile, "outputs": outputs, "model_root": model_root, + "conf_path": conf_path, "conf_name": conf_path.name, + "pipeline_revision": revision_for(encoded), "conf_revision": revision_for(conf_encoded), } return self.solution_result(path, pipeline, conf, encoded, profile, model_root, configuration) def solution_result(self, path: Path, pipeline: Any, conf: Any, encoded: bytes, profile: dict[str, Any], model_root: str, configuration: Any) -> dict[str, Any]: - conf_path = path.with_suffix(".conf") - command = self.demo_command(profile, conf_path.relative_to(PROJECT_ROOT), PROJECT_ROOT / "output" / path.stem) + conf_path = self.generated_solutions.get(path.name, {}).get("conf_path") or path.with_suffix(".conf") + command_str = "" + if profile and profile.get("dataset"): + command = self.demo_command(profile, conf_path.relative_to(PROJECT_ROOT), PROJECT_ROOT / "output" / path.stem) + command_str = f"cd {shlex.quote(str(PROJECT_ROOT))} && {shlex.join(command)}" return json_result( True, filename=path.name, conf_filename=conf_path.name, revision=revision_for(encoded), pipeline=pipeline, conf=conf, model_root=model_root, configuration=configuration, save_targets=self.save_targets(path), - command=f"cd {shlex.quote(str(PROJECT_ROOT))} && {shlex.join(command)}", + command=command_str, + deployment=self.deployment_info(path.name), ) - def start_run(self, pipeline: Any, profile_name: str, model_root: str = "models") -> dict[str, Any]: + def managed_conf_path(self, requested: str, must_exist: bool = False) -> Path: + path = Path(requested) + if path.is_absolute() or len(path.parts) not in (1, 2): + raise StudioError("INVALID_DEPLOYMENT_PATH", "只允许 configs 下的部署配置") + if len(path.parts) == 2 and path.parts[0] != "configs": + raise StudioError("INVALID_DEPLOYMENT_PATH", "路径必须位于 configs 目录") + filename = path.name + if not re.fullmatch(r"^pipeline_[a-z0-9_]+\.conf$", filename): + raise StudioError( + "INVALID_DEPLOYMENT_NAME", + "文件名必须匹配 pipeline_[a-z0-9_]+.conf", + ) + candidate = self.config_root / filename + if candidate.exists() or candidate.is_symlink(): + if candidate.is_symlink(): + raise StudioError("SYMLINK_REJECTED", "拒绝读写符号链接部署配置") + if candidate.resolve().parent != self.config_root: + raise StudioError("PATH_ESCAPE", "部署配置路径逃逸 configs 目录") + elif must_exist: + raise StudioError("DEPLOYMENT_NOT_FOUND", filename, 404) + return candidate + + def associate_deployment( + self, pipeline_name: str, conf_name: str, model_root: str = "models", + profile_name: str = "", + ) -> dict[str, Any]: + pipe_path = self.managed_path(pipeline_name, must_exist=True) + conf_path = self.managed_conf_path(conf_name, must_exist=True) + + pipe_raw = pipe_path.read_bytes() + conf_raw = conf_path.read_bytes() + try: + pipeline = json.loads(pipe_raw) + conf = json.loads(conf_raw) + except json.JSONDecodeError as error: + raise StudioError("INVALID_JSON", str(error)) from error + + if not isinstance(conf, dict) or "data" not in conf or not isinstance(conf["data"], dict): + raise StudioError("INVALID_DEPLOYMENT_CONFIG", "部署配置缺少 data 节点") + + pipe_ref = conf["data"].get("pipe_path") + if not isinstance(pipe_ref, str): + raise StudioError("INVALID_DEPLOYMENT_CONFIG", "部署配置缺少 data.pipe_path") + + report = self.invoke_tool([ + "resolve-conf", + str(conf_path.relative_to(PROJECT_ROOT)), + "--root", + str(PROJECT_ROOT), + ]) + if not report.get("ok"): + raise StudioError("DEPLOYMENT_VALIDATION_FAILED", json.dumps(report, ensure_ascii=False)) + + configuration = report.get("configuration", {}) + resolved_pipe = Path(configuration.get("pipeline_path", "")).resolve() + if resolved_pipe != pipe_path.resolve(): + raise StudioError( + "DEPLOYMENT_MISMATCH", + f"部署配置指向 {resolved_pipe.name}，与当前方案 {pipe_path.name} 不符", + 400, + ) + + if profile_name: + prof, _ = self.profile_inputs(pipeline, profile_name) + else: + prof = None + for name in read_json(PROFILE_FILE).get("profiles", {}): + try: + prof, _ = self.profile_inputs(pipeline, name) + break + except StudioError as error: + if error.code != "PROFILE_MISMATCH": + raise + if prof is None: + raise StudioError("PROFILE_MISMATCH", "没有与该业务契约匹配的运行 Profile") + configuration = self.resolve_run_conf(conf_path, prof) + if pipe_path.read_bytes() != pipe_raw or conf_path.read_bytes() != conf_raw: + raise StudioError("REVISION_CONFLICT", "关联期间文件已改变，请重新关联", 409) + + with self.solution_lock: + self.generated_solutions[pipe_path.name] = { + "conf_path": conf_path, + "conf_name": conf_path.name, + "conf_revision": revision_for(conf_raw), + "pipeline_revision": revision_for(pipe_raw), + "outputs": conf["data"].get("outputs", {}), + "model_root": model_root, + "profile": prof, + "is_associated": True, + } + + return json_result( + True, + pipeline_name=pipe_path.name, + conf_name=conf_path.name, + conf_revision=revision_for(conf_raw), + save_targets=self.save_targets(pipe_path), + configuration=configuration, + deployment=self.deployment_info(pipe_path.name), + ) + + def preflight( + self, + pipeline: Any, + profile_name: str = "", + conf_name: str = "", + model_root: str = "models", + filename: str = "", + model_path_actions: Any = None, + ) -> dict[str, Any]: + report = self.validate(pipeline, explain=True) + if not report.get("ok"): + return json_result( + False, + summary={ + "status": "validation_failed", + "biz_name": pipeline.get("biz_name", "") if isinstance(pipeline, dict) else "", + "project_root": str(PROJECT_ROOT), + "next_step": "请先修复 Pipeline 校验错误", + }, + validation=report, + ) + + staging_dir = PROJECT_ROOT / "build" if (PROJECT_ROOT / "build").is_dir() else PROJECT_ROOT + staging = Path(tempfile.mkdtemp(prefix=".studio-preflight-", dir=staging_dir)) + try: + staged_pipe = staging / "pipeline.json" + staged_pipe.write_text(json.dumps(pipeline, ensure_ascii=False, indent=2)) + staged_conf = staging / "pipeline.conf" + + if not profile_name and not conf_name and filename not in self.generated_solutions: + return json_result( + True, + summary={ + "status": "no_deployment_conf", + "biz_name": pipeline.get("biz_name", ""), + "project_root": str(PROJECT_ROOT), + "pipeline_snapshot": { + "node_count": len(pipeline.get("pipeline", [])), + "model_count": len(pipeline.get("models", [])), + }, + "next_step": "Pipeline 校验通过。请关联部署配置或选择 Profile 进行部署预检。", + }, + validation=report, + ) + + profile_obj, conf_data = self.deployment_candidate( + pipeline, profile_name, model_root, filename, conf_name, model_path_actions) + conf_data["data"]["pipe_path"] = str(staged_pipe.relative_to(PROJECT_ROOT)) + staged_conf.write_text(json.dumps(conf_data, ensure_ascii=False, indent=2)) + configuration = self.resolve_run_conf(staged_conf, profile_obj) + + assets_status = [] + all_assets_ready = True + for mpath_info in configuration.get("model_paths", []): + res_path = Path(mpath_info.get("resolved", "")) + status = "missing" + if res_path.is_file(): + status = "exists_unverified" + else: + all_assets_ready = False + assets_status.append({ + "model_id": mpath_info.get("model_id", ""), + "resolved_path": str(res_path), + "source": mpath_info.get("source", ""), + "status": status, + }) + + tools_ready = PIPELINE_TOOL.is_file() and DEMO_BINARY.is_file() + overall_status = "ready" if (all_assets_ready and tools_ready) else "attention_required" + + next_steps = [] + if not all_assets_ready: + next_steps.append("部分模型资产文件缺失，请检查路径或准备模型。") + if not DEMO_BINARY.is_file(): + next_steps.append("Demo 可执行文件未就绪，请构建 build/alg_demo。") + if all_assets_ready and tools_ready: + next_steps.append("配置与资源预检通过，可执行运行或成套保存。") + + summary = { + "status": overall_status, + "biz_name": pipeline.get("biz_name", ""), + "project_root": str(PROJECT_ROOT), + "pipeline_snapshot": { + "node_count": len(pipeline.get("pipeline", [])), + "model_count": len(pipeline.get("models", [])), + }, + "conf_path": (self.generated_solutions[filename]["conf_path"].name + if filename in self.generated_solutions else conf_name or (profile_name and f"profile:{profile_name}") or ""), + "model_paths": configuration.get("model_paths", []), + "output_pools": configuration.get("output_pools", {}), + "effective_pipeline": configuration.get("effective_pipeline", {}), + "assets": assets_status, + "tools": { + "alg_pipeline_tool": PIPELINE_TOOL.is_file(), + "alg_demo": DEMO_BINARY.is_file(), + }, + "next_step": " ".join(next_steps) if next_steps else "预检完成", + } + return json_result( + True, + summary=summary, + configuration=configuration, + validation=report, + ) + finally: + shutil.rmtree(staging, ignore_errors=True) + + + def start_run(self, pipeline: Any, profile_name: str, model_root: str = "models", + filename: str = "", conf_name: str = "", + model_path_actions: Any = None) -> dict[str, Any]: report = self.validate(pipeline) if not report.get("ok"): raise StudioError("VALIDATION_FAILED", json.dumps(report, ensure_ascii=False)) - profile, outputs = self.profile_inputs(pipeline, profile_name) - # Check the selected paths before creating a job. The worker only changes - # pipe_path to its own temporary document under the same deployment root. - conf = self.run_conf(pipeline, outputs, "build/pipeline.json", model_root) + profile, conf = self.deployment_candidate( + pipeline, profile_name, model_root, filename, conf_name, model_path_actions) with self.job_lock: if any(job["status"] in ("queued", "running") for job in self.jobs.values()): raise StudioError("RUN_BUSY", "同一工作台最多运行一个任务", 409) @@ -811,7 +1160,41 @@ def _dispatch(self, method: str) -> None: body.get("revision") or body.get("expected_revision"), body.get("tool_fingerprint"), ) + elif method == "POST" and path == "/api/v1/authoring/preview": + allowed = {"pipeline", "operation", "operations", "require_valid", + "revision", "expected_revision", "tool_fingerprint"} + unknown = set(body) - allowed + if unknown: + raise StudioError("INVALID_AUTHORING_REQUEST", "未知编排请求字段：" + ", ".join(sorted(unknown))) + if ("revision" in body and "expected_revision" in body + and body["revision"] != body["expected_revision"]): + raise StudioError("REVISION_CONFLICT", "请求包含不同的预期修订版本", 409) + payload = service.preview_authoring( + body.get("pipeline"), + operation=body.get("operation", _AUTHORING_UNSET), + operations=body.get("operations", _AUTHORING_UNSET), + require_valid=body.get("require_valid", False), + expected_revision=body.get("revision") or body.get("expected_revision"), + tool_fingerprint=body.get("tool_fingerprint"), + ) + elif method == "POST" and path == "/api/v1/preflight": + payload = service.preflight( + body.get("pipeline"), + profile_name=body.get("profile", ""), + conf_name=body.get("conf_path", "") or body.get("conf_name", ""), + model_root=body.get("model_root", "models"), + filename=body.get("filename", ""), + model_path_actions=body.get("model_path_actions"), + ) + elif method == "POST" and path == "/api/v1/deployment/associate": + payload = service.associate_deployment( + body.get("pipeline_name", "") or body.get("filename", ""), + body.get("conf_name", "") or body.get("conf_path", ""), + model_root=body.get("model_root", "models"), + profile_name=body.get("profile", ""), + ) elif method == "POST" and path == "/api/v1/init": + payload = service.init_pipeline( body.get("biz", ""), body.get("profile", ""), body.get("empty", False) ) @@ -829,9 +1212,15 @@ def _dispatch(self, method: str) -> None: body.get("pipeline"), body.get("revision"), save_as=False, + profile_name=body.get("profile", ""), + model_root=body.get("model_root", "models"), + model_path_actions=body.get("model_path_actions"), ) elif method == "POST" and path == "/api/v1/runs": - payload = service.start_run(body.get("pipeline"), body.get("profile", ""), body.get("model_root", "models")) + payload = service.start_run( + body.get("pipeline"), body.get("profile", ""), body.get("model_root", "models"), + filename=body.get("filename", ""), conf_name=body.get("conf_name", ""), + model_path_actions=body.get("model_path_actions")) elif method == "DELETE" and path.startswith("/api/v1/runs/"): payload = service.cancel_run(path.rsplit("/", 1)[-1]) else: diff --git a/tools/pipeline_studio/web/app.js b/tools/pipeline_studio/web/app.js index c543ca21..a7c80c12 100644 --- a/tools/pipeline_studio/web/app.js +++ b/tools/pipeline_studio/web/app.js @@ -1,13 +1,17 @@ import { api, initialPipeline, write } from "./api.js"; import { GraphView } from "./graph.js"; import { createHistory, createDrafts, appendDiagnostic, appendConfigField, readConfigFields, readFormBuffer, restoreFormBuffer } from "./editor.js"; -import { compatibleModels, createLatestRequestGate, modelBoundNodeIds, graphDocument, connectPorts, disconnectPorts, removeNode, compatibleBackends, modelAvailability, assertBrowsablePipeline, readPipelineFile, upsertModel, removeModel } from "./workbench.js"; +import { compatibleModels, createLatestRequestGate, modelBoundNodeIds, graphDocument, compatibleBackends, modelAvailability, assertBrowsablePipeline, readPipelineFile, upsertModel, removeModel } from "./workbench.js"; import { captureRun, runIsCurrent, runSummary, renderSamples } from "./workflow.js"; const $ = selector => document.querySelector(selector); const state = { pipeline: null, + authoringBusy: false, + deployment: null, + modelPathActions: {}, + preflight: null, filename: "", sourceName: "", revision: "", @@ -35,43 +39,53 @@ const history = createHistory(); const drafts = createDrafts(); let documentRequest = 0; let pendingAction = null; +let pendingFixReview = null; +let preflightRequest = 0; -function snapshot() { return { pipeline: state.pipeline, selected: state.selected }; } +function snapshot() { return { pipeline: state.pipeline, selected: state.selected, modelPathActions: state.modelPathActions }; } function updateEditorStatus() { const pending = drafts.pending; const badge = $("#dirtyBadge"); - badge.textContent = state.loading ? "加载中…" : !state.pipeline ? "未打开" : pending ? "有未应用修改" : state.dirty ? "未保存" : state.sourceName ? "已导入" : "已保存"; + badge.textContent = (state.loading || state.authoringBusy) ? "加载中…" : !state.pipeline ? "未打开" : pending ? "有未应用修改" : state.dirty ? "未保存" : state.sourceName ? "已导入" : "已保存"; badge.classList.toggle("dirty", pending || state.dirty); - $("#undoButton").disabled = state.loading || pending || !history.canUndo; - $("#redoButton").disabled = state.loading || pending || !history.canRedo; - $("#saveButton").disabled = state.loading || state.saving || !state.pipeline; - $("#saveAsButton").disabled = state.loading || state.saving || !state.pipeline; - $("#saveSolutionButton").disabled = state.loading || state.saving || !state.pipeline; - $("#openButton").disabled = state.loading; - $("#browseButton").disabled = state.loading; - $("#newButton").disabled = state.loading || !state.catalogReady; - $("#newEntryButton").disabled = state.loading; - $("#quickValidateButton").disabled = state.loading || !state.pipeline || !state.catalogReady; - $("#openRunButton").disabled = state.loading || !state.pipeline; - $("#validateButton").disabled = state.loading || !state.pipeline || !state.catalogReady; + $("#undoButton").disabled = (state.loading || state.authoringBusy) || pending || !history.canUndo; + $("#redoButton").disabled = (state.loading || state.authoringBusy) || pending || !history.canRedo; + $("#saveButton").disabled = (state.loading || state.authoringBusy) || state.saving || !state.pipeline; + $("#saveAsButton").disabled = (state.loading || state.authoringBusy) || state.saving || !state.pipeline; + $("#saveSolutionButton").disabled = (state.loading || state.authoringBusy) || state.saving || !state.pipeline; + $("#openButton").disabled = (state.loading || state.authoringBusy); + $("#browseButton").disabled = (state.loading || state.authoringBusy); + $("#newButton").disabled = (state.loading || state.authoringBusy) || !state.catalogReady; + $("#newEntryButton").disabled = (state.loading || state.authoringBusy); + $("#quickValidateButton").disabled = (state.loading || state.authoringBusy) || !state.pipeline || !state.catalogReady; + $("#quickValidateButton").textContent = pending ? "应用修改并校验" : "校验"; + $("#openRunButton").disabled = (state.loading || state.authoringBusy) || !state.pipeline; + $("#openRunButton").textContent = pending ? "应用修改并运行" : "运行"; + $("#validateButton").disabled = (state.loading || state.authoringBusy) || !state.pipeline || !state.catalogReady; + $("#validateButton").textContent = pending ? "应用修改并校验" : "校验方案"; + $("#runButton").textContent = pending ? "应用修改并运行" : "运行草稿"; + const preflightBtn = $("#preflightButton"); + if (preflightBtn) preflightBtn.disabled = (state.loading || state.authoringBusy) || !state.pipeline; + const assocBtn = $("#associateDeploymentButton"); + if (assocBtn) assocBtn.disabled = (state.loading || state.authoringBusy) || !state.pipeline || !state.filename; $("#browseHint").hidden = state.editing; $("#saveScope").textContent = state.pipeline ? state.filename ? `保存目标：${state.saveTargets.join(" + ")}（configs/）` : "保存将另存到 configs/；导入源文件保持原样。" : ""; $("#saveButton").title = $("#saveScope").textContent; renderRunContext(); - $("#rawJson").readOnly = !state.editing || state.loading || drafts.pendingExcept("json").length > 0; + $("#rawJson").readOnly = !state.editing || (state.loading || state.authoringBusy) || drafts.pendingExcept("json").length > 0; for (const [kind, form] of [["node", "#nodeForm"], ["model", "#modelForm"]]) { for (const control of $(form).querySelectorAll("input, select, textarea, button")) { - control.disabled = !state.editing || state.loading || !state.catalogReady || !state.pipeline || drafts.pendingExcept(kind).length > 0; + control.disabled = !state.editing || (state.loading || state.authoringBusy) || !state.catalogReady || !state.pipeline || drafts.pendingExcept(kind).length > 0; } } const modelDefinition = state.catalog.models?.find(item => item.model_type === $("#modelType").value); if (!modelAvailability(state.catalog.backends || [], modelDefinition).available) $("#applyModel").disabled = true; - $("#applyJson").disabled = !state.editing || state.loading || drafts.pendingExcept("json").length > 0; - $("#modelSelect").disabled = state.loading || !state.catalogReady || drafts.pending; - $("#newModel").disabled = !state.editing || state.loading || !state.catalogReady || drafts.pending; + $("#applyJson").disabled = !state.editing || (state.loading || state.authoringBusy) || drafts.pendingExcept("json").length > 0; + $("#modelSelect").disabled = (state.loading || state.authoringBusy) || !state.catalogReady || drafts.pending; + $("#newModel").disabled = !state.editing || (state.loading || state.authoringBusy) || !state.catalogReady || drafts.pending; for (const [kind, hint, discard] of [["json", "rawJsonHint", "discardJson"], ["node", "nodeDraftHint", "discardNode"], ["model", "modelDraftHint", "discardModel"]]) { $(`#${hint}`).hidden = !drafts.has(kind); $(`#${discard}`).disabled = !drafts.has(kind); @@ -79,7 +93,7 @@ function updateEditorStatus() { } function requireApplied(except = "", continuation = null, action = "继续操作") { - if (state.loading) { toast("方案加载中，请稍候"); return false; } + if (state.loading || state.authoringBusy) { toast("方案操作中，请稍候"); return false; } const pending = drafts.pendingExcept(except); if (!pending.length) return true; const labels = { json: "原始 JSON", node: "节点属性", model: "模型" }; @@ -160,6 +174,9 @@ function markPipelineChanged(record = true) { clearValidation("方案已修改，请重新校验"); $("#selectionReport").textContent = "方案已修改，资产与效果验收需要重新检查。"; state.selectedEdge = null; + pendingFixReview = null; + if ($("#fixReviewPanel")) $("#fixReviewPanel").hidden = true; + renderPreflightFreshness(); if (record) history.record(snapshot()); setDirty(JSON.stringify(state.pipeline) !== state.savedPipeline); } @@ -190,9 +207,7 @@ function selectNode(id) { const graph = new GraphView($("#graph"), { select: selectNode, connect: (source, sourcePort, target, targetPort) => { - if (!state.editing || !requireApplied()) return; - try { connectPorts(state.pipeline, state.catalog, source, sourcePort, target, targetPort); markPipelineChanged(); renderAll(); } - catch (error) { toast(error.message, true); } + applyAuthoring({ kind: "connect", source: { node_id: source, port: sourcePort }, target: { node_id: target, port: targetPort } }); }, selectEdge: edge => { if (state.loading) return; @@ -203,7 +218,7 @@ const graph = new GraphView($("#graph"), { }); function renderAll() { - graph.editable = state.editing && !state.loading && state.catalogReady; + graph.editable = state.editing && !state.loading && !state.authoringBusy && state.catalogReady; const nodes = effectiveNodes(); const modelIds = modelBoundNodeIds(nodes, state.catalog.nodes); const document = graphDocument(state.pipeline, state.catalog); @@ -216,6 +231,8 @@ function renderAll() { renderInspector(nodes.find(node => node.id === state.selected)); filterProfiles(); refreshModelList(); + renderPathIntent(); + renderPreflightFreshness(); updateEditorStatus(); } @@ -235,6 +252,7 @@ function renderInspector(node) { const choices = modelRef ? compatibleModels(state.pipeline.models, state.catalog.models, requiredCap).map(model => model.model_id) : null; appendConfigField(container, field, node.config || {}, choices); } + renderBindings(node, definition); if (drafts.has("node")) restoreFormBuffer($("#nodeForm"), drafts.get("node")); } @@ -273,17 +291,81 @@ function renderOperators() { } function addNode(definition) { - if (!state.pipeline) return toast("请先打开或新建方案", true); - if (!state.editing || !requireApplied()) return; - const base = definition.node_type.replace(/Node$/, "").replace(/([A-Z])/g, "_$1").toLowerCase().replace(/^_/, ""); - let index = 1, id = base; - const ids = new Set(state.pipeline.pipeline.map(node => node.id)); - while (ids.has(id)) id = `${base}_${index++}`; - const config = {}; - for (const field of definition.config_fields || []) if (field.default !== undefined) config[field.name] = field.default; - const outputs = Object.fromEntries((definition.outputs || []).map(port => [port.key, `${id}__${port.key}`])); - state.pipeline.pipeline.push({ id, node_type: definition.node_type, depends_on: [], ports: { inputs: {}, outputs }, config }); - state.selected = id; markPipelineChanged(); renderAll(); + return applyAuthoring({ kind: "add_node", node_type: definition.node_type }); +} + +async function applyAuthoring(operation, options = {}) { + if (!state.editing || !state.pipeline || !requireApplied(options.except || "")) return false; + const source = structuredClone(state.pipeline); + const version = state.pipelineVersion, documentVersion = state.documentVersion; + const buffer = JSON.stringify(drafts.get(options.except || "")); + const current = () => !state.loading && state.editing && version === state.pipelineVersion && + documentVersion === state.documentVersion && JSON.stringify(state.pipeline) === JSON.stringify(source) && + !drafts.pendingExcept(options.except || "").length && buffer === JSON.stringify(drafts.get(options.except || "")); + state.authoringBusy = true; updateEditorStatus(); graph.editable = false; + try { + const report = await write("/validate", "POST", { pipeline: source, explain: true }, true); + if (!current()) return false; + const result = await write("/authoring/preview", "POST", { + pipeline: source, operation, revision: report.revision, tool_fingerprint: report.tool_fingerprint, + }, true); + if (!current()) return false; + if (!result.ok || !result.pipeline) { + operationFeedback((result.diagnostics || []).map(d => d.message).join("\n") || "编排操作失败", true); + return false; + } + let validation = result.validation; + if (options.configure) { + options.configure(result.pipeline); + validation = await write("/validate", "POST", { pipeline: result.pipeline, explain: true }, true); + if (!current()) return false; + if (validation.tool_fingerprint !== result.tool_fingerprint) throw new Error("工具已更新，请重试操作"); + } + if (options.except) drafts.clear(options.except); + state.pipeline = result.pipeline; + if (options.selected !== undefined) state.selected = options.selected; + else if (operation.kind === "add_node") state.selected = result.changes?.find(c => c.action === "add_node")?.node_id || ""; + options.onCommit?.(); + markPipelineChanged(); + state.validationReport = { ...validation, revision: result.revision, tool_fingerprint: result.tool_fingerprint }; + if (options.configure) state.validationReport = validation; + showValidation(state.validationReport); + operationFeedback((result.changes || []).map(c => c.description).join("\n") || "编排操作完成"); + return true; + } catch (error) { + if (current()) operationFeedback(error.message, true); + return false; + } finally { state.authoringBusy = false; renderAll(); } +} + +function renderBindings(node, definition) { + const container = $("#nodeBindings"); + if (!container) return; + container.replaceChildren(); + for (const port of definition?.inputs || []) { + const row = document.createElement("div"); row.className = "row"; + const label = document.createElement("span"); label.textContent = port.key; + const button = document.createElement("button"); button.type = "button"; + button.textContent = "恢复默认绑定"; button.disabled = !state.editing || state.authoringBusy; + button.addEventListener("click", () => applyAuthoring({ kind: "reset_input_binding", target: { node_id: node.id, port: port.key } })); + row.append(label, button); container.append(row); + } + const label = document.createElement("p"); label.textContent = "执行依赖（断开数据线后保留，可单独删除）"; container.append(label); + for (const id of node.depends_on || []) { + const row = document.createElement("div"); row.className = "row"; + const text = document.createElement("span"); text.textContent = id; + const remove = document.createElement("button"); remove.type = "button"; remove.textContent = "删除依赖"; + remove.disabled = !state.editing || state.authoringBusy; + remove.addEventListener("click", () => applyAuthoring({ kind: "remove_dependency", node_id: node.id, depends_on_id: id })); + row.append(text, remove); container.append(row); + } + const select = document.createElement("select"); select.id = "dependencySource"; + select.add(new Option("选择前置节点", "")); + for (const other of effectiveNodes()) if (other.id !== node.id && !(node.depends_on || []).includes(other.id)) select.add(new Option(other.id, other.id)); + const add = document.createElement("button"); add.type = "button"; add.id = "addDependencyButton"; add.textContent = "添加依赖"; + add.disabled = !state.editing || state.authoringBusy; + add.addEventListener("click", () => { if (select.value) applyAuthoring({ kind: "add_dependency", node_id: node.id, depends_on_id: select.value }); }); + container.append(select, add); } async function refreshLists() { @@ -298,6 +380,13 @@ async function refreshLists() { if (state.pipeline) biz.value = state.pipeline.biz_name; const schemes = $("#pipelineSelect"); schemes.replaceChildren(new Option("选择方案", "")); for (const item of pipelines.pipelines) schemes.add(new Option(`${item.filename} · ${item.biz_name}`, item.filename)); + const confSelect = $("#associateConfSelect"); + if (confSelect) { + const prevConf = confSelect.value; + confSelect.replaceChildren(new Option("选择部署配置", "")); + for (const conf of pipelines.deployments || []) confSelect.add(new Option(conf, conf)); + if (prevConf && [...confSelect.options].some(o => o.value === prevConf)) confSelect.value = prevConf; + } if (state.pipeline) { await loadCatalog(state.pipeline.biz_name); } else { @@ -346,6 +435,8 @@ async function openDocument(load) { state.pipeline = result.pipeline; state.filename = result.imported ? "" : result.filename; state.sourceName = result.imported ? result.filename : ""; + state.deployment = result.deployment || null; state.modelPathActions = {}; + if (state.deployment) $("#runModelRoot").value = state.deployment.model_root; state.revision = result.revision; state.saveTargets = result.save_targets || (state.filename ? [state.filename] : []); state.selected = ""; $("#pipelineSelect").value = state.filename; drafts.clear(); state.selectedEdge = null; state.documentVersion += 1; @@ -369,6 +460,7 @@ async function createPipeline() { try { const result = await write("/init", "POST", { biz, profile, empty: !profile }); if (request !== documentRequest) return; + state.deployment = null; state.modelPathActions = {}; state.pipeline = result.pipeline; state.filename = ""; state.sourceName = ""; state.revision = ""; state.saveTargets = []; state.selected = ""; drafts.clear(); state.selectedEdge = null; state.documentVersion += 1; resetDocumentFeedback(); @@ -398,11 +490,13 @@ async function save(saveAs, runnable = false) { try { const result = await write(runnable ? "/solutions" : saveAs ? "/pipelines" : "/pipeline", saveAs ? "POST" : "PUT", { filename, pipeline, revision: state.revision, - ...(runnable ? { profile: $("#runProfile").value, model_root: $("#runModelRoot").value } : {}), + profile: $("#runProfile").value, model_root: $("#runModelRoot").value, model_path_actions: modelPathActions(), }); if (documentVersion !== state.documentVersion) return; state.filename = result.filename; state.sourceName = ""; state.revision = result.revision; state.saveTargets = result.save_targets || [result.filename]; + state.deployment = result.deployment || null; + if (JSON.stringify(state.pipeline) === JSON.stringify(pipeline)) state.modelPathActions = {}; state.savedPipeline = JSON.stringify(pipeline); savePositions(graph.positions); setDirty(JSON.stringify(state.pipeline) !== state.savedPipeline); @@ -485,19 +579,54 @@ async function handleApplyFix(fix) { }; describe(source, res.patched); const remaining = (res.report?.diagnostics || []).map(item => `${item.code} ${item.path}: ${item.message}`); - const msg = `${fix.title}\n${fix.effect}\n\n实际变更：\n${changes.join("\n\n")}\n\n${res.report?.ok ? "重新校验通过。" : `剩余诊断：\n${remaining.join("\n")}`}\n\n确认应用该修改？`; - if (window.confirm(msg) && isCurrent() && requireApplied()) { - state.pipeline = res.patched; - markPipelineChanged(); - operationFeedback(`已应用修复：${fix.title}（可通过撤销恢复）`); - await validate(); - renderAll(); - } + const reviewPanel = $("#fixReviewPanel"); + reviewPanel.replaceChildren(); reviewPanel.hidden = false; + const title = document.createElement("div"); title.className = "fix-review-title"; + title.textContent = `修复审阅：${fix.title}`; + const effect = document.createElement("p"); effect.textContent = fix.effect; + const status = document.createElement("p"); + status.textContent = res.report?.ok ? "候选方案校验通过；尚未运行" : `候选仍有 ${remaining.length} 项诊断，请审阅影响`; + const details = document.createElement("details"); + const summary = document.createElement("summary"); summary.textContent = `变更与诊断详情（${changes.length} 处）`; + const pre = document.createElement("pre"); pre.className = "code"; pre.textContent = [...changes, ...remaining].join("\n\n"); + details.append(summary, pre); + const apply = document.createElement("button"); apply.id = "applyReviewedFix"; apply.textContent = "应用此修复"; + apply.addEventListener("click", handleApplyReviewedFix); + const cancel = document.createElement("button"); cancel.id = "cancelReviewedFix"; cancel.textContent = "取消"; + cancel.addEventListener("click", () => { pendingFixReview = null; reviewPanel.hidden = true; }); + reviewPanel.append(title, effect, status, details, apply, cancel); + pendingFixReview = { fix, source, report, res, isCurrent }; + switchTab("validation"); + } catch (error) { operationFeedback(`修复应用失败: ${error.message}`); } } +async function handleApplyReviewedFix() { + const review = pendingFixReview; + if (!review || !review.isCurrent() || !requireApplied()) { + pendingFixReview = null; + $("#fixReviewPanel").hidden = true; + operationFeedback("修复候选已过期，请重新校验和预览。", true); + return false; + } + const button = $("#applyReviewedFix"); if (button) button.disabled = true; + try { + const checked = await write("/fixes/preview", "POST", { + pipeline: review.source, patch: review.fix.patch, + revision: review.report.revision, tool_fingerprint: review.report.tool_fingerprint, + }, true); + if (pendingFixReview !== review || !review.isCurrent()) return false; + if (!checked.ok || JSON.stringify(checked.patched) !== JSON.stringify(review.res.patched)) throw new Error("修复候选已改变，请重新预览"); + state.pipeline = checked.patched; + markPipelineChanged(); + operationFeedback(`已应用修复：${review.fix.title}（可撤销）`); + await validate(); return true; + } catch (error) { operationFeedback(error.message, true); return false; } + finally { if (button) button.disabled = false; } +} + async function validate() { if (!state.catalogReady) { toast("Catalog 暂不可用或版本不兼容，无法校验方案", true); @@ -525,6 +654,7 @@ function runBusy() { } function renderRunContext() { + renderPreflightFreshness(); const run = state.run; $("#modelRootLabel").textContent = $("#runModelRoot").value || "未设置"; $("#runButton").disabled = state.loading || !state.pipeline || Boolean(runBusy()) || !$("#runProfile").value; @@ -533,10 +663,10 @@ function renderRunContext() { $("#runContext").textContent = !run ? "尚未运行" : !currentDocument ? `其他方案：${run.filename} · ${runBusy() ? "运行中，可取消后运行当前方案" : "已结束；当前方案尚未运行"}` : `${run.filename} · ${new Date(run.startedAt).toLocaleString("zh-CN", { hour12: false })}\nProfile：${run.profile} · 模型目录：${run.modelRoot}`; - const stale = currentDocument && !runIsCurrent(run, { + const stale = currentDocument && ((run.deploymentSnapshot && run.deploymentSnapshot !== deploymentSnapshot()) || !runIsCurrent(run, { documentVersion: state.documentVersion, pipeline: state.pipeline, pending: drafts.pending, profile: $("#runProfile").value, modelRoot: $("#runModelRoot").value, - }); + })); $("#runFreshness").hidden = !stale; $("#runFreshness").textContent = "方案或运行设置已修改。以下结果属于提交时的版本，请重新运行验证当前修改。"; } @@ -557,9 +687,10 @@ async function runDraft() { switchTab("run"); operationFeedback(""); const run = captureRun({ documentVersion: state.documentVersion, pipeline: state.pipeline, filename: state.filename || state.sourceName || "未命名方案", profile: $("#runProfile").value, modelRoot: $("#runModelRoot").value }); + run.deploymentSnapshot = deploymentSnapshot(); state.run = run; renderRun(); try { - const result = await write("/runs", "POST", { pipeline: JSON.parse(run.pipeline), profile: run.profile, model_root: run.modelRoot }); + const result = await write("/runs", "POST", { pipeline: JSON.parse(run.pipeline), profile: run.profile, model_root: run.modelRoot, filename: state.filename, model_path_actions: modelPathActions() }); if (state.run !== run) return; run.id = result.job_id; run.job = { status: result.status || "queued" }; renderRun(); pollRun(run); } catch (error) { @@ -586,6 +717,153 @@ async function pollRun(run) { } } +async function ensureAppliedOrAction(action) { + const pending = drafts.pendingExcept(""); + for (const kind of pending) { + const ok = await applyDraft(kind); + if (!ok) return; + } + clearPendingAction(); + return await action(); +} + +function modelPathActions() { + return Object.fromEntries(Object.entries(state.modelPathActions || {}).filter(([id, action]) => + state.pipeline?.models?.some(m => m.model_id === id && m.model_path === action.path))); +} + +function deploymentSnapshot() { + return JSON.stringify({ documentVersion: state.documentVersion, pipeline: state.pipeline, pending: drafts.pending, + filename: state.filename, deployment: state.deployment, profile: $("#runProfile")?.value, + modelRoot: $("#runModelRoot")?.value, actions: modelPathActions() }); +} + +function renderPreflightFreshness() { + const summary = $("#preflightSummary"); + if (!summary || !state.preflight || state.preflight.snapshot === deploymentSnapshot()) return; + summary.hidden = false; summary.className = "preflight-summary stale"; + summary.textContent = "预检已过期：方案、部署配置或运行设置已改变，请重新检查运行条件。"; +} + +function renderPathIntent() { + const container = $("#modelPathIntent"); + if (!container) return; + container.replaceChildren(); container.hidden = !state.deployment; + if (!state.deployment) return; + const saved = JSON.parse(state.savedPipeline || "{}"); + for (const model of state.pipeline?.models || []) { + const old = saved.models?.find(m => m.model_id === model.model_id); + if (!old || old.model_path === model.model_path) continue; + const label = document.createElement("label"); label.textContent = `${model.model_id}：${old.model_path} → ${model.model_path}`; + const select = document.createElement("select"); select.dataset.modelIntent = model.model_id; + select.add(new Option("存在部署覆盖时，请明确路径意图", "")); + select.add(new Option("将新路径作为所选模型目录下的资产", "select_asset")); + select.add(new Option("保留当前部署覆盖", "preserve_override")); + select.value = modelPathActions()[model.model_id]?.action || ""; + select.addEventListener("change", () => { + state.modelPathActions ||= {}; + if (select.value) state.modelPathActions[model.model_id] = { path: model.model_path, action: select.value }; + else delete state.modelPathActions[model.model_id]; + history.record(snapshot()); setDirty(true); renderPreflightFreshness(); + }); + label.append(select); container.append(label); + } +} + +async function runPreflight() { + if (!state.pipeline || !requireApplied("", runPreflight, "运行预检")) return; + const summaryEl = $("#preflightSummary"); + if (!summaryEl) return; + const request = ++preflightRequest; + const snapshot = deploymentSnapshot(); + state.preflight = { request, snapshot }; + summaryEl.hidden = false; + summaryEl.className = "preflight-summary loading"; + summaryEl.textContent = "正在执行独立部署预检（无 Demo 执行）…"; + try { + const res = await write("/preflight", "POST", { + pipeline: state.pipeline, filename: state.filename, model_path_actions: modelPathActions(), + profile: $("#runProfile")?.value || "", + model_root: $("#runModelRoot")?.value || "models", + }, true); + if (request !== preflightRequest || snapshot !== deploymentSnapshot()) { renderPreflightFreshness(); return; } + summaryEl.className = "preflight-summary"; + const summary = res.summary || {}; + summaryEl.replaceChildren(); + const heading = document.createElement("div"); + heading.className = `preflight-status ${summary.status || "unknown"}`; + heading.textContent = summary.status === "ready" ? "✓ 部署预检通过（具备运行条件）" + : summary.status === "attention_required" ? "⚠ 预检需关注：部分工具或模型资产未就绪" + : summary.status === "validation_failed" ? "✗ Pipeline 校验未通过" + : `预检状态：${summary.status || "未知"}`; + summaryEl.append(heading); + + const desc = document.createElement("p"); + desc.textContent = `业务：${summary.biz_name || "未知"} · 项目根：${summary.project_root || "本地"}`; + summaryEl.append(desc); + + if (summary.tools) { + const toolsP = document.createElement("div"); + toolsP.textContent = `工具就绪：alg_pipeline_tool=${summary.tools.alg_pipeline_tool ? "✓" : "✗"} · alg_demo=${summary.tools.alg_demo ? "✓" : "✗"}`; + summaryEl.append(toolsP); + } + if (Array.isArray(summary.assets)) { + const assetsList = document.createElement("ul"); + for (const asset of summary.assets) { + const li = document.createElement("li"); + li.textContent = `${asset.model_id}: ${asset.resolved_path} [${asset.status}]`; + assetsList.append(li); + } + summaryEl.append(assetsList); + } + if (summary.next_step) { + const nextP = document.createElement("p"); + nextP.className = "preflight-next-step"; + nextP.textContent = `后续指引：${summary.next_step}`; + summaryEl.append(nextP); + } + } catch (error) { + if (request !== preflightRequest || snapshot !== deploymentSnapshot()) { renderPreflightFreshness(); return; } + summaryEl.className = "preflight-summary error"; + summaryEl.textContent = `预检执行失败：${error.message}`; + } +} + +async function handleAssociateDeployment() { + if (!state.pipeline || !state.filename) { + toast("请先保存当前 Pipeline 到 configs/ 再关联部署配置", true); + return; + } + if (!requireApplied()) return; + const documentVersion = state.documentVersion; + const filename = state.filename; + const confName = $("#associateConfSelect")?.value; + if (!confName) { + toast("请选择要关联的部署配置", true); + return; + } + try { + operationFeedback(`正在关联部署配置：${confName}…`); + const res = await write("/deployment/associate", "POST", { + filename, profile: $("#runProfile")?.value || "", + conf_name: confName, + model_root: $("#runModelRoot")?.value || "models", + }); + if (documentVersion !== state.documentVersion || filename !== state.filename) return; + if (res.ok) { + state.deployment = res.deployment; + renderPreflightFreshness(); + state.saveTargets = res.save_targets || [state.filename, confName]; + operationFeedback(`已成功关联部署配置：${confName}`); + toast(`已关联部署配置：${confName}`); + renderAll(); + } + } catch (error) { + operationFeedback(`关联部署配置失败：${error.message}`, true); + toast(error.message, true); + } +} + let editingModelId = ""; let modelDocument = null; @@ -699,7 +977,13 @@ function applyModel() { model_config: readConfigFields($("#modelConfigFields")), backend_config: readConfigFields($("#backendConfigFields")), }; state.pipeline.models ??= []; + const before = state.pipeline.models.find(m => m.model_id === editingModelId); + const changedPath = !before || before.model_path !== model.model_path || editingModelId !== model.model_id; upsertModel(state.pipeline, state.catalog, editingModelId, model); + if (changedPath) { + state.modelPathActions ||= {}; + state.modelPathActions[model.model_id] = { path: model.model_path, action: "select_asset" }; + } drafts.clear("model"); markPipelineChanged(); renderAll(); loadModelEditor(model.model_id); toast("模型已应用，请校验方案"); return true; @@ -753,6 +1037,7 @@ async function restoreHistory(direction) { if (!restored) return; const bizChanged = state.pipeline?.biz_name !== restored.pipeline?.biz_name || !state.catalogReady; state.pipeline = restored.pipeline; state.selected = restored.selected; + state.modelPathActions = restored.modelPathActions || {}; markPipelineChanged(false); if (bizChanged) { state.loading = true; graph.editable = false; @@ -766,9 +1051,11 @@ async function restoreHistory(direction) { } function deleteSelectedEdge() { - if (!state.editing || !state.selectedEdge || !requireApplied()) return; - disconnectPorts(state.pipeline, state.catalog, state.selectedEdge); - markPipelineChanged(); renderAll(); toast("连线已删除，可撤销恢复"); + const edge = state.selectedEdge; + if (!state.editing || !edge || !requireApplied()) return; + return applyAuthoring(edge.dependency + ? { kind: "remove_dependency", node_id: edge.target, depends_on_id: edge.source } + : { kind: "disconnect", source: { node_id: edge.source, port: edge.sourcePort }, target: { node_id: edge.target, port: edge.targetPort } }); } $("#editModeButton").addEventListener("click", () => setEditing(!state.editing)); @@ -796,6 +1083,8 @@ $("#rawJson").addEventListener("input", event => { }); for (const [kind, selector] of [["node", "#nodeForm"], ["model", "#modelForm"]]) { const remember = event => { + // Graph controls submit authoring operations, not node-parameter drafts. + if (event.target.closest?.("#nodeBindings")) return; event.target.setCustomValidity?.(""); if (!state.editing) return; operationFeedback(""); @@ -811,7 +1100,7 @@ $("#discardModel").addEventListener("click", () => { drafts.clear("model"); clea window.addEventListener("keydown", event => { const typing = event.target.closest?.("input, textarea, select, [contenteditable=true]"); if ((event.ctrlKey || event.metaKey) && event.key.toLowerCase() === "s") { - event.preventDefault(); if (state.editing) save(false); return; + event.preventDefault(); if (state.editing) ensureAppliedOrAction(() => save(false)); return; } if (typing || !state.editing) return; if ((event.ctrlKey || event.metaKey) && ["z", "y"].includes(event.key.toLowerCase())) { @@ -839,21 +1128,23 @@ $("#newEntryButton").addEventListener("click", () => { $("#operatorsToggle").setAttribute("aria-pressed", "true"); $(".create-panel").open = true; $("#bizSelect").focus(); }); -$("#quickValidateButton").addEventListener("click", validate); -$("#openRunButton").addEventListener("click", () => switchTab("run")); +$("#quickValidateButton").addEventListener("click", () => ensureAppliedOrAction(validate)); +$("#openRunButton").addEventListener("click", () => ensureAppliedOrAction(() => switchTab("run"))); for (const id of ["runProfile", "runModelRoot"]) $("#" + id).addEventListener("input", renderRunContext); $("#newButton").addEventListener("click", () => createPipeline().catch(error => toast(error.message, true))); -$("#saveButton").addEventListener("click", () => save(false)); -$("#saveAsButton").addEventListener("click", () => save(true)); -$("#saveSolutionButton").addEventListener("click", () => save(true, true)); +$("#saveButton").addEventListener("click", () => ensureAppliedOrAction(() => save(false))); +$("#saveAsButton").addEventListener("click", () => ensureAppliedOrAction(() => save(true))); +$("#saveSolutionButton").addEventListener("click", () => ensureAppliedOrAction(() => save(true, true))); $("#layoutButton").addEventListener("click", () => { graph.layout(graphDocument(state.pipeline, state.catalog).nodes, true); renderAll(); }); $("#operatorSearch").addEventListener("input", renderOperators); $("#bizSelect").addEventListener("change", filterProfiles); -$("#validateButton").addEventListener("click", validate); -$("#runButton").addEventListener("click", runDraft); +$("#validateButton").addEventListener("click", () => ensureAppliedOrAction(validate)); +$("#runButton").addEventListener("click", () => ensureAppliedOrAction(runDraft)); +$("#preflightButton")?.addEventListener("click", () => ensureAppliedOrAction(runPreflight)); +$("#associateDeploymentButton")?.addEventListener("click", handleAssociateDeployment); $("#cancelButton").addEventListener("click", async () => { const run = state.run; if (!run?.id || !runBusy()) return; @@ -862,7 +1153,7 @@ $("#cancelButton").addEventListener("click", async () => { }); document.querySelectorAll(".tabs button").forEach(button => button.addEventListener("click", () => switchTab(button.dataset.tab))); -function applyNode() { +async function applyNode() { if (!state.editing || !$("#nodeForm").reportValidity()) return false; if (!requireApplied("node")) return; if (!state.catalog.nodes.some(definition => definition.node_type === effectiveNodes().find(node => node.id === state.selected)?.node_type)) return toast("节点定义不可用，请重新打开方案或修正 JSON", true); @@ -874,24 +1165,27 @@ function applyNode() { } try { const config = readConfigFields($("#configFields")); - for (const item of state.pipeline.pipeline) item.depends_on = item.depends_on.map(id => id === node.id ? newId : id); - if (newId !== node.id && graph.positions[node.id]) { - graph.positions[newId] = graph.positions[node.id]; - delete graph.positions[node.id]; - graph.callbacks.positionsChanged(graph.positions); + const oldId = node.id; + if (newId !== oldId) { + return await applyAuthoring({ kind: "rename_node", node_id: oldId, new_id: newId }, { + except: "node", selected: newId, + configure: candidate => { candidate.pipeline.find(n => n.id === newId).config = config; }, + onCommit: () => { + if (graph.positions[oldId]) { graph.positions[newId] = graph.positions[oldId]; delete graph.positions[oldId]; savePositions(graph.positions); } + }, + }); } - node.id = newId; node.config = config; state.selected = newId; + node.config = config; drafts.clear("node"); markPipelineChanged(); renderAll(); return true; } catch (error) { operationFeedback(`配置错误：${error.message}`, true); return false; } } $("#deleteNode").addEventListener("click", () => { - if (!requireApplied()) return; - removeNode(state.pipeline, state.catalog, state.selected); - delete graph.positions[state.selected]; - graph.callbacks.positionsChanged(graph.positions); - state.selected = ""; markPipelineChanged(); renderAll(); + const id = state.selected; + applyAuthoring({ kind: "remove_node", node_id: id }, { selected: "", onCommit: () => { + delete graph.positions[id]; savePositions(graph.positions); + } }); }); async function applyJson() { diff --git a/tools/pipeline_studio/web/graph.js b/tools/pipeline_studio/web/graph.js index 15c4e70f..21cc0d21 100644 --- a/tools/pipeline_studio/web/graph.js +++ b/tools/pipeline_studio/web/graph.js @@ -368,12 +368,12 @@ export class GraphView { const route = routeOrthogonal({ x: from.x + WIDTH, y: from.y + y1 }, { x: to.x, y: to.y + y2 }, obstacles, index); this.routes.push(route); const d = route.map((point, i) => `${i ? "L" : "M"}${point.x},${point.y}`).join(" "); - const group = svg("g", { class: `edge${binding.dependency ? " dependency" : ""}${edgeKey(binding) === edgeKey(this.selectedEdge) ? " selected" : ""}`, tabindex: 0, role: "button", "aria-label": `${binding.source} ${binding.sourcePort || ""} → ${binding.target} ${binding.targetPort || "执行依赖"}` }); + const group = svg("g", { class: `edge${binding.dependency ? " dependency" : ""}${binding.ambiguous ? " ambiguous" : ""}${edgeKey(binding) === edgeKey(this.selectedEdge) ? " selected" : ""}`, tabindex: 0, role: "button", "aria-label": `${binding.source} ${binding.sourcePort || ""} → ${binding.target} ${binding.targetPort || "执行依赖"}` }); group.dataset.edgeKey = edgeKey(binding); const edge = svg("path", { class: "edge-line", d }); const hit = svg("path", { class: "edge-hit", d, "aria-hidden": "true" }); const title = svg("title"); - title.textContent = `${binding.source}${binding.sourcePort ? `.${binding.sourcePort}` : ""} → ${binding.target}${binding.targetPort ? `.${binding.targetPort}` : ""}${binding.dependency ? " · 执行依赖" : ""}`; + title.textContent = `${binding.ambiguous ? "来源存在歧义 · " : ""}${binding.source}${binding.sourcePort ? `.${binding.sourcePort}` : ""} → ${binding.target}${binding.targetPort ? `.${binding.targetPort}` : ""}${binding.dependency ? " · 执行依赖" : ""}`; group.append(title, edge, hit); const select = () => { this.setSelectedEdge(binding); this.callbacks.selectEdge?.(binding); }; group.addEventListener("click", event => { event.stopPropagation(); select(); }); diff --git a/tools/pipeline_studio/web/index.html b/tools/pipeline_studio/web/index.html index 7939d102..e45e1756 100644 --- a/tools/pipeline_studio/web/index.html +++ b/tools/pipeline_studio/web/index.html @@ -113,6 +113,7 @@
+
输入绑定与执行依赖
@@ -144,6 +145,7 @@