diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 9566b59..991f1b1 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -17,11 +17,13 @@ jobs: python-version: ${{ matrix.python }} cache: pip - run: python -m pip install -r requirements-dev.txt + - run: python -m pip install -e '.[chatgpt]' - run: python -m pytest -q - uses: actions/setup-node@v4 with: node-version: "22" - run: node --test tests/workspace_manual.test.mjs + - run: node --test tests/chatgpt_plan_ui.test.mjs render: runs-on: ubuntu-latest env: diff --git a/README.md b/README.md index 368bc46..93f8379 100644 --- a/README.md +++ b/README.md @@ -130,6 +130,7 @@ Choose one route below. Stop the existing server before running a different laun | Model route | How to connect | Compatibility boundary | | --- | --- | --- | +| ChatGPT plan direct | Select the direct route and authorize ManiLoop in your browser | Eligible account models through Responses, without Codex CLI; no server token hard cap in the current preview | | GPT through Codex | Official Codex CLI login; select the Codex source | Uses models available to that CLI/account; ManiLoop does not extract login tokens | | Hosted or self-hosted vision model | Enter the service base URL, model ID, and API key, or import provider TOML | Robot control requires **Responses API + image input + JSON Schema output** | | Local LeRobot checkpoints | Select a supported model loaded by the isolated inference worker | Three reviewed LIBERO checkpoint presets; not arbitrary model-folder import | @@ -144,6 +145,10 @@ Choose one route below. Stop the existing server before running a different laun ### GPT / compatible model APIs +To use your ChatGPT plan directly, install `python -m pip install -e '.[chatgpt]'`, start the normal local demo, and choose **ChatGPT 套餐直连 · 独立授权** in the model panel. Click **Continue with ChatGPT**, authorize ManiLoop, refresh the account's model list, and run the text/image/action-format diagnostics. This route uses ManiLoop's own prompt and control loop without launching Codex CLI. Account eligibility and model access are verified by those diagnostics. See the [bilingual setup and comparison guide](docs/CHATGPT_PLAN.md). + +The existing Codex CLI route remains available as a separate harness configuration: + Install a current official [Codex CLI](https://github.com/openai/codex), then: ```bash diff --git a/README.zh-CN.md b/README.zh-CN.md index 6c2717e..dfefde3 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -130,6 +130,7 @@ python -m maniloop demo --backend robocasa --task OpenDrawer \ | 模型接入方式 | 如何连接 | 兼容边界 | | --- | --- | --- | +| ChatGPT 套餐直连 | 选择直连来源,在浏览器中为 ManiLoop 独立授权 | 使用账号可用的 Responses 模型,不经过 Codex CLI;当前预览无服务端 token 硬上限 | | 通过 Codex 使用 GPT | 在官方 Codex CLI 登录,选择 Codex 来源 | 使用该 CLI/账户可用的模型;ManiLoop 不提取登录 token | | 云端或自托管视觉模型 | 填写服务基础地址、模型 ID、API Key,或导入供应商 TOML | 机器人控制需要 **Responses API、图像输入和 JSON Schema 输出** | | 本地 LeRobot 检查点 | 选择由独立推理进程加载的支持模型 | 当前提供三个已核对的 LIBERO 检查点预设,不是任意模型文件夹导入 | @@ -144,6 +145,10 @@ python -m maniloop demo --backend robocasa --task OpenDrawer \ ### GPT 与兼容模型 API +使用 ChatGPT 套餐直连时,先运行 `python -m pip install -e '.[chatgpt]'`,正常启动本地 demo,在模型面板选择 **ChatGPT 套餐直连 · 独立授权**。点击 **Continue with ChatGPT** 授权 ManiLoop,刷新账号模型列表,再运行文字、图像和动作格式诊断。该方式使用 ManiLoop 自己的提示和闭环,不启动 Codex CLI;账号资格与模型访问以诊断结果为准。详见[双语接入与比较指南](docs/CHATGPT_PLAN.md)。 + +现有 Codex CLI 入口继续保留,作为单独标记的 harness 配置: + 安装近期官方 [Codex CLI](https://github.com/openai/codex),然后运行: ```bash diff --git a/docs/CHATGPT_PLAN.md b/docs/CHATGPT_PLAN.md new file mode 100644 index 0000000..2362630 --- /dev/null +++ b/docs/CHATGPT_PLAN.md @@ -0,0 +1,103 @@ +# ChatGPT plan direct connection / ChatGPT 套餐直连 + +[English](#english) · [中文](#中文) + +## English + +ManiLoop can obtain its own ChatGPT authorization and call the public Responses API directly. This route uses eligible ChatGPT plan access and keeps the robot loop, prompts, observation history, and action validation in ManiLoop. It does not launch Codex CLI or load its harness. The existing **Codex CLI** and **API key** routes remain available. + +Install the authentication extra in the main Python environment: + +```bash +python -m pip install -e '.[chatgpt]' +python -m maniloop demo --port 8767 +``` + +1. Open the local workbench and choose **模型运行 → 模型连接与认证 → ChatGPT 套餐直连 · 独立授权**. +2. Click **Continue with ChatGPT**. In the browser, authorize **ManiLoop** to use your ChatGPT plan. This is separate from your Codex login; an existing CLI login does not authorize this connection. +3. Return to the workbench. It shows the connected account and whether plan usage is enabled. Use the account selector to switch saved accounts, **重新授权此账号** to renew authorization, or **退出此账号** to disconnect locally and attempt session revocation. Account changes are locked while a run is active. +4. Click **刷新模型列表**, then choose a model from this account's catalog. GPT-6 Astra access depends on the selected account and the current service. A catalog entry alone does not establish successful inference. +5. Run **文本诊断**, **图像诊断**, and **动作格式诊断** in order. Each diagnostic makes a model request; the action-format diagnostic does not execute a robot action. After these pass, use **单步执行** before starting a complete episode. + +Click **取消授权** to cancel a sign-in attempt. Closing the browser window alone does not cancel it: return to the workbench and cancel explicitly before starting again, or wait for the attempt to expire. If registration issued a client ID but code exchange failed, use **继续上次注册** to retry that registration without creating a new one. If the browser blocks the popup, allow popups for the local workbench and click again. A failed sign-in does not silently replace another saved account. Use ChatGPT **Settings → Usage** to manage app access and usage. If remote revocation cannot be confirmed during logout, disconnect the app there as well. + +### Record one LIBERO episode + +Install the [independent LIBERO environment](LIBERO.md), finish the workbench authorization and diagnostics, and select the account you want to use. The CLI reuses that active ManiLoop account without opening a login window. If you set `MANILOOP_CHATGPT_AUTH_DIR` for the workbench, use the same value in this terminal. + +```bash +python -m maniloop benchmark \ + --backend libero --libero-suite libero_spatial --libero-task-id 0 --init-state-id 0 \ + --seed 0 --agent llm_cloud --chatgpt-plan --model gpt-6-astra \ + --timing controlled --context-mode paired --reasoning-effort medium \ + --llm-control tcp_target_servo_v2 --observation-profile llm_rgb512 \ + --max-calls 0 --max-wall-seconds 3600 --max-sim-seconds 120 \ + --request-timeout-seconds 120 --record-episode \ + --output runs/chatgpt-plan-recorded +``` + +This makes real model requests and records every LIBERO control step. `--max-calls 0` removes the decision-count cap; environment termination and wall/simulation time limits remain. Use a model verified for the selected account. `--chatgpt-plan`, `--codex-login`, and `--provider-config` are mutually exclusive; errors never switch billing paths. Every attempt is saved separately, and success is determined by the independent evaluator. See [episode recording and MP4 export](EPISODE_RECORDING.md) for integrity checks and export commands. + +### Comparison and budget behavior + +The direct route shares ManiLoop's sensor-only input construction, image bytes, history, action schema, and local action checks with the Responses API policy. Its application-side input manifest records the request protocol and content hashes; it does not claim visibility into server-side processing. The Codex CLI route remains a separately labeled harness configuration. + +The ChatGPT plan preview requires streamed Responses requests with `store: false` and explicit input history. It does **not** accept a server-side `max_output_tokens` parameter. Set the reasoning effort and local request deadline explicitly, and distinguish these from actual reasoning-token usage. A client text-length limit or timeout is not a server token budget. Strict equal-token-budget comparisons therefore require a different supported connection. An incomplete, failed, interrupted, or invalid response never authorizes a robot action. There is no automatic switch to another model, provider, or paid API key when access fails. + +### Local credentials and validation status + +ManiLoop stores its own authorization records in protected local storage, separate from Codex. It never needs you to copy a token into the workbench. OAuth credentials are not sent to a custom provider, added to experiment logs, or written into browser storage. The public GitHub Pages site is a static replay and cannot run this local sign-in flow. + +This feature's offline protocol and UI checks do not establish access for your account. The first successful text/image/structured-action diagnostics are the acceptance check for your selected account and model; they do not demonstrate task success or a benchmark success rate. + +Official references, checked 2026-10-28: [registration and sign-in](https://developers.openai.com/siwc/token-sharing-open-source/sign-in), [models and inference](https://developers.openai.com/siwc/token-sharing-open-source/models-and-inference), [preview limitations](https://developers.openai.com/siwc/token-sharing-open-source/preview-limitations). + +## 中文 + +ManiLoop 可以独立获得 ChatGPT 授权,直接请求官方 Responses API,使用符合条件的 ChatGPT 套餐。机器人闭环、提示、观测历史和动作校验均由 ManiLoop 管理,不启动 Codex CLI,也不加载其 harness。现有 **Codex CLI** 和 **API Key** 接入继续保留。 + +在主 Python 环境中安装认证扩展: + +```bash +python -m pip install -e '.[chatgpt]' +python -m maniloop demo --port 8767 +``` + +1. 打开本地工作台,选择 **模型运行 → 模型连接与认证 → ChatGPT 套餐直连 · 独立授权**。 +2. 点击 **Continue with ChatGPT**,在浏览器中授权 **ManiLoop** 使用你的 ChatGPT 套餐。它与 Codex 登录独立,已登录 CLI 不代表此连接已授权。 +3. 返回工作台,确认账号和套餐授权状态。可选择已保存的账号、点击 **重新授权此账号**,或 **退出此账号**。退出会清除本地凭据并尝试撤销远端会话;实验运行中不能切换账号。 +4. 点击 **刷新模型列表** 并选择模型。GPT-6 Astra 是否可用取决于此账号与服务当前权限;出现在目录中不等于推理已通过。 +5. 依次运行 **文本诊断、图像诊断、动作格式诊断**。每项都会请求模型,动作格式诊断不会移动机器人。通过后先用 **单步执行** 检查,再开始完整 episode。 + +点击 **取消授权** 可取消本次登录。仅关闭浏览器窗口不会取消授权;请回到工作台明确取消后再重试,或等待本次授权过期。若已取得注册 ID 但授权码交换失败,可点击 **继续上次注册** 复用该注册重试,无需创建新注册。如果弹窗被阻止,请允许本地工作台打开弹窗后重试。登录失败不会静默替换另一个已保存账号。可在 ChatGPT **设置 → 使用情况** 管理应用授权与用量;若退出时未确认远端撤销成功,也可在那里断开应用。 + +### 录制一个 LIBERO 完整回合 + +先按 [LIBERO 安装说明](LIBERO.md)准备独立环境,在工作台完成授权、诊断并选择账号。CLI 复用 ManiLoop 当前账号,不会打开登录窗口。如果工作台设置了 `MANILOOP_CHATGPT_AUTH_DIR`,请在此终端使用相同值。 + +```bash +python -m maniloop benchmark \ + --backend libero --libero-suite libero_spatial --libero-task-id 0 --init-state-id 0 \ + --seed 0 --agent llm_cloud --chatgpt-plan --model gpt-6-astra \ + --timing controlled --context-mode paired --reasoning-effort medium \ + --llm-control tcp_target_servo_v2 --observation-profile llm_rgb512 \ + --max-calls 0 --max-wall-seconds 3600 --max-sim-seconds 120 \ + --request-timeout-seconds 120 --record-episode \ + --output runs/chatgpt-plan-recorded +``` + +这会发起真实模型请求,并录制每个 LIBERO 原生控制步。`--max-calls 0` 取消决策次数上限,环境终止及墙钟、仿真时限仍有效。模型必须已在所选账号中验证可用。`--chatgpt-plan`、`--codex-login`、`--provider-config` 互斥,出错不会切换计费路径。每次尝试独立保存,成功与否由独立评分器判定。完整性检查和视频导出见[完整 episode 录制](EPISODE_RECORDING.md)。 + +### 公平比较与预算 + +套餐直连与 Responses API 策略共用传感输入构造、图片字节、历史、动作 Schema 和本地校验。应用侧请求记录包含协议版本与输入摘要,但不代表能够看到服务端内部处理。Codex CLI 仍作为独立的 harness 配置标记。 + +当前套餐预览要求流式请求、`store: false` 和显式发送历史,**不支持服务端 `max_output_tokens` 上限**。请明确设置推理强度与本地请求时限,并将它们和实际推理 token 消耗区分。客户端文本长度限制或超时不是服务端 token 预算;要求严格 token 上限的对照应使用支持该能力的其他连接。断流、失败、中断或格式无效的响应都不能触发动作。权限或额度不足时,不会自动改用其他模型、供应商或付费 API Key。 + +### 凭据与验证范围 + +认证记录保存在本机受保护位置,与 Codex 分开,不需要手动粘贴 token。OAuth 凭据不会发送给自定义供应商,也不会进入实验日志或浏览器存储。GitHub Pages 是静态回放页面,不能运行此本地认证流程。 + +离线协议和界面检查不代表你的账号已通过接入。首次成功的文字、图像、结构化动作诊断用于验收所选账号和模型,不代表任务成功或基准成功率。 + +官方依据,核对日期 2026-10-28:[注册与登录](https://developers.openai.com/siwc/token-sharing-open-source/sign-in)、[模型与推理](https://developers.openai.com/siwc/token-sharing-open-source/models-and-inference)、[当前预览限制](https://developers.openai.com/siwc/token-sharing-open-source/preview-limitations)。 diff --git a/docs/LIBERO.md b/docs/LIBERO.md index 58211a6..0104729 100644 --- a/docs/LIBERO.md +++ b/docs/LIBERO.md @@ -88,6 +88,8 @@ ManiLoop 保留 `maniloop_libero_rgb_proprio_v1` 作为基线,并增加 LLM - 原生 VLA 接口是 `Action(kind="osc_pose", values=(七个归一化值), frame="world")`。前六维为世界坐标系 TCP 平移 / 旋转增量,最后一维 -1 张开、+1 闭合。每维范围 [-1,1],采样间隔必须为 0.05 秒(20 Hz)。动作由官方控制器执行;已接入真实 SmolVLA / ACT / Diffusion 学习策略,使用独立推理环境;见 [本地策略说明](VLA.md)。 - 原生单步基线 `osc_step` 使用米 / 弧度动作;适配器显式转换为原生 OSC 输入。平移范数最多 0.05 m,旋转范数最多 0.5 rad,每次 move 执行一个控制步。并不保证末端立即到达目标,模型须重新观察。夹爪只接受 0 / 1,执行 10 个控制步;wait 为一个零机械臂增量控制步。 - 网页与云端批量实验默认使用 `llm_rgb512` 的 512×512 JPEG 和 `tcp_target_servo_v2`。米/弧度增量转换为固定目标位姿,本地以 20 Hz 原生 OSC 反馈追踪,最多 20 步;位置容差 2 mm、姿态容差 0.02 rad,低于速度门槛并稳定两步才报告 reached。夹爪命令保持 TCP 目标至少 10 步,并等待位姿与开度变化速度稳定;张开时还要求开度达到 90%。completed 表示夹爪稳定,不代表抓到物体;超过 20 步仍未满足条件则报告 timed_out。位姿和夹爪均只使用本体反馈。 +- LIBERO 另提供 `tcp_target_servo_v3`,可在网页动作执行选项中选择,或通过 `--llm-control tcp_target_servo_v3` 启用。它将 **move 的规划目标增量**放宽为平移向量范数最多 2.5 m、旋转向量范数最多 π rad(180°),最多追踪 200 个原生控制步(10 秒);到达目标或环境终止时提前停止。夹爪命令仍最多 20 步(1 秒),其他收敛判据保持 v2 不变。**原生 OSC 的 20 Hz、每步 0.05 m / 0.5 rad 控制尺度和 2 ms 物理步长均未改变**;大目标由本地反馈分步执行,不是单步瞬移。手动目标跟踪时限独立设置,仍可覆盖到最多 10 秒。切换到非 LIBERO 后端时网页禁用 v3 并回落 v2;默认值与原有演示预设继续使用 v2。 +- v3 的 2.5 m 仅是异常输入保护,**不是可达工作空间或建议步长**。固定 Panda 从 joint1 到 grip_site 的平移链长上界为 `L = 0.316 + 0.0825 + sqrt(0.0825² + 0.384²) + 0.088 + 0.1065 + 0.097 ≈ 1.082762 m`,任意两构型末端间距不超过 `2L ≈ 2.165525 m`,据此向上取 2.5 m。π rad 覆盖任意目标姿态的最短旋转。10 秒沿用手动目标跟踪的最大工程预算;关节范围、碰撞或接触仍可能阻止到达。v3 与 v2 是不同控制协议,比较时必须分组记录。 - `--llm-control osc_step --observation-profile debug_rgb128` 可复现原始基线;批量实验支持 `--request-timeout-seconds 120 --reasoning-effort low`,套件相应字段为 `llm_control`、`observation_profile`、`request_timeout_seconds`、`reasoning_effort`。模式、图像、控制器容差与实际请求参数分别进入 manifest 和比较组。 - 保留官方控制器意味着不会套用自建任务的 IK、碰撞预检或运动规划。两类环境的动作能力不同,分别形成比较组。 - 模型推理时默认冻结物理;实时模式单独比较。网页闲置时不会消耗 LIBERO 控制步预算。成功或预算终止后不再推进;没有自建任务的额外 1.2 秒静置窗口。 diff --git a/pyproject.toml b/pyproject.toml index 38cdae7..e4515c8 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -13,6 +13,7 @@ dependencies = ["mujoco==3.12.0", "numpy==2.5.3", "Pillow==12.3.0", "openai==3.8 [project.optional-dependencies] dev = ["pytest==9.1.1"] +chatgpt = ["PyJWT[crypto]>=2.10,<3", "httpx>=0.28,<1"] [project.scripts] maniloop = "maniloop.cli:main" diff --git a/src/maniloop/agents/llm.py b/src/maniloop/agents/llm.py index 95e6cc3..1b278c4 100644 --- a/src/maniloop/agents/llm.py +++ b/src/maniloop/agents/llm.py @@ -31,6 +31,10 @@ def last_latency(self): def last_usage(self): return self.provider.last_usage + @property + def last_request_manifest(self): + return getattr(self.provider, "last_request_manifest", None) + def reset(self, seed=0): # Requests have no persistent remote conversation; episode history is supplied explicitly. pass diff --git a/src/maniloop/backends/libero/control.py b/src/maniloop/backends/libero/control.py new file mode 100644 index 0000000..ce591b2 --- /dev/null +++ b/src/maniloop/backends/libero/control.py @@ -0,0 +1,42 @@ +"""LLM target bounds are separate from the pinned upstream OSC command scale.""" + +import math +from maniloop.controllers.target import TargetSettings + + +TARGET_MODES = ("tcp_target_servo_v2", "tcp_target_servo_v3") +LLM_CONTROL_MODES = ("osc_step", *TARGET_MODES) + +# Pinned robosuite 1.4.0 Panda XML: joint1 origin -> grip_site. The fixed +# base -> joint1 offset cancels between poses. Triangle inequality bounds any +# TCP displacement by 2 * this length, ignoring joint limits and obstacles. +PANDA_CHAIN_LENGTH_M = .316 + .0825 + math.hypot(.0825, .384) + .088 + .1065 + .097 +PANDA_DISPLACEMENT_BOUND_M = 2 * PANDA_CHAIN_LENGTH_M + + +def target_settings(mode, *, gripper=False): + # 10 s is an explicit engineering deadline (also supported by manual mode), + # not a guaranteed reach time. Leave every v2/shared-backend setting intact. + return TargetSettings(max_steps=200 if mode == "tcp_target_servo_v3" and not gripper else 20) + + +def command_limits(mode): + return (2.5, math.pi) if mode == "tcp_target_servo_v3" else (.05, .5) + + +def wide_target_description(): + return { + "translation_max_m": 2.5, + "rotation_max_rad": math.pi, + "bounds_semantics": "invalid_input_guard_not_reachable_workspace", + "translation_basis": "2 * joint1_to_grip_site_fixed_translation_chain_length", + "robot_chain_length_m": PANDA_CHAIN_LENGTH_M, + "geometric_displacement_bound_m": PANDA_DISPLACEMENT_BOUND_M, + "rotation_basis": "shortest_SO3_rotation_at_most_pi", + "native_translation_norm_max_m": .05, + "native_rotation_norm_max_rad": .5, + "target_timeout_seconds": 10.0, + "gripper_max_control_steps": 20, + "gripper_timeout_seconds": 1.0, + "timeout_basis": "explicit_engineering_deadline_not_reachability_guarantee", + } diff --git a/src/maniloop/backends/libero/environment.py b/src/maniloop/backends/libero/environment.py index a25d917..6c89b0f 100644 --- a/src/maniloop/backends/libero/environment.py +++ b/src/maniloop/backends/libero/environment.py @@ -11,6 +11,7 @@ from maniloop.core.actions import Action from maniloop.controllers.target import PoseTarget from .transport import Worker, SUITES +from .control import LLM_CONTROL_MODES, TARGET_MODES, command_limits, target_settings, wide_target_description class LiberoEnvironment: @@ -70,7 +71,7 @@ def _clear_episode(self): self.terminated = False self._queue.clear() self._target = None - self._grip = 0.0 if self.llm_control == "tcp_target_servo_v2" else -1.0 + self._grip = 0.0 if self.llm_control in TARGET_MODES else -1.0 self.feedback = { "status": "ready", "message": "Official LIBERO initialization loaded", @@ -101,19 +102,22 @@ def reset(self, seed=0): self._clear_episode() def set_llm_control(self, mode): - if mode not in ("osc_step", "tcp_target_servo_v2") or self.busy: + if mode not in LLM_CONTROL_MODES or self.busy: raise ValueError("Unknown control mode or action still executing") self.llm_control = mode if self.simulation_time == 0: # Official warmup uses zero gripper action; preserve it in target mode. - self._grip = 0.0 if mode == "tcp_target_servo_v2" else -1.0 + self._grip = 0.0 if mode in TARGET_MODES else -1.0 def describe(self): result = copy.deepcopy(self._description) result["llm_control"] = self.llm_control - if self.llm_control == "tcp_target_servo_v2": - result["target_controller"] = PoseTarget.description() - result["llm_action_adapter"] = "Fixed TCP target; proprioceptive OSC feedback up to 20 control steps; no object state" + if self.llm_control in TARGET_MODES: + settings = target_settings(self.llm_control) + result["target_controller"] = PoseTarget.description(settings) + result["llm_action_adapter"] = f"Fixed TCP target; proprioceptive OSC feedback up to {settings.max_steps} control steps; no object state" + if self.llm_control == "tcp_target_servo_v3": + result["target_command_limits"] = wide_target_description() return result def observe(self): @@ -143,16 +147,32 @@ def observe(self): "Native osc_pose chunks contain 7 normalized values: world xyz/rotation delta, then -1 open / +1 close. " "Images use top-left pixel origin, camera_to_world is OpenCV camera coordinates.", } - if self.llm_control == "tcp_target_servo_v2": - observation["action_limits"]["target_tracking"] = PoseTarget.description() + if self.llm_control in TARGET_MODES: + settings = target_settings(self.llm_control) + observation["action_limits"]["target_tracking"] = PoseTarget.description(settings) observation["robot_description"] = ( "LIBERO Panda, world frame, RGB sensors only. move captures one fixed TCP pose target " "from the current pose plus your delta, then uses proprioceptive feedback at 20 Hz " - "for up to 1 second. Rotation is Exp(world rotation vector) @ current rotation. " + + ("for up to 1 second. " if self.llm_control == "tcp_target_servo_v2" else "for up to 10 seconds. ") + + "Rotation is Exp(world rotation vector) @ current rotation. " "Inspect reached/timed_out/interrupted feedback and actual displacement; accepted is not reached. " "Gripper accepts only 0 closed or 1 open and holds arm pose; completed does not prove a grasp. " "No object poses or success feedback are available. Camera pixels have top-left origin." ) + if self.llm_control == "tcp_target_servo_v3": + observation["action_limits"].update(wide_target_description()) + observation["robot_description"] += ( + " Your delta sets the final pose goal, not a single native control step. " + "The 2.5 m / pi rad limits are broad invalid-input guards, not a reachable workspace " + "or recommended movement size. Choose the distance from the task and visual evidence; " + "you need not split a distant goal into 5 cm requests. The local controller tracks " + "that fixed goal using bounded 0.05 m / 0.5 rad OSC residuals at 20 Hz; " + "these are command scales, not measured motion or speed guarantees. " + "Joint limits, obstacles and fixed-orientation reachability still apply. " + "No collision-free path planner is provided. Tracking ends early when reached " + "or the environment terminates; an unreachable goal may time out." + " Gripper-only commands retain the 20-step (1 second) tracking deadline." + ) self._snapshot = copy.deepcopy(observation) return observation, { k: base64.b64decode(v) for k, v in result["images"].items() @@ -200,12 +220,14 @@ def execute(self, action): or not np.isfinite(rot).all() ): raise ValueError("Expected finite xyz and rotation vectors") - if ( - np.linalg.norm(pos) > 0.05 + 1e-12 - or np.linalg.norm(rot) > 0.5 + 1e-12 - ): - raise ValueError("Move exceeds .05 m / .5 rad per-step limits") - native[:3], native[3:6] = pos / 0.05, rot / 0.5 + position_limit, rotation_limit = command_limits(self.llm_control) + if (np.linalg.norm(pos) > position_limit + 1e-12 + or np.linalg.norm(rot) > rotation_limit + 1e-12): + if self.llm_control != "tcp_target_servo_v3": + raise ValueError("Move exceeds .05 m / .5 rad per-step limits") + raise ValueError(f"Move exceeds {position_limit:g} m / {rotation_limit:g} rad command limits") + if self.llm_control == "osc_step": + native[:3], native[3:6] = pos / 0.05, rot / 0.5 elif kind == "gripper": opening = action.get("gripper_opening") if type(opening) not in (int, float) or opening not in (0, 1): @@ -217,13 +239,14 @@ def execute(self, action): "LIBERO supports move/gripper/wait or native osc_pose chunks" ) self._grip = float(native[6]) - if self.llm_control == "tcp_target_servo_v2" and kind in ("move", "gripper"): + if self.llm_control in TARGET_MODES and kind in ("move", "gripper"): self._target = PoseTarget( self._sensors, pos if kind == "move" else np.zeros(3), rot if kind == "move" else np.zeros(3), self._grip, gripper=kind == "gripper", + settings=target_settings(self.llm_control, gripper=kind == "gripper"), ) - self.feedback = {"status": "accepted", "message": "Tracking fixed TCP target from proprioception", "max_control_steps": 20} + self.feedback = {"status": "accepted", "message": "Tracking fixed TCP target from proprioception", "max_control_steps": self._target.settings.max_steps} return dict(self.feedback) self._queue.extend(native.tolist() for _ in range(count)) self.feedback = { diff --git a/src/maniloop/cli.py b/src/maniloop/cli.py index 854a3d4..6aef8d6 100644 --- a/src/maniloop/cli.py +++ b/src/maniloop/cli.py @@ -38,6 +38,7 @@ def main(argv=None): type=Path, default=Path("runs") if p != smoke else Path("artifacts"), ) + connection_groups = {} for p in (demo, batch): p.add_argument( "--timing", choices=["controlled", "realtime"], default="controlled" @@ -45,10 +46,14 @@ def main(argv=None): p.add_argument("--observation-max-age-seconds", type=float, default=None, help="Realtime observation age limit; 0 disables only the time limit (default: 60 seconds)") p.add_argument("--model", default=None) - p.add_argument("--codex-login", action="store_true", help="Use the official Codex CLI's ChatGPT login; no API key") + connection_groups[p] = p.add_mutually_exclusive_group() + connection_groups[p].add_argument("--codex-login", action="store_true", help="Use the official Codex CLI's ChatGPT login; no API key") + connection_groups[p].add_argument("--chatgpt-plan", action="store_true", + help="Use ManiLoop's independently authorized ChatGPT plan account; no Codex CLI or API key") demo.add_argument("--port", type=int, default=8765) for p in (demo, batch): - p.add_argument("--llm-control", choices=["tcp_target_servo_v2", "osc_step"], default="tcp_target_servo_v2") + p.add_argument("--llm-control", choices=["tcp_target_servo_v2", "tcp_target_servo_v3", "osc_step"], default="tcp_target_servo_v2", + help="LIBERO v3 allows broad pose goals with 10 s tracking; v2 preserves the original short-step protocol") p.add_argument("--observation-profile", choices=["llm_rgb512", "debug_rgb128"], default="llm_rgb512") batch.add_argument("--request-timeout-seconds", type=float, default=120) batch.add_argument("--context-mode", choices=["current", "paired"], default="current") @@ -65,7 +70,7 @@ def main(argv=None): batch.add_argument("--max-calls", type=int, default=30, help="Decision budget; 0 disables this limit (simulation and wall-time limits remain)") batch.add_argument("--max-sim-seconds", type=float, default=120.0) batch.add_argument("--max-wall-seconds", type=float, default=600.0) - batch.add_argument( + connection_groups[batch].add_argument( "--provider-config", type=Path, help="Provider TOML/JSON or CC Switch DB; key may use OPENAI_API_KEY", @@ -74,6 +79,8 @@ def main(argv=None): "--no-render", action="store_true", help="Only for offline mock adapter checks" ) args = parser.parse_args(argv) + if getattr(args, "llm_control", None) == "tcp_target_servo_v3" and args.backend != "libero": + parser.error("tcp_target_servo_v3 requires the LIBERO backend") if args.command == "chat": from maniloop.ui.server import serve_chat serve_chat(args) @@ -174,9 +181,9 @@ def main(argv=None): if args.suite and args.record_episode: cases = [replace(case, record_episode=True) for case in cases] connection = { - "credential_source": "codex" if args.codex_login else "file" if args.provider_config else "manual", + "credential_source": "chatgpt_plan" if args.chatgpt_plan else "codex" if args.codex_login else "file" if args.provider_config else "manual", "config_path": str(args.provider_config) if args.provider_config else None, - "api_key": None if args.codex_login else os.environ.get("OPENAI_API_KEY"), + "api_key": None if args.codex_login or args.chatgpt_plan else os.environ.get("OPENAI_API_KEY"), "model": args.model, } args.output.mkdir(parents=True, exist_ok=True) diff --git a/src/maniloop/controllers/target.py b/src/maniloop/controllers/target.py index 7ab627f..eb46922 100644 --- a/src/maniloop/controllers/target.py +++ b/src/maniloop/controllers/target.py @@ -19,8 +19,8 @@ class TargetSettings: class PoseTarget: """Capture the target once; produce bounded deltas relative to current sensors.""" - def __init__(self, sensors, position_delta, rotation_delta, grip, *, gripper=False): - self.settings = TargetSettings() + def __init__(self, sensors, position_delta, rotation_delta, grip, *, gripper=False, settings=None): + self.settings = settings if settings is not None else TargetSettings() self.start = np.array(sensors["tcp_position"], dtype=float) self.previous_position = self.start.copy() self.previous_rotation = np.array(sensors["tcp_rotation_matrix"], dtype=float) @@ -78,5 +78,5 @@ def update(self, sensors, interval=0.05, interrupted=False): "control_steps": self.steps, "gripper_opening": opening, "gripper_speed_s": gripper_speed} @staticmethod - def description(): - return asdict(TargetSettings()) + def description(settings=None): + return asdict(settings if settings is not None else TargetSettings()) diff --git a/src/maniloop/evaluation/benchmark.py b/src/maniloop/evaluation/benchmark.py index 2e40931..553ce60 100644 --- a/src/maniloop/evaluation/benchmark.py +++ b/src/maniloop/evaluation/benchmark.py @@ -100,8 +100,10 @@ def validate(self): raise ValueError( "Local checkpoints require LIBERO and a supported model/device" ) - if self.llm_control not in ("osc_step", "tcp_target_servo_v2") or self.observation_profile not in ("llm_rgb512", "debug_rgb128"): + if self.llm_control not in ("osc_step", "tcp_target_servo_v2", "tcp_target_servo_v3") or self.observation_profile not in ("llm_rgb512", "debug_rgb128"): raise ValueError("Unknown LLM control / observation profile") + if self.llm_control == "tcp_target_servo_v3" and (self.backend != "libero" or self.agent != "llm_cloud"): + raise ValueError("tcp_target_servo_v3 requires a LIBERO LLM policy") if type(self.request_timeout_seconds) not in (int, float) or not 0 < self.request_timeout_seconds <= 600: raise ValueError("Request timeout must be 0–600 seconds") if self.reasoning_effort not in ("auto", "low", "medium", "high", "xhigh"): diff --git a/src/maniloop/providers/chatgpt_auth.py b/src/maniloop/providers/chatgpt_auth.py new file mode 100644 index 0000000..d8ba730 --- /dev/null +++ b/src/maniloop/providers/chatgpt_auth.py @@ -0,0 +1,552 @@ +"""Independent, local Sign in with ChatGPT authorization for ManiLoop. + +Uses the documented OSS OAuth flow, not Codex's credential files or backend API. +Credentials never leave this module except through access_token(), for the fixed +official Responses endpoint. All public views and exceptions are allowlisted. +Official protocol checked 2026-10-28: +https://developers.openai.com/siwc/token-sharing-open-source/sign-in +https://developers.openai.com/siwc/token-sharing-open-source/profiles-and-sessions +""" + +from __future__ import annotations + +import base64 +from contextlib import contextmanager +import hashlib +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer +import json +import math +import os +from pathlib import Path +import secrets +import stat +import tempfile +import threading +import time +from urllib.error import HTTPError, URLError +from urllib.parse import parse_qs, urlencode, urlsplit +from urllib.request import HTTPRedirectHandler, Request, build_opener +import uuid + +ISSUER = "https://auth.openai.com" +RESOURCE = "https://api.openai.com/v1" +AUTHORIZE_URL = ISSUER + "/api/accounts/authorize" +TOKEN_URL = ISSUER + "/api/accounts/oauth/token" +DISCOVERY_URL = ISSUER + "/.well-known/openid-configuration" +SCOPES = "openid profile email offline_access resource.invoke chatgpt.tokens.use.direct" +REQUIRED_PLAN_SCOPES = {"resource.invoke", "chatgpt.tokens.use.direct"} +_TERMINAL_REFRESH_ERRORS = { + "invalid_grant", "invalid_refresh_token", "token_expired", + "refresh_token_expired", "refresh_token_invalidated", "refresh_token_reused", +} +_TOKEN_FIELDS = ("access_token", "refresh_token", "id_token", "expires_at", "earliest_refresh_at") +_MAX_BYTES = 1024 * 1024 + + +class ChatGPTAuthError(ValueError): + """Safe-to-display error; never includes provider text or credential data.""" + + def __init__(self, message, *, category="authentication", code=None): + super().__init__(message) + self.category = category + self.code = code if code in _TERMINAL_REFRESH_ERRORS | {"invalid_client"} else None + + +class _NoRedirect(HTTPRedirectHandler): + def redirect_request(self, req, fp, code, msg, headers, newurl): + return None + + +def _official_auth_url(value): + try: + parts = urlsplit(value) + valid = (parts.scheme == "https" and parts.netloc == "auth.openai.com" + and not parts.query and not parts.fragment and parts.path.startswith("/")) + except (ValueError, TypeError): + valid = False + if not valid: + raise ChatGPTAuthError("OpenAI 认证服务地址无效。", category="configuration") + return value + + +def _http_json(method, url, *, data=None): + """Bounded requests; normal OS proxy support, no credential-bearing redirects.""" + _official_auth_url(url) + body = urlencode(data).encode() if data is not None else None + headers = {"Accept": "application/json", "User-Agent": "ManiLoop/ChatGPTPlan"} + if body is not None: + headers["Content-Type"] = "application/x-www-form-urlencoded" + request = Request(url, data=body, headers=headers, method=method) + try: + with build_opener(_NoRedirect()).open(request, timeout=20) as response: + raw = response.read(_MAX_BYTES + 1) + if len(raw) > _MAX_BYTES: + raise ChatGPTAuthError("OpenAI 认证响应过大。") + value = json.loads(raw) if raw else {} + if not isinstance(value, dict): + raise ChatGPTAuthError("OpenAI 认证响应格式无效。") + return value + except HTTPError as exc: + code = None + try: + payload = json.loads(exc.read(_MAX_BYTES)) + if isinstance(payload, dict): + code = payload.get("error") + if isinstance(code, dict): + code = code.get("code") + if not isinstance(code, str): + code = None + except (ValueError, OSError): + pass + raise ChatGPTAuthError("OpenAI 认证请求未完成,请重试或重新授权。", code=code) from None + except (URLError, OSError, ValueError): + raise ChatGPTAuthError("无法连接 OpenAI 认证服务,请检查网络后重试。", category="connection") from None + + +def _string(value, *, maximum=32768): + return isinstance(value, str) and 0 < len(value) <= maximum and not any(ord(c) < 32 for c in value) + + +def _number(value): + return type(value) in (int, float) and math.isfinite(value) + + +class ChatGPTAuthStore: + def __init__(self, directory=None, *, http=None, clock=None, validate_id_token=None): + selected = directory or os.environ.get("MANILOOP_CHATGPT_AUTH_DIR") + self.directory = Path(selected).expanduser().absolute() if selected else Path.home() / ".config/maniloop/chatgpt" + self._http = http or _http_json + self._clock = clock or time.time + self._validator = validate_id_token + self._mutex = threading.RLock() + self._pending = None + self._server = None + self._error = "" + self._generation = 0 + self._discovery_cache = None + self._retry_client = None + + @contextmanager + def _locked(self): + """Serialize read/modify/refresh across processes as well as threads.""" + with self._mutex: + try: + if self.directory.is_symlink(): + raise OSError("symlink") + self.directory.mkdir(parents=True, exist_ok=True, mode=0o700) + if os.name != "nt": + self.directory.chmod(0o700) + lock_path = self.directory / "session.lock" + fd = os.open(lock_path, os.O_CREAT | os.O_RDWR | getattr(os, "O_NOFOLLOW", 0), 0o600) + if os.name != "nt": + os.fchmod(fd, 0o600) + handle = os.fdopen(fd, "r+b") + except OSError: + raise ChatGPTAuthError("无法访问 ManiLoop 的本地认证存储。", category="configuration") from None + try: + if os.name == "nt": + import msvcrt + if os.fstat(handle.fileno()).st_size == 0: + handle.write(b"\0") + handle.flush() + handle.seek(0) + msvcrt.locking(handle.fileno(), msvcrt.LK_LOCK, 1) + else: + import fcntl + fcntl.flock(handle.fileno(), fcntl.LOCK_EX) + yield + except OSError: + raise ChatGPTAuthError("无法更新 ManiLoop 的本地认证存储。", category="configuration") from None + finally: + if os.name == "nt": + import msvcrt + handle.seek(0) + try: + msvcrt.locking(handle.fileno(), msvcrt.LK_UNLCK, 1) + except OSError: + pass + else: + import fcntl + fcntl.flock(handle.fileno(), fcntl.LOCK_UN) + handle.close() + + def _read(self): + path = self.directory / "sessions.json" + try: + if not path.exists() and not path.is_symlink(): + return {"version": 1, "host_id": "urn:uuid:" + str(uuid.uuid4()), "accounts": {}, "active": None} + if path.is_symlink() or not path.is_file() or path.stat().st_size > _MAX_BYTES: + raise ValueError + if os.name != "nt" and stat.S_IMODE(path.stat().st_mode) & 0o077: + raise ValueError + value = json.loads(path.read_text(encoding="utf-8")) + if (not isinstance(value, dict) or value.get("version") != 1 + or not _string(value.get("host_id"), maximum=256) + or not isinstance(value.get("accounts"), dict) + or len(value["accounts"]) > 100): + raise ValueError + for key, record in value["accounts"].items(): + if (not _string(key, maximum=64) or not isinstance(record, dict) + or not _string(record.get("client_id"), maximum=256) + or not _string(record.get("subject"), maximum=256) + or not isinstance(record.get("scopes", []), list)): + raise ValueError + if value.get("active") is not None and value["active"] not in value["accounts"]: + raise ValueError + return value + except (OSError, ValueError, TypeError): + raise ChatGPTAuthError("ManiLoop 认证存储格式或权限无效;未覆盖原记录。", category="configuration") from None + + def _write(self, state): + fd, name = tempfile.mkstemp(prefix=".sessions-", dir=self.directory) + try: + if os.name != "nt": + os.fchmod(fd, 0o600) + with os.fdopen(fd, "w", encoding="utf-8") as stream: + json.dump(state, stream, ensure_ascii=False) + stream.flush() + os.fsync(stream.fileno()) + os.replace(name, self.directory / "sessions.json") + finally: + if os.path.exists(name): + os.unlink(name) + + @staticmethod + def _public_account(key, record): + email = record.get("email") + label = email if _string(email, maximum=254) else "ChatGPT 账号" + return {"id": key, "label": f"{label} · {key[:6]}", + "plan_enabled": REQUIRED_PLAN_SCOPES.issubset(record.get("scopes", [])), + "logged_in": bool(record.get("access_token")), + "reauth_required": not bool(record.get("access_token"))} + + def status(self): + with self._locked(): + state = self._read() + pending = self._pending + expired = pending is not None and pending["expires_at"] <= self._clock() + if expired: + self._pending = None + self._generation += 1 + self._error = "授权已超时,请重新开始。" + accounts = [self._public_account(key, record) for key, record in state["accounts"].items()] + active = next((account for account in accounts if account["id"] == state.get("active")), None) + result = {"accounts": accounts, "active_account_id": state.get("active"), + "pending": self._pending is not None, "error": self._error, + "registration_retry_available": bool(self._retry_client), + "logged_in": bool(active and active["logged_in"]), + "plan_enabled": bool(active and active["logged_in"] and active["plan_enabled"])} + if expired: + self._stop_listener(pending["server"]) + return result + + def _discovery(self): + if self._discovery_cache is None: + data = self._http("GET", DISCOVERY_URL) + if data.get("issuer") != ISSUER: + raise ChatGPTAuthError("OpenAI 身份提供者验证失败。") + for field in ("jwks_uri", "revocation_endpoint"): + _official_auth_url(data.get(field)) + self._discovery_cache = data + return self._discovery_cache + + def _validate_identity(self, token, client_id, nonce): + if not _string(token): + raise ChatGPTAuthError("OpenAI 未返回有效的身份凭据。") + if self._validator: + claims = self._validator(token, client_id, nonce) + else: + try: + import jwt + except ImportError: + raise ChatGPTAuthError("请先安装 ManiLoop 的 chatgpt 可选依赖以验证登录。", category="configuration") from None + try: + header = jwt.get_unverified_header(token) + if header.get("alg") not in {"RS256", "ES256"}: + raise ValueError + keys = self._http("GET", self._discovery()["jwks_uri"]).get("keys") + if not isinstance(keys, list) or len(keys) > 100: + raise ValueError + matches = [key for key in keys if isinstance(key, dict) and key.get("kid") == header.get("kid") + and key.get("use", "sig") == "sig" and key.get("alg", header["alg"]) == header["alg"]] + if len(matches) != 1: + raise ValueError + key = jwt.PyJWK.from_dict(matches[0], algorithm=header["alg"]) + claims = jwt.decode(token, key.key, algorithms=[header["alg"]], issuer=ISSUER, + audience=client_id, options={"require": ["iss", "aud", "exp", "sub", "nonce"]}) + except (ValueError, KeyError, TypeError, jwt.PyJWTError): + raise ChatGPTAuthError("OpenAI 身份凭据验证失败,未保存登录。") from None + audience = claims.get("aud") if isinstance(claims, dict) else None + if (not isinstance(claims, dict) or claims.get("iss") != ISSUER + or not (audience == client_id or isinstance(audience, list) and client_id in audience) + or not _number(claims.get("exp")) or claims["exp"] <= self._clock() + or not _string(claims.get("sub"), maximum=256) + or not isinstance(claims.get("nonce"), str) + or not secrets.compare_digest(claims["nonce"], nonce) + or isinstance(audience, list) and len(audience) > 1 and claims.get("azp") != client_id): + raise ChatGPTAuthError("OpenAI 身份凭据验证失败,未保存登录。") + return claims + + def start_login(self, account_id=None, *, retry_registration=False): + self.cancel_login() + if self._validator is None: + try: + import jwt # noqa: F401: fail before asking the user to authorize + except ImportError: + raise ChatGPTAuthError("请先安装 ManiLoop 的 chatgpt 可选依赖。", category="configuration") from None + with self._locked(): + state = self._read() + record = state["accounts"].get(account_id) if account_id else None + if account_id is not None and record is None: + raise ChatGPTAuthError("请选择已保存的 ChatGPT 账号。", category="configuration") + self._write(state) # persist a stable host ID before the first browser visit + owner = self + + class CallbackHandler(BaseHTTPRequestHandler): + def log_message(self, *_args): + pass # authorization URLs contain codes; never log them + + def do_GET(self): + expected_host = f"127.0.0.1:{self.server.server_port}" + if self.headers.get("Host") != expected_host: + self.send_error(400) + return + if urlsplit(self.path).path != "/auth/callback": + self.send_error(404) + return + try: + owner.handle_callback(self.path) + status, message = 200, "ChatGPT 授权完成。请返回 ManiLoop 工作台。" + except ChatGPTAuthError: + status, message = 400, "ChatGPT 授权未完成。请返回 ManiLoop 工作台查看状态。" + body = ("ManiLoop" + "

" + message + "

").encode() + self.send_response(status) + self.send_header("Content-Type", "text/html; charset=utf-8") + self.send_header("Content-Length", str(len(body))) + self.send_header("Cache-Control", "no-store") + self.send_header("Referrer-Policy", "no-referrer") + self.send_header("Content-Security-Policy", "default-src 'none'; frame-ancestors 'none'") + self.end_headers() + self.wfile.write(body) + + try: + server = ThreadingHTTPServer(("127.0.0.1", 0), CallbackHandler) + except OSError: + raise ChatGPTAuthError("无法启动本机授权回调,请重试。", category="connection") from None + server.daemon_threads = True + self._server = server + verifier = secrets.token_urlsafe(48) + nonce = secrets.token_urlsafe(32) + oauth_state = secrets.token_urlsafe(32) + redirect_uri = f"http://127.0.0.1:{server.server_port}/auth/callback" + if retry_registration and (record is not None or not self._retry_client): + server.server_close() + self._server = None + raise ChatGPTAuthError("没有可重试的注册,请重新开始登录。") + client_id = record["client_id"] if record else ( + self._retry_client if retry_registration else "dynamic_agent_client") + self._pending = {"state": oauth_state, "nonce": nonce, "verifier": verifier, + "redirect_uri": redirect_uri, "client_id": client_id, + "account_id": account_id, "expires_at": self._clock() + 600, + "generation": self._generation, "server": server} + self._error = "" + params = {"client_id": client_id, "ext_agent_host_id": state["host_id"], + "response_type": "code", "redirect_uri": redirect_uri, "scope": SCOPES, + "resource": RESOURCE, "state": oauth_state, "nonce": nonce, + "code_challenge_method": "S256", + "code_challenge": base64.urlsafe_b64encode(hashlib.sha256(verifier.encode()).digest()).decode().rstrip("=")} + if client_id == "dynamic_agent_client": + params["agent_name_hint"] = "ManiLoop" + if record and not REQUIRED_PLAN_SCOPES.issubset(record.get("scopes", [])): + params["prompt"] = "consent" + threading.Thread(target=server.serve_forever, daemon=True, name="maniloop-oauth-callback").start() + return {"authorization_url": AUTHORIZE_URL + "?" + urlencode(params), "expires_in": 600} + + def _token_values(self, response, *, old=None): + if (not isinstance(response, dict) or not _string(response.get("access_token")) + or not isinstance(response.get("token_type"), str) + or response["token_type"].lower() != "bearer" + or not _number(response.get("expires_in")) or not 0 < response["expires_in"] <= 86400): + raise ChatGPTAuthError("OpenAI 返回的授权凭据不完整。") + refresh = response.get("refresh_token") + if refresh is not None and not _string(refresh): + raise ChatGPTAuthError("OpenAI 返回的续期凭据无效。") + if old is not None and not refresh: + raise ChatGPTAuthError("OpenAI 未返回轮换后的续期凭据。") + scope = response.get("scope") + if scope is None and old is not None: + scopes = list(old["scopes"]) + elif isinstance(scope, str) and len(scope) <= 4096: + scopes = sorted(set(scope.split())) + else: + raise ChatGPTAuthError("OpenAI 未返回有效的授权范围。") + values = {"access_token": response["access_token"], "refresh_token": refresh, + "scopes": scopes, "expires_at": self._clock() + response["expires_in"]} + earliest = response.get("earliest_refresh_at") + if earliest is not None: + if not _number(earliest): + raise ChatGPTAuthError("OpenAI 返回的续期时间无效。") + values["earliest_refresh_at"] = earliest + return values + + def handle_callback(self, callback): + """Used by the private listener and offline tests; callback never enters logs.""" + pending = None + try: + if not isinstance(callback, str) or len(callback) > 32768: + raise ChatGPTAuthError("授权回调无效。") + parsed = urlsplit(callback) + params = parse_qs(parsed.query, keep_blank_values=True, max_num_fields=20) + if parsed.path != "/auth/callback" or any(len(values) != 1 for values in params.values()): + raise ChatGPTAuthError("授权回调无效。") + with self._mutex: + candidate = self._pending + supplied_state = params.get("state", [""])[0] + if (candidate is None or candidate["expires_at"] <= self._clock() + or candidate.get("consumed") + or not secrets.compare_digest(candidate["state"], supplied_state)): + raise ChatGPTAuthError("授权回调已过期或不匹配,请重新开始。") + pending = candidate + pending["consumed"] = True # no replay; still pending while exchange/validation run + if params.get("error"): + raise ChatGPTAuthError("未完成 ChatGPT 授权,可在工作台重新登录。") + code = params.get("code", [""])[0] + client = params.get("client_id", [pending["client_id"]])[0] + if (not _string(code, maximum=8192) or not _string(client, maximum=256) + or client == "dynamic_agent_client" + or pending["client_id"] != "dynamic_agent_client" and client != pending["client_id"]): + raise ChatGPTAuthError("ChatGPT 注册信息不完整或不匹配。") + if pending["client_id"] == "dynamic_agent_client": + self._retry_client = client + response = self._http("POST", TOKEN_URL, data={"grant_type": "authorization_code", + "client_id": client, "code": code, "code_verifier": pending["verifier"], + "redirect_uri": pending["redirect_uri"], "resource": RESOURCE}) + values = self._token_values(response) + claims = self._validate_identity(response.get("id_token"), client, pending["nonce"]) + key = hashlib.sha256((ISSUER + "\0" + client + "\0" + claims["sub"]).encode()).hexdigest()[:24] + with self._locked(): + if pending["generation"] != self._generation: + raise ChatGPTAuthError("授权已取消,未保存登录。") + state = self._read() + existing = state["accounts"].get(pending["account_id"]) + if pending["account_id"] is not None and (existing is None or existing["subject"] != claims["sub"] + or existing["client_id"] != client): + raise ChatGPTAuthError("授权账号与所选账号不一致,未覆盖原登录。") + email = claims.get("email") + state["accounts"][key] = {"client_id": client, "subject": claims["sub"], + "email": email if _string(email, maximum=254) else "", "id_token": response["id_token"], **values} + state["active"] = key + self._write(state) + self._retry_client = None + self._error = "" if REQUIRED_PLAN_SCOPES.issubset(values["scopes"]) else "已登录,但尚未授权使用 ChatGPT 套餐。" + except ChatGPTAuthError as exc: + if pending is not None: + with self._mutex: + if pending["generation"] == self._generation: + self._error = str(exc) + raise + except (ValueError, TypeError, KeyError, OSError): + if pending is not None: + with self._mutex: + if pending["generation"] == self._generation: + self._error = "ChatGPT 授权未完成,请重新登录。" + raise ChatGPTAuthError("ChatGPT 授权未完成,请重新登录。") from None + finally: + if pending is not None: + with self._mutex: + if self._pending is pending: + self._pending = None + self._stop_listener(pending["server"]) + + def _stop_listener(self, expected=None): + with self._mutex: + server = expected or self._server + if self._server is server: + self._server = None + if server is not None: + server.shutdown() + server.server_close() + + def cancel_login(self): + with self._mutex: + self._pending = None + self._generation += 1 + self._stop_listener() + + def select_account(self, account_id): + self.cancel_login() + with self._locked(): + state = self._read() + if not isinstance(account_id, str) or account_id not in state["accounts"]: + raise ChatGPTAuthError("请选择已保存的 ChatGPT 账号。", category="configuration") + state["active"] = account_id + self._write(state) + self._error = "" + return self.status() + + def access_token(self, account_id=None): + """Return a token to the official transport; never serialize this result.""" + with self._locked(): + state = self._read() + key = account_id or state.get("active") + record = state["accounts"].get(key) + if not record or not record.get("access_token"): + raise ChatGPTAuthError("请先为 ManiLoop 登录 ChatGPT。") + if not REQUIRED_PLAN_SCOPES.issubset(record.get("scopes", [])): + raise ChatGPTAuthError("此账号尚未授权 ManiLoop 使用 ChatGPT 套餐。") + expiry = record.get("expires_at", 0) + if not _number(expiry): + raise ChatGPTAuthError("本地授权时间无效,请重新登录。") + now = self._clock() + if expiry > now + 60: + return record["access_token"] + earliest = record.get("earliest_refresh_at", 0) + if _number(earliest) and earliest > now: + if expiry > now: + return record["access_token"] + raise ChatGPTAuthError("授权尚不能续期,请稍后重试。") + if not record.get("refresh_token"): + raise ChatGPTAuthError("授权已过期,请重新登录 ChatGPT。") + try: + response = self._http("POST", TOKEN_URL, data={"grant_type": "refresh_token", + "client_id": record["client_id"], "refresh_token": record["refresh_token"], "resource": RESOURCE}) + values = self._token_values(response, old=record) + except ChatGPTAuthError as exc: + if exc.code in _TERMINAL_REFRESH_ERRORS: + for field in _TOKEN_FIELDS: + record.pop(field, None) + self._write(state) + self._error = "ChatGPT 授权已失效,请重新登录。" + raise ChatGPTAuthError(self._error) from None + raise + # Retain the ID token whose nonce/identity was validated at sign-in. + record.update(values) + self._write(state) + if not REQUIRED_PLAN_SCOPES.issubset(record["scopes"]): + raise ChatGPTAuthError("ChatGPT 套餐授权已变更,请重新授权。") + return record["access_token"] + + def logout(self, account_id=None): + self.cancel_login() + confirmed = True + with self._locked(): + state = self._read() + key = account_id or state.get("active") + record = state["accounts"].get(key) + if record: + refresh = record.get("refresh_token") + if refresh: + try: + self._http("POST", self._discovery()["revocation_endpoint"], data={ + "token": refresh, "token_type_hint": "refresh_token", "client_id": record["client_id"]}) + except ChatGPTAuthError: + confirmed = False + for field in _TOKEN_FIELDS: + record.pop(field, None) + self._write(state) + self._error = "" if confirmed else "已清除本地登录,远端撤销未确认;可在 ChatGPT 设置中断开 ManiLoop。" + return {**self.status(), "revocation_confirmed": confirmed} + + def close(self): + self.cancel_login() diff --git a/src/maniloop/providers/chatgpt_plan.py b/src/maniloop/providers/chatgpt_plan.py new file mode 100644 index 0000000..cd4dfae --- /dev/null +++ b/src/maniloop/providers/chatgpt_plan.py @@ -0,0 +1,492 @@ +"""Direct official ChatGPT-plan Responses transport, without a Codex process. + +Only the app's dedicated OAuth store supplies tokens. No API-key fallback, custom +endpoint, organization headers or hidden conversation continuation is supported. +""" +from __future__ import annotations + +import codecs +import hashlib +import json +import math +import re +import socket +import threading +import time + +import httpx + +from .policy_request import request_manifest +from .responses import DEFAULT_MODEL, GPTPolicy, PolicyError, _reject_constant, _unique_object + +API_BASE = "https://api.openai.com/v1" +MAX_STREAM_BYTES = 4 * 1024 * 1024 +_ALLOWED_OPTIONS = {"timeout_seconds", "reasoning_effort", "client_max_response_chars"} +_ALLOWED_REQUEST = {"model", "instructions", "input", "text", "store", "reasoning"} +_UNSUPPORTED = ["max_output_tokens", "temperature", "previous_response_id"] + + +# Exact documented codes only; free-form error text and unknown codes never enter logs. +# https://developers.openai.com/siwc/token-sharing-open-source/errors-and-recovery +_PLAN_ERRORS = { + "subscription_sharing_user_not_eligible": ("plan_not_eligible", "ChatGPT plan usage is unavailable for this account, workspace or policy. Check eligibility; repeated sign-in will not resolve this restriction."), + "subscription_sharing_usage_limit_exceeded": ("rate_limit", "The ChatGPT plan usage allowance for this app or account has been reached. Pause requests and check ChatGPT Settings > Usage; no reset time is inferred."), + "subscription_sharing_usage_unavailable": ("service_unavailable", "ChatGPT usage availability could not be checked. Keep the saved authorization and retry later."), + "subscription_sharing_unsupported_capability": ("unsupported_capability", "This ChatGPT plan route does not support a requested capability. Check the model, input, tools and execution settings before retrying."), + "subscription_sharing_route_not_supported": ("route_not_supported", "This ChatGPT plan route is not supported. Verify POST /v1/responses and the integration configuration."), + "subscription_sharing_invalid_user": ("authentication", "The subscriber context could not be validated. Check the selected account and granted permissions; sign in again only after confirmed revocation or a terminal refresh failure."), + "chatpass_v2_scope_not_authorized": ("authorization", "The signed grant does not authorize this operation. Check the client and granted scopes before retrying."), + "chatpass_v2_invalid_authorization_context": ("authorization", "The signed permission context is invalid for this operation. Check the client and grant configuration before retrying."), + "subscription_sharing_user_unavailable": ("service_unavailable", "ChatGPT account or workspace information is temporarily unavailable. Keep the saved authorization and retry later."), +} +for _legacy_quota in ("rate_limit_exceeded", "usage_limit_reached", "insufficient_quota", "quota_exceeded"): + _PLAN_ERRORS[_legacy_quota] = ("rate_limit", "ChatGPT usage or rate limits prevented this request. Pause new requests and check the account's usage status.") +_SAFE_ERROR_PARAMS = frozenset({"model", "input", "tools", "tool_choice", "service_tier", "stream", "store", "instructions", "text", "reasoning", "reasoning.effort", "max_output_tokens", "temperature", "previous_response_id"}) +_DIAGNOSTIC_TOP_FIELDS = frozenset({"id", "object", "status", "model", "output", "usage", "error", + "detail", "type", "code", "message", "incomplete_details"}) +_RESPONSE_STATUSES = frozenset({"completed", "failed", "incomplete", "in_progress", "queued", "cancelled"}) +_EVENT_TYPE_PATTERN = re.compile(r"[a-z][a-z0-9_.-]{0,79}") +_MIME_PATTERN = re.compile(r"[!#$%&'*+.^_`|~0-9A-Za-z-]+/[!#$%&'*+.^_`|~0-9A-Za-z-]+") + + +def _media_type(value): + if value is None or value == "": + return "missing" + if not isinstance(value, str) or len(value) > 512 or any(ord(c) < 32 or ord(c) > 126 for c in value): + return "other" + media = value.partition(";")[0].strip().lower() + return media if len(media) <= 128 and _MIME_PATTERN.fullmatch(media) else "other" + + +def _prefix_framing(raw): + data = bytes(raw).removeprefix(b"\xef\xbb\xbf").lstrip() + # SSE may begin with blank lines or comments/heartbeats. + for line in data.splitlines(): + if not line or line.startswith(b":"): + continue + if line.startswith(b"data:"): + return "sse_data" + if line.startswith(b"event:"): + return "sse_event" + break + lower = data.lower() + if lower.startswith((b" 65536: + break + raw.extend(chunk) + value = json.loads(raw) + except Exception: + value = None + code, param = _error_fields(value) + return _plan_error(code, status=response.status_code, param=param) + + +def _token(auth_store, account_id): + try: + value = auth_store.access_token(account_id) + except Exception as exc: + category = getattr(exc, "category", "authentication") + if category in {"connection", "timeout"}: + raise PolicyError("ChatGPT authorization service is temporarily unavailable; check the connection and retry.", category=category) from None + if category == "configuration": + raise PolicyError("ChatGPT authorization is not configured for the selected account.", category="configuration") from None + raise PolicyError("ChatGPT authorization is unavailable; sign in again.", category="authentication") from None + if not isinstance(value, str) or not value or any(c.isspace() or ord(c) < 32 for c in value): + raise PolicyError("ChatGPT authorization is unavailable; sign in again.", category="authentication") + return value + + +def list_plan_models(auth_store, account_id=None): + """Only explicitly listed, visible slugs; no guesses from API-key catalogs.""" + token = _token(auth_store, account_id) + try: + # trust_env permits standard network proxies, never changes this URL or headers. + with httpx.Client(timeout=20, follow_redirects=False) as client: + response = client.get(API_BASE + "/models", headers={"Authorization": "Bearer " + token}) + if response.status_code != 200: + raise _http_error(response) + if len(response.content) > MAX_STREAM_BYTES: + raise PolicyError("ChatGPT model catalog exceeded the size limit.") + data = response.json() + models = data.get("models") if isinstance(data, dict) else None + if not isinstance(models, list): + raise PolicyError("ChatGPT returned an unsupported model catalog.") + result = [] + for item in models: + if not isinstance(item, dict) or item.get("visibility") != "list": + continue + slug = item.get("slug") + if isinstance(slug, str) and slug.strip() == slug and 0 < len(slug) <= 120 and not any(ord(c) < 32 for c in slug): + if slug not in result: + result.append(slug) + return result + except PolicyError: + raise + except httpx.TimeoutException: + raise PolicyError("ChatGPT model catalog request timed out.", category="timeout") from None + except Exception: + raise PolicyError("ChatGPT model catalog request failed.", category="connection") from None + + +class ChatGPTPlanPolicy(GPTPolicy): + def __init__(self, model=None, *, auth_store, account_id=None, request_options=None): + self.model = model or DEFAULT_MODEL + if not isinstance(self.model, str) or not self.model.strip() or len(self.model) > 120 or any(ord(c) < 32 for c in self.model): + raise PolicyError("Invalid ChatGPT model name.", category="configuration") + if request_options is not None and type(request_options) is not dict: + raise PolicyError("Invalid ChatGPT request options.", category="configuration") + options = dict(request_options or {}) + self._requested_options = dict(options) + if "max_output_tokens" in options or "server_max_output_tokens" in options: + raise PolicyError("ChatGPT plan inference does not support a server token cap; use the separate client response-size limit.", category="configuration") + if set(options) - _ALLOWED_OPTIONS: + raise PolicyError("Unsupported ChatGPT request option.", category="configuration") + self.timeout = options.get("timeout_seconds", 120) + self.reasoning_effort = options.get("reasoning_effort", "low") + if self.reasoning_effort in (None, "auto"): + self.reasoning_effort = "low" + self.client_max_response_chars = options.get("client_max_response_chars", 65536) + if (type(self.timeout) not in (int, float) or not math.isfinite(self.timeout) + or not 0 < self.timeout <= 600 or type(self.reasoning_effort) is not str or self.reasoning_effort not in {"low", "medium", "high", "xhigh", "max"} + or type(self.client_max_response_chars) is not int or not 256 <= self.client_max_response_chars <= 524288): + raise PolicyError("Invalid ChatGPT request options.", category="configuration") + self.max_output_tokens = None # Intentionally omitted by the common encoder. + self.auth_store, self.account_id = auth_store, account_id + self.last_usage, self.last_latency, self.last_response_id = {}, 0.0, None + self.last_request_manifest = None + self._lock = threading.Lock() + self._cancelled = threading.Event() + self._deadline_reached = threading.Event() + self._deadline_at = None + self._response = None + self._client = None + self._worker = None + + @property + def transport_busy(self): + """True until the I/O worker exits, including after caller cancellation. + + Owners must retain the policy and block replacement requests while this + is true; a returned timeout does not prove the transport has stopped. + """ + worker = self._worker + return worker is not None and worker.is_alive() + + @property + def request_options(self): + return {"transport": "chatgpt_plan_responses_v1", "timeout_seconds": self.timeout, + "reasoning_effort": self.reasoning_effort, + "client_max_response_chars": self.client_max_response_chars, + "server_max_output_tokens": None, "policy_retries": 0, "transport_retries": 0, + "stream": True, "store": False, "harness": "maniloop", "codex_process": False} + + def reset(self, seed=0): + if self.transport_busy: + raise PolicyError("The previous ChatGPT request is still stopping.", category="connection") + self._cancelled.clear() + + def _check_interrupted(self): + if self._cancelled.is_set(): + raise PolicyError("ChatGPT request was cancelled; no action was executed.", category="interrupted") + if self._deadline_reached.is_set() or (self._deadline_at is not None and time.monotonic() >= self._deadline_at): + self._deadline_reached.set() + raise PolicyError("ChatGPT request exceeded its deadline; no action was executed.", category="timeout") + + def _abort_transport(self): + with self._lock: + response, client = self._response, self._client + def close_resources(): + # Shutdown interrupts a blocked socket read when the transport exposes it. + if response is not None: + try: + stream = response.extensions.get("network_stream") + sock = stream.get_extra_info("socket") if stream else None + if sock is not None: + sock.shutdown(socket.SHUT_RDWR) + except Exception: + pass + for resource in (response, client): + if resource is not None: + try: + resource.close() + except Exception: + pass + # Transport close can itself contend with a connect; never block the UI or + # caller's cancellation/deadline on that operation. + threading.Thread(target=close_resources, daemon=True, name="plan-transport-close").start() + + def cancel(self): + self._cancelled.set() + self._abort_transport() + + def close(self): + self.cancel() + + def _request_text(self, request): + if self.transport_busy: + raise PolicyError("The previous ChatGPT request is still stopping; no new request was started.", category="connection") + self.last_usage, self.last_response_id = {}, None + self.last_response_model = None + self._deadline_reached.clear() + self._check_interrupted() + if set(request) - _ALLOWED_REQUEST or not isinstance(request.get("input"), list): + raise PolicyError("Unsupported ChatGPT plan request fields.", category="configuration") + request = dict(request) + # Text/vision diagnostics also use an explicit instruction, never an implicit session. + request.setdefault("instructions", "Answer only the explicit diagnostic request using the supplied observations.") + request.update(store=False, stream=True) + self.last_request_manifest = request_manifest(request, transport="chatgpt_plan_responses_v1", + requested=self._requested_options, unsupported=_UNSUPPORTED) + self.last_request_manifest["client_controls"] = {"deadline_seconds": self.timeout, + "max_response_chars": self.client_max_response_chars, + "policy_retries": 0, "transport_retries": 0} + started = time.monotonic() + self._deadline_at = started + self.timeout + finished = threading.Event() + result = {} + self._worker = threading.Thread(target=self._perform_request, args=(request, result, finished), + daemon=True, name="plan-response") + try: + self._worker.start() + # Unlike a socket read timeout this bounds the whole request, including + # OAuth refresh, connect and headers. Late worker results are discarded. + while not finished.wait(min(0.05, max(0.001, self._deadline_at - time.monotonic()))): + self._check_interrupted() + self._check_interrupted() + if "error" in result: + raise result["error"] + text = self._parse_response(result["response"]) + if len(text) > self.client_max_response_chars: + raise PolicyError("ChatGPT final response exceeded the client size limit.") + self._check_interrupted() + return text + except PolicyError as exc: + if isinstance(self.last_request_manifest, dict): + self.last_request_manifest["error"] = {"category": exc.category, "http_status": exc.http_status, + "code": exc.code, "param": getattr(exc, "param", None)} + if self._cancelled.is_set() or self._deadline_reached.is_set(): + self._abort_transport() + raise + except Exception: + raise PolicyError("ChatGPT returned an invalid response; no action was executed.") from None + finally: + self._deadline_at = None + self.last_latency = time.monotonic() - started + + def _perform_request(self, request, result, finished): + try: + self._check_interrupted() + token = _token(self.auth_store, self.account_id) + self._check_interrupted() + with httpx.Client(timeout=self.timeout, follow_redirects=False) as client: + with self._lock: + self._client = client + self._check_interrupted() + with client.stream("POST", API_BASE + "/responses", json=request, + headers={"Authorization": "Bearer " + token, "Accept": "text/event-stream"}) as response: + with self._lock: + self._response = response + self._check_interrupted() + media_type = _media_type(response.headers.get("content-type")) + self.last_request_manifest["http"] = {"http_status": response.status_code, + "media_type": media_type, + "declared_mime": media_type, + "parsed_format": "unverified"} + if response.status_code != 200: + raise _http_error(response) + # Some admitted direct-route streams omit or mislabel MIME. + # Validate the actual complete SSE protocol, never JSON fallback. + evidence = {"prefix": bytearray(), "truncated": False} + try: + result["response"] = self._read_stream(response, evidence) + self.last_request_manifest["http"]["parsed_format"] = "sse" + except PolicyError as exc: + if exc.category == "invalid_response" and media_type != "text/event-stream": + raise PolicyError(str(exc) + f" (declared media type: {media_type})", category=exc.category, + http_status=exc.http_status, code=exc.code) from None + raise + finally: + self.last_request_manifest["body_diagnostic"] = self._diagnose_prefix( + evidence["prefix"], evidence["truncated"]) + + except PolicyError as exc: + result["error"] = exc + except httpx.RemoteProtocolError: + result["error"] = PolicyError("ChatGPT connection closed before a complete HTTP response; no action was executed.", category="connection") + except httpx.TimeoutException: + result["error"] = PolicyError("ChatGPT request timed out; no action was executed.", category="timeout") + except Exception: + result["error"] = PolicyError("ChatGPT response stream failed; no action was executed.", category="connection") + finally: + with self._lock: + self._response = self._client = None + finished.set() + + def _diagnose_prefix(self, raw, truncated): + """Bounded shape-only evidence; raw bytes never enter the manifest.""" + result = {"prefix_bytes": len(raw), "prefix_sha256": hashlib.sha256(raw).hexdigest(), + "truncated": truncated, "is_json": False, "top_level_fields": [], + "object_is_response": False, "framing": _prefix_framing(raw)} + if not truncated: + try: + value = json.loads(raw, parse_constant=_reject_constant, object_pairs_hook=_unique_object) + except (ValueError, TypeError, RecursionError): + value = None + else: + result["is_json"] = True + if isinstance(value, dict): + result["top_level_fields"] = sorted(set(value) & _DIAGNOSTIC_TOP_FIELDS) + result["object_is_response"] = value.get("object") == "response" + status = value.get("status") + if type(status) is str and status in _RESPONSE_STATUSES: + result["response_status"] = status + code, _ = _error_fields(value) + if type(code) is str and code in _PLAN_ERRORS: + result["error_code"] = code + return result + + def _read_stream(self, response, evidence): + completed = None + data_lines = [] + saw_done = False + + def dispatch(): + nonlocal completed, saw_done + if not data_lines: + return + payload = "\n".join(data_lines) + data_lines.clear() + if payload == "[DONE]": + if completed is None or saw_done: + raise PolicyError("ChatGPT stream ended without one completed response.") + saw_done = True + return + try: + event = json.loads(payload, parse_constant=_reject_constant, object_pairs_hook=_unique_object) + except (ValueError, TypeError): + raise PolicyError("ChatGPT returned invalid stream data.") from None + kind = event.get("type") if isinstance(event, dict) else None + safe_kind = self._record_event_type(kind) + if safe_kind is None: + raise PolicyError("ChatGPT returned an invalid stream event type.") + kind = safe_kind + # Observed direct-route heartbeat. It conveys no model output and + # does not extend the request deadline, including after completion. + if kind == "keepalive": + return + if completed is not None or saw_done: + raise PolicyError("ChatGPT returned events after its terminal response.") + if kind in {"response.output_item.added", "response.output_item.done"}: + item = event.get("item") + if not isinstance(item, dict) or item.get("type") not in {"message", "reasoning"}: + raise PolicyError("ChatGPT returned a tool or unsupported output item.") + if kind.startswith(("response.function_call", "response.mcp", "response.web_search", "response.file_search", "response.code_interpreter")): + raise PolicyError("ChatGPT returned an unsupported tool event.") + if kind in {"error", "response.failed", "response.incomplete"}: + source = event.get("response") if isinstance(event.get("response"), dict) else event + code, param = _error_fields(source) + raise _plan_error(code, param=param) + if kind == "response.completed": + completed = event.get("response") + if not isinstance(completed, dict): + raise PolicyError("ChatGPT returned an invalid terminal response.") + elif not kind.startswith("response."): + raise PolicyError(f"ChatGPT returned an unsupported stream event: {kind}.") + + for line in self._stream_lines(response, evidence): + self._check_interrupted() + if line == "": + dispatch() + elif line.startswith("data:"): + data_lines.append(line[5:].removeprefix(" ")) + elif line.startswith((":", "event:", "id:", "retry:")): + continue + else: + raise PolicyError("ChatGPT returned invalid SSE framing.") + dispatch() + self._check_interrupted() + if completed is None: + raise PolicyError("ChatGPT stream disconnected before response.completed; no action was executed.") + return completed + + def _record_event_type(self, kind): + safe_kind = kind if type(kind) is str and _EVENT_TYPE_PATTERN.fullmatch(kind) else None + observed = self.last_request_manifest.setdefault("observed_event_types", []) + label = safe_kind if safe_kind is not None else "invalid" + if label not in observed: + if len(observed) < 64: + observed.append(label) + else: + self.last_request_manifest["event_type_list_truncated"] = True + if safe_kind is None: + # Unknown/arbitrary text stays out of manifests and exception messages. + digest = hashlib.sha256(json.dumps(kind, ensure_ascii=False, sort_keys=True, + separators=(",", ":")).encode()).hexdigest() + invalid = self.last_request_manifest.setdefault("invalid_event_type_sha256", []) + if digest not in invalid and len(invalid) < 8: + invalid.append(digest) + return safe_kind + + def _stream_lines(self, response, evidence): + # Bound raw bytes before buffering lines, including streams without newlines. + decoder = codecs.getincrementaldecoder("utf-8-sig")("strict") + buffered, size = "", 0 + for chunk in response.iter_bytes(): + self._check_interrupted() + size += len(chunk) + remaining = 65536 - len(evidence["prefix"]) + evidence["prefix"].extend(chunk[:remaining]) + if len(chunk) > remaining: + evidence["truncated"] = True + if size > MAX_STREAM_BYTES: + raise PolicyError("ChatGPT response stream exceeded the size limit.") + buffered += decoder.decode(chunk) + while "\n" in buffered: + line, buffered = buffered.split("\n", 1) + yield line.removesuffix("\r") + buffered += decoder.decode(b"", final=True) + if buffered: + yield buffered.removesuffix("\r") diff --git a/src/maniloop/providers/policy_request.py b/src/maniloop/providers/policy_request.py new file mode 100644 index 0000000..eafb02d --- /dev/null +++ b/src/maniloop/providers/policy_request.py @@ -0,0 +1,62 @@ +"""Shared sensor request encoding and credential-free provenance. + +This describes client-submitted data, not hidden provider instructions or actual +reasoning compute. Image hashes preserve order and identify exact source bytes. +""" +import base64 +import hashlib +import json + + +def _hash(value): + data = json.dumps(value, ensure_ascii=False, allow_nan=False, + sort_keys=True, separators=(",", ":")).encode("utf-8") + return hashlib.sha256(data).hexdigest() + + +def build_policy_request(*, model, instructions, task, observation, history, + geometry_results, images, schema, max_output_tokens, + reasoning_effort): + content = [{"type": "input_text", "text": json.dumps( + {"task": task, "observation": observation, "history": history[-12:], + "geometry_results": geometry_results[-8:]}, ensure_ascii=False, + allow_nan=False, separators=(",", ":"))}] + for camera, data in images.items(): + mime = "image/png" if data.startswith(b"\x89PNG") else "image/jpeg" + content.extend([ + {"type": "input_text", "text": f"Camera: {camera}. This labels the input image, not the action camera field."}, + {"type": "input_image", "image_url": f"data:{mime};base64," + base64.b64encode(data).decode("ascii"), "detail": "high"}, + ]) + request = {"model": model, "instructions": instructions, + "input": [{"role": "user", "content": content}], + "text": {"format": {"type": "json_schema", "name": "robot_action", + "strict": True, "schema": schema}}, "store": False} + if max_output_tokens is not None: + request["max_output_tokens"] = max_output_tokens + if reasoning_effort is not None: + request["reasoning"] = {"effort": reasoning_effort} + return request + + +def request_manifest(request, *, transport, requested=None, unsupported=None): + """No keys, account identifiers, image data, task text, or provider URLs.""" + sensor = {key: request[key] for key in ("instructions", "input", "text") if key in request} + images = [] + for message in request.get("input", []): + for part in message.get("content", []): + if part.get("type") == "input_image": + uri = part["image_url"] + header, data = uri.split(",", 1) + images.append({"sha256": hashlib.sha256(base64.b64decode(data, validate=True)).hexdigest(), + "mime": header.split(";")[0].removeprefix("data:"), + "detail": part.get("detail")}) + return {"version": "policy_request_v1", "transport": transport, + "model_requested": request.get("model"), + "sensor_request_sha256": _hash(sensor), + "instructions_sha256": _hash(request.get("instructions", "")), + "input_sha256": _hash(request.get("input", [])), + "schema_sha256": _hash(request.get("text", {})), "images": images, + "requested": dict(requested or {}), + "sent": {key: request[key] for key in ("reasoning", "max_output_tokens", "store", "stream") if key in request}, + "unsupported": list(unsupported or []), + "scope": "client_submitted_request; not provider internals or equal compute"} diff --git a/src/maniloop/providers/responses.py b/src/maniloop/providers/responses.py index bb9a8c4..83cc997 100644 --- a/src/maniloop/providers/responses.py +++ b/src/maniloop/providers/responses.py @@ -13,6 +13,7 @@ import json import math import os +import re import time from typing import Any @@ -27,6 +28,7 @@ DEFAULT_MODEL = "gpt-6-astra" from maniloop.core.observations import PUBLIC_OBSERVATION_FIELDS, guard_sensor_tree +from .policy_request import build_policy_request, request_manifest ACTION_SCHEMA = { "type": "object", @@ -156,10 +158,11 @@ def action_schema_for(observation: dict, image_names) -> dict: class PolicyError(RuntimeError): """A sanitized failure; metadata never contains provider response bodies.""" - def __init__(self, message, *, category="invalid_response", http_status=None): + def __init__(self, message, *, category="invalid_response", http_status=None, code=None): super().__init__(message) self.category = category self.http_status = http_status + self.code = code def _get(value: Any, key: str, default: Any = None) -> Any: @@ -400,56 +403,16 @@ def decide( ) ): raise PolicyError("At least one named, nonempty JPEG image is required.") - content = [ - { - "type": "input_text", - "text": _strict_json( - { - "task": task, - "observation": observation, - "history": history[-12:], - "geometry_results": geometry_results[-8:], - } - ), - } - ] - for camera, jpeg in images.items(): - content.append( - { - "type": "input_text", - "text": f"Camera: {camera}. This labels the input image, not the action camera field.", - } - ) - content.append( - { - "type": "input_image", - "image_url": ( - "data:image/png;base64," - if jpeg.startswith(b"\x89PNG") - else "data:image/jpeg;base64," - ) - + base64.b64encode(jpeg).decode("ascii"), - "detail": "high", - } - ) schema = action_schema_for(observation, images) - request = { - "model": self.model, - "instructions": SYSTEM_PROMPT, - "input": [{"role": "user", "content": content}], - "text": { - "format": { - "type": "json_schema", - "name": "robot_action", - "strict": True, - "schema": schema, - } - }, - "store": False, - "max_output_tokens": self.max_output_tokens, - } - if self.reasoning_effort is not None: - request["reasoning"] = {"effort": self.reasoning_effort} + try: + request = build_policy_request( + model=self.model, instructions=SYSTEM_PROMPT, task=task, + observation=observation, history=history, geometry_results=geometry_results, + images=images, schema=schema, max_output_tokens=self.max_output_tokens, + reasoning_effort=self.reasoning_effort, + ) + except (TypeError, ValueError, OverflowError, RecursionError): + raise PolicyError("Policy inputs must be finite, JSON-serializable sensor data.") from None text = self._request_text(request) try: action = json.loads( @@ -492,15 +455,19 @@ def diagnose(self, stage, task, observation, images): {"type": "input_image", "image_url": f"data:{mime};base64," + base64.b64encode(data).decode("ascii"), "detail": "high"}, ] request = {"model": self.model, "input": [{"role": "user", "content": content}], - "store": False, "max_output_tokens": self.max_output_tokens} + "store": False} + if self.max_output_tokens is not None: + request["max_output_tokens"] = self.max_output_tokens if self.reasoning_effort is not None: request["reasoning"] = {"effort": self.reasoning_effort} text = self._request_text(request) return {"stage": stage, "output": text[:1000], "executed": False} def _request_text(self, request): + self.last_request_manifest = request_manifest(request, transport="responses_api", requested=self.request_options) self.last_usage = {} self.last_response_id = None + self.last_response_model = None started = time.monotonic() try: response = self.client.responses.create(**request) @@ -532,6 +499,10 @@ def _request_text(self, request): finally: self.last_latency = time.monotonic() - started + return self._parse_response(response) + + def _parse_response(self, response): + """Shared terminal response validation; partial stream text is never an action.""" response_id = _get(response, "id") # IDs are opaque; compatible providers need not use OpenAI's resp_ prefix. if ( @@ -541,12 +512,27 @@ def _request_text(self, request): ): raise PolicyError("OpenAI returned a missing or invalid response ID.") self.last_response_id = response_id + reported_model = _get(response, "model") + self.last_response_model = None + if type(reported_model) is str and re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9._:/-]{0,255}", reported_model): + self.last_response_model = reported_model + manifest = getattr(self, "last_request_manifest", None) + if isinstance(manifest, dict): + manifest["model_reported"] = reported_model + manifest["model_reported_scope"] = "provider_identifier_not_snapshot_attestation" usage = _get(response, "usage") if usage is not None: for field in ("input_tokens", "output_tokens", "total_tokens"): count = _get(usage, field) if type(count) is int and count >= 0: self.last_usage[field] = count + for group, field, recorded in ( + ("output_tokens_details", "reasoning_tokens", "reasoning_tokens"), + ("input_tokens_details", "cached_tokens", "cached_input_tokens"), + ): + count = _get(_get(usage, group), field) + if type(count) is int and count >= 0: + self.last_usage[recorded] = count if _get(_get(response, "incomplete_details"), "reason") == "max_output_tokens": raise PolicyError( "模型输出达到长度上限,动作未完成。请简化任务或提高 OPENAI_MAX_OUTPUT_TOKENS 后重试。" diff --git a/src/maniloop/recording/manifest.py b/src/maniloop/recording/manifest.py index 37dbf17..c20765c 100644 --- a/src/maniloop/recording/manifest.py +++ b/src/maniloop/recording/manifest.py @@ -95,9 +95,12 @@ def describe_run(runner): if runner.policy_kind == "llm_cloud": effective = getattr(runner.policy, "request_options", None) policy["request_options"] = effective if isinstance(effective, dict) else {"source": "unavailable_adapter"} - if sim.backend == "libero" and sim.describe().get("llm_control") == "tcp_target_servo_v2": - policy["action_interface"] = "tcp_target_servo_v2" - policy["target_controller"] = sim.describe()["target_controller"] + if sim.backend == "libero" and sim.describe().get("llm_control") in ("tcp_target_servo_v2", "tcp_target_servo_v3"): + info = sim.describe() + policy["action_interface"] = info["llm_control"] + policy["target_controller"] = info["target_controller"] + if info["llm_control"] == "tcp_target_servo_v3": + policy["target_command_limits"] = info["target_command_limits"] if runner.policy_kind == "jev": policy.update(action_interface="jev_primitives_v1", representation="text_proprioception_v1", endpoint="https://api.typesafe.ai/v1/systemone", diff --git a/src/maniloop/runtime/runner.py b/src/maniloop/runtime/runner.py index 89e01d5..e032ac2 100644 --- a/src/maniloop/runtime/runner.py +++ b/src/maniloop/runtime/runner.py @@ -124,7 +124,9 @@ def __init__( self.pending_observation, self.request_images = sim.observe() self.api_latency = None self.policy = None + self._closing_policies = [] self.request_options = {} + self._chatgpt_auth = None self.typesafe_key = "" self.policy_decision = None self.diagnostic_stage = None @@ -162,7 +164,7 @@ def stop(self, message="任务已停止", reason="user_stop"): self.phase = "stopped" self.termination_reason = reason self.token += 1 - if self.future is not None and hasattr(self.policy, "cancel"): + if (self.future is not None or self.transport_busy) and hasattr(self.policy, "cancel"): self.policy.cancel() self.sim.hold() self.event("info", message) @@ -312,6 +314,17 @@ def refresh_local_connection(self): def connection_request(self, name, payload): # Called on an HTTP worker: listing models must never block physics. + if payload.get("credential_source") == "chatgpt_plan": + from maniloop.providers.chatgpt_plan import list_plan_models + account = self.chatgpt_account(payload.get("chatgpt_account_id")) + models = (list_plan_models(self.chatgpt_auth_store(), account["id"]) + if name == "models" else []) + public = {"source": "chatgpt_plan", "provider": "ChatGPT 套餐直连", + "base_url": "https://api.openai.com/v1", "path": "", + "model": self.default_model, "models": models, + "key_configured": account["logged_in"] and account["plan_enabled"], + "account_label": account["label"], "error": ""} + return {"ok": True, "config": public, "models": models} if payload.get("credential_source") == "codex": from maniloop.providers.codex import login_status status = login_status() @@ -332,12 +345,64 @@ def connection_request(self, name, payload): "config": config.public(), } + def chatgpt_auth_store(self): + # Only this app's independent OAuth store; never load Codex credentials. + with self.lock: + if self._chatgpt_auth is None: + from maniloop.providers.chatgpt_auth import ChatGPTAuthStore + self._chatgpt_auth = ChatGPTAuthStore() + return self._chatgpt_auth + + def chatgpt_account(self, account_id=None): + status = self.chatgpt_auth_store().status() + if status.get("pending"): + raise ValueError("请先完成或取消 ChatGPT 授权") + selected = account_id or status.get("active_account_id") + if not isinstance(selected, str): + raise ValueError("请先为 ManiLoop 登录 ChatGPT 并启用套餐使用") + for account in status["accounts"]: + if account["id"] == selected: + if not account["logged_in"] or not account["plan_enabled"]: + raise ValueError("此账号尚未授权 ChatGPT 套餐使用,请重新授权") + return account + raise ValueError("请选择已保存的 ChatGPT 账号") + + def chatgpt_auth_command(self, name, payload): + if self.running or self.future is not None or self.transport_busy or self.sim.busy or self.chunk is not None: + raise ValueError("请先停止任务并等待在途请求和动作结束,再更改 ChatGPT 账号") + if set(payload) - {"account_id", "retry_registration"}: + raise ValueError("ChatGPT 账号操作包含不支持的字段") + retry = payload.get("retry_registration", False) + if type(retry) is not bool or (retry and (name != "chatgpt/login" or payload.get("account_id") is not None)): + raise ValueError("注册重试不能与账号选择或其他操作混用") + account_id = payload.get("account_id") + if account_id is not None and (not isinstance(account_id, str) or not account_id or len(account_id) > 256): + raise ValueError("ChatGPT 账号标识无效") + auth = self.chatgpt_auth_store() + # A completed decision cannot retain a previous account's client. + if self.credential_source == "chatgpt_plan": + self.close_policy() + self.diagnostic_result = None + if name == "chatgpt/login": + return {"ok": True, **(auth.start_login(retry_registration=True) if retry else auth.start_login(account_id))} + if name == "chatgpt/select": + auth.select_account(account_id) + elif name == "chatgpt/logout": + auth.logout(account_id) + elif name == "chatgpt/cancel": + auth.cancel_login() + else: + raise ValueError("未知 ChatGPT 账号操作") + return {"ok": True, **auth.status()} + def command(self, name, payload): + if name.startswith("chatgpt/"): + return self.chatgpt_auth_command(name, payload) if name == "stop": self.stop() elif name == "typesafe-key": # Explicit local-memory settings only: no provider call or physics step. - if self.running or self.future is not None or self.sim.busy: + if self.running or self.future is not None or self.transport_busy or self.sim.busy: raise ValueError("请先停止任务并等待在途请求和动作结束,再更新 Jev 密钥") if set(payload) == {"typesafe_api_key"}: from maniloop.providers.typesafe import normalize_api_key @@ -374,10 +439,15 @@ def command(self, name, payload): self.phase = "observing" self.event("resume", "执行下一次决策" if self.single_step else "恢复连续执行") elif name == "configure": - if self.running or self.future is not None: + if self.running or self.future is not None or self.transport_busy: raise ValueError("请先停止任务并等待请求结束") max_age = validate_observation_max_age(payload.get("observation_max_age_seconds", self.max_age)) backend = payload.get("backend", self.sim.backend) + mode = payload.get("llm_control", "osc_step") + if mode not in ("osc_step", "tcp_target_servo_v2", "tcp_target_servo_v3"): + raise ValueError("未知 LLM 控制模式") + if mode == "tcp_target_servo_v3" and backend != "libero": + raise ValueError("tcp_target_servo_v3 仅支持 LIBERO") replacement = create_environment( backend=backend, robocasa_layout=payload.get("robocasa_layout", 11), @@ -403,7 +473,7 @@ def command(self, name, payload): self.timing = timing self.max_age = max_age if self.sim.backend in ("libero", "robosuite", "robocasa"): - self.sim.set_llm_control(payload.get("llm_control", "osc_step")) + self.sim.set_llm_control(mode) self.chunk = None self._physics_remainder = 0.0 self.command("reset", {"seed": payload.get("seed", 0)}) @@ -428,13 +498,15 @@ def command(self, name, payload): self.pending_observation, self.request_images = self.sim.observe() self.event("info", "场景已重置") elif name in ("start", "diagnose"): + if payload.get("llm_control") == "tcp_target_servo_v3" and self.sim.backend != "libero": + raise ValueError("tcp_target_servo_v3 仅支持 LIBERO") diagnostic = payload.get("stage") if name == "diagnose" else None if name == "diagnose" and diagnostic not in ("text", "vision", "action"): raise ValueError("诊断阶段必须为 text / vision / action") task = "Connection diagnostic; never execute actions" if diagnostic else str(payload.get("task", "")).strip() if not task or len(task) > 4000: raise ValueError("请输入1至4000字的任务") - if self.running or self.future is not None or self.sim.busy: + if self.running or self.future is not None or self.transport_busy or self.sim.busy: raise ValueError("已有任务或API请求尚未结束,请停止并等待请求返回") context_mode = payload.get("context_mode", "current") single_step = payload.get("single_step", False) @@ -527,7 +599,7 @@ def command(self, name, payload): return self.policy_kind = "llm_cloud" source = payload.get("credential_source", "manual") - config = None if source == "codex" else self.resolve_connection(payload) + config = None if source in {"codex", "chatgpt_plan"} else self.resolve_connection(payload) override = payload.get("model") or None if override is not None and ( not isinstance(override, str) @@ -544,7 +616,19 @@ def command(self, name, payload): if type(options) is not dict: raise ValueError("请求设置必须是对象") self.request_options = dict(options) - if source == "codex": + if source == "chatgpt_plan": + from maniloop.providers.chatgpt_plan import ChatGPTPlanPolicy + account = self.chatgpt_account(payload.get("chatgpt_account_id")) + policy = ChatGPTPlanPolicy(model=model, auth_store=self.chatgpt_auth_store(), + account_id=account["id"], request_options=self.request_options) + self.close_policy() + self.policy = policy + self.key = "" + self.model = model + self.credentials = {"source": "chatgpt_plan", "provider": "ChatGPT 套餐直连", + "path": "", "base_url": "https://api.openai.com/v1", "model": model, + "models": [model], "key_configured": True, "error": ""} + elif source == "codex": from maniloop.providers.codex import CodexPolicy policy = CodexPolicy(model=model, request_options=self.request_options) self.close_policy() @@ -559,7 +643,7 @@ def command(self, name, payload): if not diagnostic and self.sim.backend == "libero": profile = payload.get("observation_profile", self.sim.describe().get("observation_profile", "debug_rgb128")) mode = payload.get("llm_control", self.sim.llm_control) - if mode not in ("osc_step", "tcp_target_servo_v2"): + if mode not in ("osc_step", "tcp_target_servo_v2", "tcp_target_servo_v3"): raise ValueError("未知 LLM 控制模式") if profile != self.sim.describe().get("observation_profile", "debug_rgb128"): info = self.sim.describe() @@ -612,7 +696,7 @@ def command(self, name, payload): self.manual_base_url = config.base_url self.begin_episode(task, max_steps) elif name == "manual": - if self.running or self.future is not None: + if self.running or self.future is not None or self.transport_busy: raise ValueError("请先停止GPT任务并等待在途请求结束,再手动点动") if self.sim.busy or self.chunk is not None: raise ValueError("请等待上一条手动动作结束,或先停止动作") @@ -754,8 +838,18 @@ def advance(self, simulation_seconds=0.02): self.stop("环境已结束", reason="environment_terminated") break + @property + def transport_busy(self): + # Cancellation/deadline may finish the Future before socket I/O exits. + # Retain closing clients so replacing a policy cannot bypass this gate. + self._closing_policies = [policy for policy in getattr(self, "_closing_policies", ()) + if getattr(policy, "transport_busy", False) is True] + return bool(self._closing_policies) or getattr(getattr(self, "policy", None), "transport_busy", False) is True + def close_policy(self): policy, self.policy = self.policy, None + if policy is not None and getattr(policy, "transport_busy", False) is True: + self._closing_policies.append(policy) if policy is not None and hasattr(policy, "close"): policy.close() elif policy is not None and hasattr(policy, "client"): @@ -767,6 +861,8 @@ def close(self): self.token += 1 self.close_policy() self.pool.shutdown(wait=True, cancel_futures=True) + if self._chatgpt_auth is not None: + self._chatgpt_auth.close() self.sim.close() def _visual_changed(self, old_images, new_images): @@ -790,8 +886,9 @@ def tick(self): except queue.Empty: break try: - self.command(name, payload) - done.put({"ok": True, "task_instruction": self.sim.instruction}) + result = self.command(name, payload) + done.put(result if isinstance(result, dict) else + {"ok": True, "task_instruction": self.sim.instruction}) except Exception as exc: message = str(exc) self.event("error", message) @@ -824,6 +921,10 @@ def tick(self): try: action = future.result() finally: + manifest = getattr(self.policy, "last_request_manifest", None) + if isinstance(manifest, dict) and manifest: + self.event("request_manifest", "模型请求协议与实际发送参数", + request_index=self.api_calls, manifest=deepcopy(manifest)) if self.policy_kind == "jev": self.policy_decision = deepcopy(self.policy.last_decision) if self.policy_decision: @@ -925,7 +1026,7 @@ def redact(value): result = (self.policy.execute(self.sim, executable) if self.policy_kind == "jev" else self.sim.execute(executable)) self.awaiting_execution = (result["status"] == "accepted" and - self.sim.describe().get("llm_control") == "tcp_target_servo_v2") + self.sim.describe().get("llm_control") in ("tcp_target_servo_v2", "tcp_target_servo_v3")) self.history.append({"action": action, "feedback": result}) self.history = self.history[-8:] self.geometry = [] @@ -1086,7 +1187,7 @@ def publish(self, render=True): "api_calls": self.api_calls, "max_steps": self.max_steps, "api_latency": self.api_latency, - "pending_request": self.future is not None, + "pending_request": self.future is not None or self.transport_busy, "diagnostic_result": self.diagnostic_result, "diagnostic_stage": self.diagnostic_stage, "observation_max_age_seconds": self.max_age, diff --git a/src/maniloop/ui/chatgpt-plan.js b/src/maniloop/ui/chatgpt-plan.js new file mode 100644 index 0000000..5f74775 --- /dev/null +++ b/src/maniloop/ui/chatgpt-plan.js @@ -0,0 +1,121 @@ +// Account controls share the local backend; credentials never enter browser storage. +export function authorizationURL(value) { + const url = new URL(value); + if (url.origin !== 'https://auth.openai.com' || url.pathname !== '/api/accounts/authorize' + || url.username || url.password || url.hash) throw new Error('授权地址无效;未打开登录窗口。'); + return url.href; +} + +export function connectionModels(source, configuredModels, fetchedModels, selected) { + const candidates = source === 'chatgpt_plan' ? fetchedModels : [...configuredModels, ...fetchedModels, selected]; + return [...new Set(candidates.filter(model => typeof model === 'string' && model && model !== '__custom__'))]; +} + +export function defaultReasoningLabel(source) { + return source === 'chatgpt_plan' ? 'low(套餐直连默认)' : '模型默认'; +} + +export function createChatGPTPlanControls({request, onChanged, showError, showToast, + document = globalThis.document, window = globalThis.window}) { + const get = id => document.getElementById(id); + const account = get('chatgpt-account'); + const login = get('chatgpt-login'), again = get('chatgpt-reauthorize'); + const retry = get('chatgpt-retry-registration'); + const logout = get('chatgpt-logout'), cancel = get('chatgpt-cancel'); + const status = get('chatgpt-status'); + let data = {accounts: [], active_account_id: null, pending: false, error: ''}; + let locked = false, busy = false, polling = false, timer = null, popup = null; + let fingerprint = '', modelIdentity = ''; + const active = () => data.accounts.find(item => item.id === data.active_account_id); + const ready = () => !!(active()?.logged_in && active()?.plan_enabled); + function render() { + const next = JSON.stringify([data.accounts, data.active_account_id]); + // Status polling must not continually rebuild an open account select. + if (next !== fingerprint) { + fingerprint = next; + account.replaceChildren(); + const empty = document.createElement('option'); + empty.value = ''; empty.textContent = '选择已保存的 ChatGPT 账号'; account.appendChild(empty); + for (const item of data.accounts) { + const option = document.createElement('option'); option.value = item.id; + option.textContent = item.label + (item.logged_in ? item.plan_enabled ? '' : ' · 未启用套餐授权' : ' · 已退出'); + account.appendChild(option); + } + account.value = data.active_account_id || ''; + } + status.textContent = data.pending ? '请在授权窗口完成登录;完成后此处会自动更新。若已关闭窗口,可点击「取消授权」后重新开始。' + : data.error || (ready() ? '已授权 · 使用 ChatGPT 套餐直连;模型可用性需通过诊断验证。' + : active()?.logged_in ? '已登录,但未授予套餐使用权限。请重新授权后再诊断。' + : '尚未连接。授权仅用于此 ManiLoop 安装,与 Codex CLI 登录独立。'); + account.disabled = locked || busy || data.pending; + login.disabled = locked || busy || data.pending; + again.disabled = locked || busy || data.pending || !active(); + retry.hidden = !data.registration_retry_available; + retry.disabled = locked || busy || data.pending || !data.registration_retry_available; + logout.disabled = locked || busy || data.pending || !active()?.logged_in; + cancel.hidden = !data.pending; + cancel.disabled = locked || busy || !data.pending; + const identity = JSON.stringify([data.active_account_id, ready()]); + if (identity !== modelIdentity) { modelIdentity = identity; onChanged(); } + } + function schedule() { + if (timer !== null) window.clearTimeout(timer); + timer = data.pending ? window.setTimeout(() => refresh(), 1000) : null; + } + async function refresh({quiet = false} = {}) { + if (polling) return; + polling = true; + try { + const result = await request('/api/chatgpt/status'); + data = {...result, accounts: Array.isArray(result.accounts) ? result.accounts : []}; + // Cross-origin opener isolation and embedded browsers can report a live + // authorization window as closed. Only the backend callback/expiry or an + // explicit cancel may end the attempt; never infer cancellation here. + render(); + } catch (error) { if (!quiet) showError(error); } + finally { polling = false; schedule(); } + } + async function authenticate(accountId, retryRegistration = false) { + if (locked || busy || data.pending) return; + if (retryRegistration && !data.registration_retry_available) return; + // Open synchronously in the click handler to avoid popup blockers. + popup = window.open('about:blank', '_blank'); + if (!popup) { showError(new Error('浏览器阻止了授权窗口,请允许此本地页面打开弹窗后重试。')); return; } + popup.opener = null; + busy = true; render(); + try { + const body = retryRegistration ? {retry_registration: true} : accountId ? {account_id: accountId} : {}; + const result = await request('/api/chatgpt/login', body); + popup.location.replace(authorizationURL(result.authorization_url)); + data = {...data, pending: true, error: ''}; + } catch (error) { + popup.close(); popup = null; + // A rejected or malformed URL must not leave a live authorization attempt. + try { await request('/api/chatgpt/cancel', {}); } catch (_) { /* backend expires pending attempts */ } + showError(error); + } finally { busy = false; render(); schedule(); } + } + async function change(path, body, message) { + if (locked || busy || (data.pending && path !== 'cancel')) return; + busy = true; render(); + try { + const result = await request('/api/chatgpt/' + path, body); + if (path === 'cancel' && popup) { popup.close(); popup = null; } + if (result.warning) showToast(result.warning); else if (message) showToast(message); + await refresh(); + } catch (error) { showError(error); await refresh({quiet: true}); } + finally { busy = false; render(); } + } + login.addEventListener('click', () => authenticate(null)); + again.addEventListener('click', () => authenticate(data.active_account_id)); + retry.addEventListener('click', () => authenticate(null, true)); + logout.addEventListener('click', () => change('logout', {account_id: data.active_account_id}, '已退出此 ManiLoop 账号连接。')); + cancel.addEventListener('click', () => change('cancel', {}, '已取消本次授权。')); + account.addEventListener('change', () => { + if (!account.value) { account.value = data.active_account_id || ''; return; } + change('select', {account_id: account.value}, '已切换 ChatGPT 账号,请重新读取模型列表。'); + }); + window.addEventListener('pagehide', () => { if (timer !== null) window.clearTimeout(timer); }); + return {refresh, ready, accountId: () => data.active_account_id, + update(value) { locked = value; render(); }, pending: () => busy || data.pending}; +} diff --git a/src/maniloop/ui/index.html b/src/maniloop/ui/index.html index 431adbf..1252c44 100644 --- a/src/maniloop/ui/index.html +++ b/src/maniloop/ui/index.html @@ -105,9 +105,23 @@
从简单、明确的单物体任务开始⌘ / Ctrl + Enter
- - - + + + + -

Rx/Ry/Rz 是一次旋转向量的分量,不是绝对欧拉角。按当前坐标轴解释;受后端单步向量范数与可达性限制,不会静默裁剪。

+

Rx/Ry/Rz 是一次旋转向量的分量,不是绝对欧拉角。按当前坐标轴解释;受后端声明的向量范数上限与可达性限制,不会静默裁剪。

@@ -217,7 +232,9 @@ -