Skip to content
89 changes: 89 additions & 0 deletions docs/dev/scene/reflection-probe-bake.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# Reflection Probe Bake

CLI 通过 MCP 工具 `scene-bake-reflection-probe` 烘焙立方体反射探针。它会捕获六面纹理、调用 cmft 生成 RGBM latlong PNG、导入 TextureCube、绑定组件,并按需保存场景。

## 使用条件

- CLI HTTP/MCP 服务已启动。
- 浏览器已打开 `/scene-editor/`,目标场景显示为 `Loaded`。
- 烘焙期间场景编辑器需要保持可见且可渲染;浏览器后台标签页或最小化窗口可能被节流并导致捕获超时。
- `nodePath` 指向包含 `cc.ReflectionProbe` 的节点。

Node 场景进程使用 EmptyDevice,不能进行有效的 GPU 捕获。因此烘焙会将捕获请求转发给浏览器中的 WebGL 场景渲染器。没有可用渲染器时会明确失败,不会回退生成黑图;六面像素全部为空时也会停止并保留已有资源。

## MCP 调用

工具名:`scene-bake-reflection-probe`

MCP Inspector 切换到 JSON 输入时,调用参数如下:

```json
{
"options": {
"nodePath": "Reflection Probe",
"saveScene": true,
"timeoutMs": 120000
}
}
```

参数:

- `nodePath`:探针节点在当前场景中的路径,必填。
- `fastBake` 直接读取场景中 ReflectionProbe 组件的当前配置。
- `saveScene`:绑定后保存场景,默认 `true`。
- `timeoutMs`:完整流程超时,默认 120 秒,最大 600 秒。

成功结果包含探针节点、组件 UUID、probe ID,以及生成的 TextureCube UUID 和 URL。

调用前应先通过 `scene-open` 打开场景,并在 `/scene-editor/` 中加载同一个场景。`nodePath` 是相对于场景根节点的节点路径,不是资源 URL 或 UUID。

## Pink 场景 Webview

Pink 场景 Webview 中的场景服务运行在本地 WebGL 环境。完整烘焙仍应通过 MCP 工具调用:Sharp、cmft、文件写入和 Asset DB 导入依赖 Node 环境,不能只在 Webview 中完成。

MCP 工具在 Node 主进程执行,并经 Node IPC 进入 scene-process。由于 Node scene-process 使用 EmptyDevice,MCP 路径会额外请求已加载同一场景的 Pink Webview,通过其本地 `window.cli.Scene.ReflectionProbe.capturePixels()` 完成六面捕获;该方法是内部渲染桥,不是公开的完整烘焙入口。Asset DB、配置和文件系统等 Node 能力继续通过 RPC 调用。

## 处理链路

```text
MCP scene-bake-reflection-probe
-> scene process: 校验场景与探针
-> main process: 请求已连接的 WebGL renderer
-> browser /scene-editor/: 捕获六面 RGBA
-> scene process: 写入临时 PNG
-> cmft: 生成 reflectionProbe_<id>.png
-> asset-db: 导入 /textureCube 子资源
-> ReflectionProbe.cubemap: 绑定、刷新预览球、保存场景
```

主要实现:

- API:`src/api/scene/reflection-probe.ts`
- WebGL 请求桥:`src/core/scene/main-process/reflection-probe-renderer.ts`
- 浏览器监听:`src/core/scene/scene-process/engine-bootstrap.ts`
- 捕获、转换、导入和绑定:`src/core/scene/scene-process/service/reflection-probe.ts`

## 输出与兼容行为

- 输出位置:`assets/<scene-name>/reflectionProbe_<probeId>.png`
- TextureCube 子资源:`db://assets/<scene-name>/reflectionProbe_<probeId>.png/textureCube`
- 捕获分辨率、clear flag、背景色、visibility、probe size 和 `fastBake` 均读取 `ReflectionProbe` 组件当前配置;MCP 参数不会覆盖这些值。
- cmft 参数保持 Creator 的 RGBM latlong 行为。
- `fastBake=true` 写入 `mipBakeMode=1`;否则写入 `mipBakeMode=2`。
- 六面 RGBA 会通过同一条 Socket.IO 消息从 WebGL 场景渲染器返回;1024 分辨率约为 24 MiB 原始数据、32 MiB Base64 数据,因此服务端保留 128 MiB 的单消息上限。
- 重复烘焙复用资源身份,并清理旧卷积缓存后重新导入。
- 绑定操作进入 Undo,成功后刷新探针管理器与预览球。

## 验证

```powershell
npm.cmd run compile
npm.cmd test -- --runInBand tests/reflection-probe-bake-api.test.ts tests/reflection-probe-renderer.test.ts
```

端到端验证还应确认:

1. `/scene-editor/` 能看到天空盒和测试模型。
2. MCP 调用返回 `code: 200` 和 TextureCube URL。
3. Creator 重新打开场景后,探针预览球仍显示烘焙结果。
Original file line number Diff line number Diff line change
Expand Up @@ -6707,6 +6707,22 @@ export declare interface IReferenceImageState {
export declare interface IReferenceImageVisibilityOptions {
desiredVisible: boolean;
}
export declare interface IReflectionProbeBakeOptions {
nodePath: string;
saveScene?: boolean;
timeoutMs?: number;
}
export declare interface IReflectionProbeBakeResult {
nodePath: string;
componentUuid: string;
probeId: number;
cubemapUuid: string;
cubemapUrl: string;
fastBake: boolean;
}
export declare interface IReflectionProbeService extends IServiceEvents {
bake(options: IReflectionProbeBakeOptions): Promise<IReflectionProbeBakeResult>;
}
export declare interface IReloadOptions {
urlOrUUID?: string;
preserveUndoHistory?: boolean;
Expand Down Expand Up @@ -6851,6 +6867,7 @@ export declare interface IServiceManager {
SceneView: ISceneViewService;
Preview: IPreviewService;
UI: IUIService;
ReflectionProbe: IReflectionProbeService;
ReferenceImage: IReferenceImageService;
}
export declare interface ISetParentParams {
Expand Down
20 changes: 20 additions & 0 deletions src/api/scene/reflection-probe-schema.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
import { z } from 'zod';

export const SchemaReflectionProbeBakeOptions = z.object({
nodePath: z.string().trim().min(1).describe('Path of the node containing cc.ReflectionProbe'),
saveScene: z.boolean().optional().default(true).describe('Save the current scene after binding the cubemap'),
timeoutMs: z.number().int().positive().max(600_000).optional().default(120_000)
.describe('Timeout for capture, cmft, asset import, binding, and scene save'),
}).describe('Reflection probe bake options');

export const SchemaReflectionProbeBakeResult = z.object({
nodePath: z.string(),
componentUuid: z.string(),
probeId: z.number().int(),
cubemapUuid: z.string(),
cubemapUrl: z.string(),
fastBake: z.boolean(),
}).describe('Reflection probe bake result');

export type TReflectionProbeBakeOptions = z.infer<typeof SchemaReflectionProbeBakeOptions>;
export type TReflectionProbeBakeResult = z.infer<typeof SchemaReflectionProbeBakeResult>;
30 changes: 30 additions & 0 deletions src/api/scene/reflection-probe.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
import { description, param, result, title, tool } from '../decorator/decorator';
import { COMMON_STATUS, CommonResultType } from '../base/schema-base';
import { Scene } from '../../core/scene';
import {
SchemaReflectionProbeBakeOptions,
SchemaReflectionProbeBakeResult,
TReflectionProbeBakeOptions,
TReflectionProbeBakeResult,
} from './reflection-probe-schema';

export class ReflectionProbeApi {
@tool('scene-bake-reflection-probe')
@title('Bake reflection probe')
@description('Capture and bake a cube reflection probe, import its TextureCube, bind it to the component, and optionally save the scene.')
@result(SchemaReflectionProbeBakeResult)
async bake(
@param(SchemaReflectionProbeBakeOptions) options: TReflectionProbeBakeOptions,
): Promise<CommonResultType<TReflectionProbeBakeResult>> {
try {
const data = await Scene.ReflectionProbe.bake(options);
return { code: COMMON_STATUS.SUCCESS, data };
} catch (error) {
console.error(error);
return {
code: COMMON_STATUS.FAIL,
reason: error instanceof Error ? error.message : String(error),
};
}
}
}
3 changes: 3 additions & 0 deletions src/api/scene/scene.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,19 +22,22 @@ import { Scene, TSceneTemplateType } from '../../core/scene';
import { ComponentApi } from './component';
import { NodeApi } from './node';
import { PrefabApi } from './prefab';
import { ReflectionProbeApi } from './reflection-probe';
import { ReferenceImageApi } from './reference-image';
import { options } from '../../core/builder/platforms/android/i18n/en';

export class SceneApi {
public component: ComponentApi;
public node: NodeApi;
public prefab: PrefabApi;
public reflectionProbe: ReflectionProbeApi;
public referenceImage: ReferenceImageApi;

constructor() {
this.component = new ComponentApi();
this.node = new NodeApi();
this.prefab = new PrefabApi();
this.reflectionProbe = new ReflectionProbeApi();
this.referenceImage = new ReferenceImageApi();
}

Expand Down
1 change: 1 addition & 0 deletions src/core/scene/common/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,6 @@ export * from './gizmo';
export * from './scene-view';
export * from './preview';
export * from './ui';
export * from './reflection-probe';
export * from './message';
export * from './reference-image';
27 changes: 27 additions & 0 deletions src/core/scene/common/reflection-probe.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
import type { IServiceEvents } from '../scene-process/service/core';

export interface IReflectionProbeBakeOptions {
nodePath: string;
saveScene?: boolean;
timeoutMs?: number;
}

export interface IReflectionProbeBakeResult {
nodePath: string;
componentUuid: string;
probeId: number;
cubemapUuid: string;
cubemapUrl: string;
fastBake: boolean;
}

export interface IReflectionProbeEvents {
'reflection-probe:bake-start': [nodePath: string];
'reflection-probe:bake-end': [nodePath: string, error?: string];
}

export interface IReflectionProbeService extends IServiceEvents {
bake(options: IReflectionProbeBakeOptions): Promise<IReflectionProbeBakeResult>;
}

export type IPublicReflectionProbeService = Omit<IReflectionProbeService, keyof IServiceEvents>;
4 changes: 4 additions & 0 deletions src/core/scene/main-process/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ import { ComponentProxy } from './proxy/component-proxy';
import { AssetProxy } from './proxy/asset-proxy';
import { EngineProxy } from './proxy/engine-proxy';
import { PrefabProxy } from './proxy/prefab-proxy';
import { ReflectionProbeProxy } from './proxy/reflection-probe-proxy';
import { reflectionProbeRenderer } from './reflection-probe-renderer';
import { ReferenceImageProxy } from './proxy/reference-image-proxy';

import { assetManager } from '../../assets';
Expand All @@ -20,6 +22,7 @@ export interface IMainModule {
'programming': typeof scriptManager;
'sceneConfigInstance': typeof sceneConfigInstance;
'i18n': typeof i18n;
'reflectionProbeRenderer': typeof reflectionProbeRenderer;
'referenceImageFiles': typeof referenceImageFiles;
'referenceImageStore': typeof referenceImageStore;
}
Expand All @@ -35,6 +38,7 @@ export const Scene = {
Node: NodeProxy,
// 组件相关的接口
Component: ComponentProxy,
ReflectionProbe: ReflectionProbeProxy,
// 场景进程
worker: sceneWorker,
};
12 changes: 12 additions & 0 deletions src/core/scene/main-process/proxy/reflection-probe-proxy.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
import type {
IPublicReflectionProbeService,
IReflectionProbeBakeOptions,
IReflectionProbeBakeResult,
} from '../../common';
import { Rpc } from '../rpc';

export const ReflectionProbeProxy: IPublicReflectionProbeService = {
bake(options: IReflectionProbeBakeOptions): Promise<IReflectionProbeBakeResult> {
return Rpc.getInstance().request('ReflectionProbe', 'bake', [options]);
},
};
63 changes: 63 additions & 0 deletions src/core/scene/main-process/reflection-probe-renderer.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
import type { RemoteSocket } from 'socket.io';
import type { DefaultEventsMap } from 'socket.io/dist/typed-events';
import { SCENE_RENDERER_ROOM, socketService } from '../../../server/socket';

export interface IReflectionProbeCaptureResult {
resolution: number;
faces: string[];
}

interface ICaptureResponse {
result?: IReflectionProbeCaptureResult;
error?: string;
}

interface ICaptureRequest {
sceneUrl: string;
nodePath: string;
timeoutMs: number;
}

type Socket = RemoteSocket<DefaultEventsMap, unknown>;

function requestSocket(socket: Socket, request: ICaptureRequest): Promise<IReflectionProbeCaptureResult> {
return new Promise((resolve, reject) => {
socket.timeout(request.timeoutMs).emit(
'scene:capture-reflection-probe',
request,
(error: Error | null, response?: ICaptureResponse) => {
if (error) {
reject(error);
} else if (response?.result) {
resolve(response.result);
} else {
reject(new Error(response?.error || 'WebGL scene renderer returned no reflection-probe data.'));
}
},
);
});
}

export const reflectionProbeRenderer = {
async capture(sceneUrl: string, nodePath: string, timeoutMs: number): Promise<IReflectionProbeCaptureResult> {
const io = socketService.io;
if (!io) {
throw new Error('The WebGL scene renderer is unavailable because the HTTP server is not running.');
}
const sockets = await io.in(SCENE_RENDERER_ROOM).fetchSockets();
if (sockets.length === 0) {
throw new Error('Reflection Probe Bake requires a WebGL scene renderer. Open /scene-editor/ in a browser and retry.');
}

const socket = sockets.find((candidate) => candidate.data.sceneUrl === sceneUrl) ?? sockets[0];
try {
return await requestSocket(socket, { sceneUrl, nodePath, timeoutMs });
} catch (error) {
const detail = error instanceof Error ? error.message : String(error);
throw new Error(
'The selected WebGL scene renderer could not complete the reflection-probe capture. '
+ `Open /scene-editor/ and wait for it to finish loading, then retry. (${detail})`,
);
}
},
};
2 changes: 2 additions & 0 deletions src/core/scene/main-process/rpc.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import { assetManager } from '../../assets';
import scriptManager from '../../scripting';
import { sceneConfigInstance } from '../scene-configs';
import i18n from '../../base/i18n';
import { reflectionProbeRenderer } from './reflection-probe-renderer';
import { referenceImageFiles } from './reference-image-files';
import { referenceImageStore } from './reference-image-store';

Expand Down Expand Up @@ -40,6 +41,7 @@ export class RpcProxy {
// Feature-owned Node modules: external file reads and serialized local configuration writes.
referenceImageFiles,
referenceImageStore,
reflectionProbeRenderer,
});
console.log(`[Node] Scene Process RPC ready ${prc ? '(Attached)' : '(Detached - Web Mode)'}`);
}
Expand Down
Loading
Loading