refactor(engine)!: simplify execution architecture

[laipz8200]

Important

1. Make sure you have read our contribution guidelines
2. Search existing issues and pull requests to confirm this change is not a duplicate
3. Open or identify the issue this pull request resolves or advances
4. Use a Conventional Commits title for this pull request, and mark breaking changes with !
5. Remember that the pull request title will become the squash merge commit message
6. If CLA Assistant prompts you, sign CLA.md in the pull request conversation

Related Issue

Closes #248

Summary

This refactor reduces the engine's public vocabulary and collapses overlapping execution responsibilities. It intentionally removes compatibility aliases: downstream integrations must upgrade imports, constructor calls, extension hooks, and event field access together.

Architecture changes

• Replace GraphEngine and the graph_engine package with Engine and graphon.engine.
• Give each execution frame one graph and one RuntimeState; child frames receive only their container-scoped graph and can read variables inherited from their parent state without exposing child variables upward.
• Replace graph traversal/state-manager layers with Scheduler, and keep frame creation/restoration in FrameRegistry.
• Use a fixed worker count and direct worker lifecycle management; dynamic worker scaling and GraphEngineConfig are removed.
• Collapse event management into event, command channels/processing into command, and use singular container_handler, filter, layer, and worker modules.
• Keep one command channel boundary, one event stream, and direct failure handling instead of manager/wrapper layers.

Downstream migration

Imports and public names

Before	After
graphon.graph_engine	graphon.engine
graphon.graph_events	graphon.engine_events
graphon.filters / old engine filter paths	graphon.engine.filter
old layers, command_channels, command_processing, container_handlers, and worker_management paths	singular modules under graphon.engine
GraphEngine	Engine
GraphEngineEvent	EngineEvent
GraphNodeEventBase	NodeEvent
NodeEventBase	NodeEventPayload
GraphRuntimeState	RuntimeState
GraphInitParams	InitParams
GraphEngineLayer	Layer
GraphEventFilter	EngineEventFilter
GraphEventFilterContext	EngineEventFilterContext
filter_graph_events(...)	filter_engine_events(...)
TaskEvent	NodeEventTask

Concrete GraphRun*Event, GraphEdge*Event, and NodeRun*Event names remain unchanged. There are no aliases for the old imports or type names.

Engine construction and identity

• Remove GraphEngineConfig; pass the fixed worker count as workers= when loading or constructing an engine.
• Stop passing workflow_id to Engine. Set it on the root RuntimeState; restored and child states share its GraphExecution identity.
• The command channel is optional and defaults to InMemoryChannel.
• Replace engine.layer(layer) with engine.add_layer(layer). The method mutates the engine and returns None.

Events and containers

• Replace in_loop_id and in_iteration_id with one container_id field on engine events.
• container_id identifies the event's direct owning container. Top-level events use container_id == ""; nested events do not expose ancestor container IDs.
• Node implementations yield NodeEventPayload; Node.run() adds execution context and emits NodeEvent, which is part of the EngineEvent stream.
• Consumers that persist, filter, transform, or render events must update their payload schemas and field access. Concrete event payloads otherwise retain their existing fields.

Layers and filters

• Layer lifecycle methods have default no-op implementations; custom layers only override the hooks they use.
• Remove DebugLoggingLayer and GraphEngineLayerNotInitializedError; invalid layer use now raises the standard runtime error.
• Remove ResumableEngineEventFilter and filter_id; filters operate directly on the engine event stream.
• ResponseStreamFilter and ExecutionLimitsLayer use their canonical singular-module imports. Nonessential ExecutionLimitsLayer helpers and LimitType are no longer public API.

Commands and remote control

• Remove GraphEngineManager. Downstream applications now own task IDs, Redis key naming, and command routing policy.
• Replace GraphEngineCommand / CommandType branching with the concrete discriminated command types such as AbortCommand, PauseCommand, and UpdateVariablesCommand.
• Remove the VariableUpdate wrapper; UpdateVariablesCommand.updates receives variables directly.
• Redis command storage no longer uses a pending marker. Custom Redis clients only need list read/delete plus push/expiry operations. The reader accepts the previous { "value": variable } update shape for the bounded Redis TTL migration window.

Extension APIs

ContainerHandler hooks change as follows:

Before	After
start_await(...)	handle_request(...)
complete_frame(...)	complete_frame_if_ready(...)
should_collect(...)	should_emit(...)

FrameRegistry changes as follows:

Before	After
materialize_frame(...)	create(...)
materialize_child_frame(...)	create_child(...)
materialize_child_frame_from_state(...)	restore_child(...)
registry.get(frame_id)	registry[frame_id]

Execution frame fields change as follows:

Before	After
frame.graph_runtime_state	frame.state
frame.state_manager / frame.edge_processor	frame.scheduler
frame.error_handler	frame.failure_handler

Custom node factories must continue binding nodes through with_runtime_state(...). Custom ready queues must consume ReadyTask values and implement serialization through the new ready-queue module; ReadyQueueState is no longer public.

Removed internal boundaries

• Remove GraphStateManager, EdgeProcessor, graph_traversal, event_management, orchestration, error_handler, and dynamic worker-management modules.
• Move GraphExecution and NodeExecution to graphon.runtime.execution.
• Move the dispatcher, event stream/processor, node failure handler, worker pool, and scheduler to their direct owner modules.
• Keep WorkflowExecution temporarily for downstream compatibility, with a TODO marking its future removal. The existing event-stream RWLock is unchanged and will be handled separately.

Persistence compatibility

• Runtime snapshots continue to serialize field data and snapshot versions, not Python class names; existing pause/resume JSON remains readable.
• InitParams.model_dump() data is unchanged, although its generated schema title changes with the class name.
• Python imports, isinstance checks, schema-title assertions, and pickles that refer to removed class/module names are not compatible.
• Event persistence must migrate the former loop/iteration owner fields to container_id.

Dify adaptation scope

Dify should upgrade Graphon and migrate in one change. The main affected areas are workflow engine construction, runtime-state creation and restoration, layer registration, command/stop routing, event filters, persistence and response conversion, custom container handlers, and type annotations. In particular, all in_loop_id / in_iteration_id access in runners, persistence layers, and response converters must use direct-owner container_id semantics rather than mechanically preserving both ancestor fields.

Checklist

• This pull request links the issue it resolves or advances
• This pull request title follows Conventional Commits, and any breaking change is marked with !
• If CLA Assistant prompted me, I signed CLA.md in the pull request conversation