-
Notifications
You must be signed in to change notification settings - Fork 53
Add EnvGraphSpec to ArenaEnv method #724
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
xyao-nv
wants to merge
19
commits into
main
Choose a base branch
from
xyao/dev/graph_to_aren_env
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
Show all changes
19 commits
Select commit
Hold shift + click to select a range
6a83dbe
add registry
xyao-nv 04f7b4d
Trigger ensure_assets_registered for ObjectRelationLibraryRegistry
xyao-nv 70ff4c2
add task registry
xyao-nv 820ae6d
fix circular deps import error
xyao-nv fc456c2
wip
xyao-nv e8d7d98
refactor
xyao-nv 3a08638
address comments from qian
xyao-nv 63ef493
updates
xyao-nv b4fa3f1
refactor assert
xyao-nv 9a96f62
refactor
xyao-nv c430855
simplify
xyao-nv 189d740
simplify based on assumptions
xyao-nv 94d8f0f
simplify based on assumptions
xyao-nv 17d4589
simplify based on assumptions
xyao-nv 41afa9b
remove task related logics
xyao-nv b57a950
refactor
xyao-nv 863bb20
refactor
xyao-nv a1c6921
refactor
xyao-nv 4b56480
self review
xyao-nv File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
150 changes: 150 additions & 0 deletions
150
isaaclab_arena/environments/arena_env_graph_conversion_utils.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,150 @@ | ||
| # Copyright (c) 2026, The Isaac Lab Arena Project Developers (https://github.com/isaac-sim/IsaacLab-Arena/blob/main/CONTRIBUTORS.md). | ||
| # All rights reserved. | ||
| # | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| from typing import TYPE_CHECKING, Any | ||
|
|
||
| from isaaclab_arena.assets.asset import Asset | ||
| from isaaclab_arena.assets.object_reference import ObjectReference | ||
| from isaaclab_arena.assets.registries import AssetRegistry | ||
| from isaaclab_arena.environments.arena_env_graph_task_conversion_utils import build_task_from_specs | ||
| from isaaclab_arena.environments.arena_env_graph_types import ( | ||
| ArenaEnvGraphNodeSpec, | ||
| ArenaEnvGraphNodeType, | ||
| ArenaEnvGraphObjectReferenceNodeSpec, | ||
| ArenaEnvGraphSpatialConstraintType, | ||
| ArenaEnvGraphStateSpec, | ||
| ) | ||
| from isaaclab_arena.environments.graph_spec_utils import relation_class_for_spatial_constraint_type | ||
| from isaaclab_arena.environments.isaaclab_arena_environment import IsaacLabArenaEnvironment | ||
| from isaaclab_arena.scene.scene import Scene | ||
| from isaaclab_arena.utils.pose import Pose | ||
|
|
||
| if TYPE_CHECKING: | ||
| from isaaclab_arena.environments.arena_env_graph_spec import ArenaEnvGraphSpec | ||
|
|
||
|
|
||
| def build_arena_env_from_graph_spec(graph_spec: ArenaEnvGraphSpec) -> Any: | ||
| """Build an IsaacLabArenaEnvironment from a validated ``ArenaEnvGraphSpec``. | ||
|
|
||
| Precondition: ``graph_spec`` is already validated (node refs exist, ids unique, etc.). | ||
| """ | ||
| # TODO(xinjieyao, 2026-05-26): aggregate every state_spec into a single combined initial state instead of | ||
| # picking one. For now we just take the first state_spec, which is the initial state | ||
| # for the first task — this matches the previous default behavior. | ||
| initial_state_spec = graph_spec.state_specs[0] if graph_spec.state_specs else None | ||
|
|
||
| # 1. Materialize every graph node into a live asset, keyed by node id so spatial | ||
| # constraints and task args can reference each node by its graph-local id. | ||
| assets_by_node_id = _instantiate_assets_from_nodes(graph_spec.nodes, AssetRegistry()) | ||
|
|
||
| # 2. Wire the initial state's spatial relations / fixed poses into those assets. | ||
| if initial_state_spec is not None: | ||
| _attach_spatial_constraints_to_assets(initial_state_spec, assets_by_node_id) | ||
|
|
||
| # 3. Partition nodes into the env's embodiment (exactly one) and its scene assets. | ||
| embodiment, scene_assets = _partition_nodes_into_embodiment_and_scene(graph_spec.nodes, assets_by_node_id) | ||
|
|
||
| # 4. Resolve task specs against the same assets_by_node_id so task args bind to the | ||
| # actual asset instances created in step 1 (not duplicates). | ||
| return IsaacLabArenaEnvironment( | ||
| name=graph_spec.env_name, | ||
| scene=Scene(assets=scene_assets), | ||
| embodiment=embodiment, | ||
| task=build_task_from_specs(graph_spec.tasks, assets_by_node_id), | ||
| ) | ||
|
|
||
|
|
||
| def _partition_nodes_into_embodiment_and_scene( | ||
| node_specs: list[ArenaEnvGraphNodeSpec], assets_by_node_id: dict[str, Any] | ||
| ) -> tuple[Any, list[Asset]]: | ||
| """Split materialized nodes into the single embodiment asset and the list of scene assets. | ||
|
|
||
| Asserts exactly one EMBODIMENT node. OBJECT / OBJECT_REFERENCE nodes become scene assets; | ||
| any other node type raises. Lighting is not handled yet. | ||
| """ | ||
| embodiment = None | ||
| scene_assets: list[Asset] = [] | ||
| # TODO(xinjieyao, 2026-05-26): include lighting later | ||
| for node_spec in node_specs: | ||
| if node_spec.type == ArenaEnvGraphNodeType.EMBODIMENT: | ||
| assert embodiment is None, "Only one embodiment node can be converted to an IsaacLabArenaEnvironment" | ||
| embodiment = assets_by_node_id[node_spec.id] | ||
| elif node_spec.type in (ArenaEnvGraphNodeType.OBJECT, ArenaEnvGraphNodeType.OBJECT_REFERENCE): | ||
| scene_assets.append(assets_by_node_id[node_spec.id]) | ||
| else: | ||
| raise ValueError(f"Unsupported node type: {node_spec.type}") | ||
| return embodiment, scene_assets | ||
|
|
||
|
|
||
| def _instantiate_assets_from_nodes(node_specs: list[ArenaEnvGraphNodeSpec], asset_registry: Any) -> dict[str, Any]: | ||
| """Return ``{node.id: live_asset}`` after a single pass over ``node_specs``. | ||
|
|
||
| Each ``node_spec.params`` is forwarded verbatim to the asset constructor. Assumes parent | ||
| nodes precede their OBJECT_REFERENCE children — guaranteed by ``assert_references_exist``. | ||
| """ | ||
| assets_by_node_id: dict[str, Any] = {} | ||
| for node_spec in node_specs: | ||
| # OBJECT_REFERENCE wraps a USD prim inside an already-instantiated parent asset | ||
| # (e.g. a table inside a kitchen background). Validation guarantees the parent | ||
| # precedes the reference, so it is already in assets_by_node_id here. | ||
| if node_spec.type == ArenaEnvGraphNodeType.OBJECT_REFERENCE: | ||
| assert isinstance(node_spec, ArenaEnvGraphObjectReferenceNodeSpec) | ||
| assets_by_node_id[node_spec.id] = ObjectReference( | ||
| name=node_spec.name, | ||
| prim_path=node_spec.prim_path, | ||
| parent_asset=assets_by_node_id[node_spec.parent], | ||
| object_type=node_spec.object_type, | ||
| **node_spec.params, | ||
| ) | ||
| else: | ||
| # Standard nodes (object / background / embodiment): look up the registered class | ||
| # by name and instantiate with the spec's verbatim kwargs. | ||
| asset_class = asset_registry.get_asset_by_name(node_spec.name) | ||
| assets_by_node_id[node_spec.id] = asset_class(**node_spec.params) | ||
| return assets_by_node_id | ||
|
|
||
|
|
||
| def _attach_spatial_constraints_to_assets( | ||
| state_spec: ArenaEnvGraphStateSpec, assets_by_node_id: dict[str, Any] | ||
| ) -> None: | ||
| """Attach one Relation per spatial constraint to the asset(s) it targets, in place. | ||
|
|
||
| AT_POSE is special-cased to ``set_initial_pose`` since it has no Relation class. | ||
| Raises AssertionError on unsupported constraint types or malformed AT_POSE params. | ||
| """ | ||
| for spatial_constraint in state_spec.spatial_constraints: | ||
| parent_asset = assets_by_node_id[spatial_constraint.parent] | ||
|
|
||
| # AT_POSE has no Relation class — it pins the parent's initial pose directly, | ||
| # bypassing the solver. Pop the two known fields and reject any extras so a typo | ||
| # in the spec ('postion_xyz') raises here instead of silently being ignored. | ||
| # TODO(xinjieyao, 2026-05-26): move at_pose handling into the placer module. | ||
| if spatial_constraint.type == ArenaEnvGraphSpatialConstraintType.AT_POSE: | ||
| at_pose_params = dict(spatial_constraint.params) | ||
| position_xyz = at_pose_params.pop("position_xyz", None) | ||
| rotation_xyzw = at_pose_params.pop("rotation_xyzw", (0.0, 0.0, 0.0, 1.0)) | ||
| assert ( | ||
| position_xyz is not None | ||
| ), f"at_pose constraint '{spatial_constraint.id}' requires params.position_xyz" | ||
| assert ( | ||
| not at_pose_params | ||
| ), f"Unsupported at_pose params for constraint '{spatial_constraint.id}': {sorted(at_pose_params)}" | ||
| parent_asset.set_initial_pose(Pose(position_xyz=position_xyz, rotation_xyzw=rotation_xyzw)) | ||
| else: | ||
| # All other constraint types resolve through ObjectRelationLibraryRegistry. The | ||
| # registry returning None signals an enum value with no registered class — would | ||
| # be a programming error, not a YAML error. | ||
| relation_class = relation_class_for_spatial_constraint_type(spatial_constraint.type) | ||
| assert relation_class is not None, f"Unsupported spatial constraint type '{spatial_constraint.type.value}'" | ||
| # Unary relations (IS_ANCHOR, POSITION_LIMITS, ...) attach to the parent asset. | ||
| # Binary relations (ON, NEXT_TO, ...) attach to the *child* asset, with parent | ||
| # as the relation's first constructor arg — matches how add_relation is wired. | ||
| if relation_class.is_unary(): | ||
| parent_asset.add_relation(relation_class(**spatial_constraint.params)) | ||
| else: | ||
| child_asset = assets_by_node_id[spatial_constraint.child] | ||
| child_asset.add_relation(relation_class(parent_asset, **spatial_constraint.params)) | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice! Super clean!