diff --git a/README.md b/README.md index d3430569a..5c02a4a4e 100644 --- a/README.md +++ b/README.md @@ -27,25 +27,27 @@ NeMo Fabric gives applications, evaluation systems, and rollout platforms a cons NeMo Fabric includes adapters for the following harnesses and execution targets. Select a harness name for its adapter documentation: -| Agent Harness | Tool Policy | MCP | Skills | Telemetry | -| --- | :---: | :---: | :---: | :---: | -| [Claude Code](adapters/python/claude/README.md) | ✅ | ✅ | ✅ | ✅ | -| [Cline](adapters/typescript/cline/README.md) | ✅ | ✅ | ✅ | — | -| [Codex](adapters/python/codex/README.md) | — | ✅ | ✅ | ✅ | -| [Factory Droid](adapters/typescript/droid/README.md) | ✅ | ✅ | ✅ | — | -| [Hermes Agent](adapters/python/hermes/README.md) | ✅ | ✅ | ✅ | ✅ | -| [Kilo Code](adapters/typescript/kilo/README.md) | ✅ | ✅ | ✅ | — | -| [AWS Kiro](adapters/typescript/kiro/README.md) | — | ✅ | — | — | -| [AWS Strands](adapters/typescript/strands/README.md) | — | ✅ | — | — | -| [LangChain Deep Agents](adapters/python/deepagents/README.md) | ✅ | ✅ | ✅ | ✅ | -| [mini-SWE-agent](adapters/python/mini-swe-agent/README.md) | — | — | — | ✅ | -| [NOOA](adapters/python/nooa/README.md) | — | ✅ | ✅ | ✅ | -| [OpenClaw](adapters/python/openclaw/README.md) | ✅ | ✅ | ✅ | — | -| [OpenHands](adapters/python/openhands/README.md) | ✅ | ✅ | ✅ | — | -| [OpenCode](adapters/typescript/opencode/README.md) | — | ✅ | ✅ | — | -| [Pi](adapters/typescript/pi/README.md) | ✅ | — | ✅ | ✅ | -| [Qwen Code](adapters/typescript/qwen/README.md) | ✅ | ✅ | ✅ | — | -| [Remote Agent](adapters/python/remote-agent/README.md) | — | — | — | ✅ | +| Agent Harness | Tool Policy | MCP | Skills | Telemetry | Native Interfaces | +| --- | :---: | :---: | :---: | :---: | :---: | +| [Claude Code](adapters/python/claude/README.md) | ✅ | ✅ | ✅ | ✅ | — | +| [Cline](adapters/typescript/cline/README.md) | ✅ | ✅ | ✅ | — | — | +| [Codex](adapters/python/codex/README.md) | — | ✅ | ✅ | ✅ | — | +| [Factory Droid](adapters/typescript/droid/README.md) | ✅ | ✅ | ✅ | — | — | +| [Hermes Agent](adapters/python/hermes/README.md) | ✅ | ✅ | ✅ | ✅ | ✅ | +| [Kilo Code](adapters/typescript/kilo/README.md) | ✅ | ✅ | ✅ | — | — | +| [AWS Kiro](adapters/typescript/kiro/README.md) | — | ✅ | — | — | — | +| [AWS Strands](adapters/typescript/strands/README.md) | — | ✅ | — | — | — | +| [LangChain Deep Agents](adapters/python/deepagents/README.md) | ✅ | ✅ | ✅ | ✅ | — | +| [mini-SWE-agent](adapters/python/mini-swe-agent/README.md) | — | — | — | ✅ | — | +| [NOOA](adapters/python/nooa/README.md) | — | ✅ | ✅ | ✅ | — | +| [OpenClaw](adapters/python/openclaw/README.md) | ✅ | ✅ | ✅ | — | ✅ | +| [OpenHands](adapters/python/openhands/README.md) | ✅ | ✅ | ✅ | — | — | +| [OpenCode](adapters/typescript/opencode/README.md) | — | ✅ | ✅ | — | — | +| [Pi](adapters/typescript/pi/README.md) | ✅ | — | ✅ | ✅ | — | +| [Qwen Code](adapters/typescript/qwen/README.md) | ✅ | ✅ | ✅ | — | — | +| [Remote Agent](adapters/python/remote-agent/README.md) | — | — | — | ✅ | — | + +Native interfaces: harness-native API, dashboard, or TUI access enabled through Fabric. ✅ Supported · — Not supported. Support may come through normalized configuration, native harness behavior, or a specific target. Refer to each adapter guide or the [complete compatibility matrix](adapters/README.md#configuration-compatibility) for details and limitations. diff --git a/adapters/python/hermes/README.md b/adapters/python/hermes/README.md index 45bcbda43..d95e46524 100644 --- a/adapters/python/hermes/README.md +++ b/adapters/python/hermes/README.md @@ -5,24 +5,40 @@ SPDX-License-Identifier: Apache-2.0 # NVIDIA NeMo Fabric Hermes Agent Adapter -This adapter runs Hermes Agent through its Python SDK. +Start with [hermes.mdx](../../../docs/integrations/harness/hermes/hermes.mdx) for shared setup and an overview of all four invocation paths. -Managed API and dashboard children launch through an adapter-owned bootstrap that applies the same behavioral mapping when Hermes constructs native agent instances. Instructions and behavioral settings configured by NeMo Fabric take precedence over native constructor defaults. Explicit tool selection and nonempty blocked-tool policies override native defaults; unspecified tool policies retain native interface defaults. The bootstrap retains native routing, session identity, execution, and transport; it does not modify the installed Hermes source. A mode-`0600` configuration snapshot under the runtime's `interfaces` directory supplies the mapping and is removed on stop. Gateway processes independently started outside this launcher do not receive this guarantee. +This adapter hosts Hermes Agent SDK, API, browser, and terminal invocation paths within one NeMo Fabric runtime. All four use normalized `FabricConfig`, translated into Hermes's native configuration and agent settings. + +**Experimental terminal interface:** `interfaces.terminal` (including `terminal.port`), the `tui.json` / `tui.token` attachment files, and the `nemo_fabric_adapters.hermes.terminal` helper may change across releases as the [#579 launcher contract](https://github.com/NVIDIA/NeMo-Fabric/issues/579) is adopted. Compatibility is not guaranteed for these surfaces; see the [terminal guide](../../../docs/integrations/harness/hermes/hermes-tui.mdx). + +## Execution Paths + +The adapter supports the following client paths: + +| Path | Execution backend | Guide | +| --- | --- | --- | +| SDK | Embedded Hermes `AIAgent` in the adapter host; enabled by default | [Hermes SDK](../../../docs/integrations/harness/hermes/hermes-sdk.mdx) | +| HTTP API | Managed native API gateway | [Hermes API](../../../docs/integrations/harness/hermes/hermes-api.mdx) | +| Dashboard/browser TUI | Managed TUI service with browser assets and a terminal bridge | [Hermes Dashboard](../../../docs/integrations/harness/hermes/hermes-dashboard.mdx) | +| Terminal TUI | Terminal renderer attached to the same TUI service; terminal-only mode runs without serving the browser dashboard | [Hermes Terminal TUI](../../../docs/integrations/harness/hermes/hermes-tui.mdx) | + +Browser and terminal clients share one backend process and use separate conversations. Neither requires the API gateway. + +Set `harness.settings.sdk.enabled` to `false` for native-only execution, with at least one native interface enabled. Your application can still start, check, and stop that runtime through the NeMo Fabric SDK. SDK invocation returns a failed `RunResult` with `hermes_sdk_disabled`, leaving native interfaces and health checks available. ## Runtime Health -`Fabric.check()` supports `live`, `active`, and `ready` for a started runtime. +`Fabric.check()` reports normalized `live`, `active`, and `ready` results for the SDK agent when enabled, or for the native backends when the SDK is disabled: -| Level | Evidence | -| --- | --- | -| `live` | The owned native session matches the runtime, remains open, and answers diagnostics. | -| `active` | The initialized `run_conversation()` interface remains available and answers diagnostics. | -| `ready` | A fresh `get_activity_summary()` response passes validation after the earlier checks. | +| Level | SDK Enabled (Including Mixed Runtimes) | SDK Disabled: Managed Native Backends | +| --- | --- | --- | +| `live` | Session/database ownership and fresh native diagnostics | Owned API/TUI processes have not exited | +| `active` | Available `run_conversation()` interface and fresh native diagnostics | Process checks plus API `/health` or TUI `/api/status`; TUI also requires an authenticated RPC | +| `ready` | A valid `get_activity_summary()` response after session/interface checks | The same native process and endpoint checks used for `active` | -The adapter qualifies Hermes' read-only diagnostic response as the MVP readiness -check. Health checks preserve conversation state and respond during an invocation. +Hermes's `/health` and `/api/status` are native endpoints. Startup probes every enabled interface. After startup, mixed-runtime health and `idle`/`busy` activity describe the SDK agent; a failed API/TUI child does not fail SDK runtime health. Probe native endpoints and interface logs separately to diagnose those services. Native-only runtimes check native backends and report activity as `unknown` because native turns are not fully aggregated. -Refer to the [SDK health-check overview](../../../docs/sdk/health-checks.mdx) for shared behavior and a comparison of native evidence across adapters. +Refer to the [SDK health-check overview](../../../docs/sdk/health-checks.mdx) and [Hermes API health guidance](../../../docs/integrations/harness/hermes/hermes-api.mdx#health-testing-and-limitations) for the distinction between native health and normalized checks. ## Install @@ -54,6 +70,17 @@ matching NeMo Fabric release versions. Refer to the Hermes Agent v0.22.5 supports ordinary `Runtime.invoke()` calls, Relay telemetry, and `Runtime.invoke_stream()`. +The invocation paths have different additional requirements: + +| Path | Requirements Beyond Shared Installation | +| --- | --- | +| SDK | No interface-specific packages or frontend builds; install dependencies required by selected tools and MCP servers. | +| HTTP API | `aiohttp` pinned by Hermes v0.22.5 (`3.14.5`), already included by `just install-hermes-agent`; see [API installation](../../../docs/integrations/harness/hermes/hermes-api.mdx#installation). | +| Dashboard/browser TUI | Python web-serving dependencies, built browser and terminal assets, and a supported Node runtime; see [dashboard installation](../../../docs/integrations/harness/hermes/hermes-dashboard.mdx#installation). | +| Terminal TUI | Python web-serving dependencies for the backend, plus a built terminal renderer and supported Node runtime for the client; terminal-only mode needs no browser assets. See [terminal installation](../../../docs/integrations/harness/hermes/hermes-tui.mdx#installation). | + +Native interfaces require a POSIX host. Runtime startup does not build or install frontends. The [manual TUI example](../../../examples/interfaces/hermes/README.md#4-terminal-tui) provides the setup commands and interaction flow. + ## What It Maps The adapter receives a normalized payload from NeMo Fabric and materializes a native Hermes Agent configuration for: @@ -72,22 +99,19 @@ The adapter receives a normalized payload from NeMo Fabric and materializes a na Tool selectors are Hermes toolset names because that is the native policy surface Hermes exposes. -During run-plan resolution, NeMo Fabric Core validates the -`models..top_p` (from 0 through 1) and positive -`models..max_tokens` fields against the Hermes descriptor before -runtime startup. Hermes reads these prevalidated values from `AgentConfig`. A -model-level `max_tokens` overrides `harness.settings.max_tokens` for the selected model role. If both are omitted, the adapter passes `None` to preserve Hermes' native response token limit behavior. +System instructions support `instructions.system.mode: replace`. Use `runtime.max_turns` to control the agent loop. The descriptor validates the following `harness.settings` fields: | Setting | Type | Default | Description | | --- | --- | --- | --- | +| `sdk.enabled` | boolean | `true` | Initialize the embedded Hermes `AIAgent`. Disable only with an enabled native interface; invocation then fails with `hermes_sdk_disabled` while runtime management remains available. | | `reasoning_config` | object | `{"effort": "none"}` | Configures Hermes model reasoning. The closed object accepts an optional `enabled` boolean and an optional `effort` value of `none`, `minimal`, `low`, `medium`, `high`, or `xhigh`. | -| `plugins_enabled` | array of nonempty strings | `[]` | Enables Hermes plugins by identifier. Relay telemetry uses the generated `plugins.toml`; do not list the removed `observability/nemo_relay` plugin. | +| `plugins_enabled` | array of nonempty strings | `[]` | List of native Hermes plugin identifiers to enable. Configure NeMo Relay through NeMo Fabric telemetry settings. | | `save_trajectories` | boolean | `false` | Enables Hermes-native JSONL conversation trajectory saving. This is separate from normalized NeMo Fabric telemetry. | -| `max_tokens` | positive integer | Hermes native (`None`) | Limits the number of tokens in each Hermes model response. | -| `terminal_timeout` | positive number | `60` | Limits a Hermes terminal operation in seconds. | -| `interfaces.api`, `interfaces.dashboard` | object | None | Serve Hermes's API server or dashboard from a runtime, each with an exact `port` and optional `enabled`; see [Access and Use](../../../docs/integrations/harness/hermes-interfaces.mdx#3-access-and-use). | +| `max_tokens` | positive integer | Hermes native default | Used when the selected model has no explicit `max_tokens`. A model-level limit takes precedence; if both are omitted, Hermes uses its native default. | +| `terminal_timeout` | positive number | `60` | Limits a Hermes terminal tool operation in seconds; it does not enable a TUI. | +| `interfaces.api`, `interfaces.dashboard`, `interfaces.terminal` | object | None | Serve Hermes's API or shared dashboard/terminal TUI backend, with an exact `port` and optional `enabled`. Dashboard and terminal ports must match when both are enabled; API requires a distinct port; see [Hermes invocation paths](../../../docs/integrations/harness/hermes/hermes.mdx). | | `startup_timeout_seconds` | number, greater than 0 and at most 75 | `60` | Time from the start of the runtime for every enabled interface to pass its health check. | For example: @@ -107,24 +131,27 @@ harness = HarnessConfig( ) ``` -Use `runtime.max_turns` to set the Hermes agent-loop budget. The adapter maps -that normalized field to `AIAgent.max_iterations`; `max_iterations` is not a -Hermes harness setting. +## Configuration Mapping and Ownership + +Managed native processes use the adapter's launcher to apply explicit instructions and behavioral settings before constructing agents. Explicit enabled tools, including an empty list, override native selection; a nonempty blocked list overrides native exclusions. Unspecified tool policies can differ between SDK and native paths. + +The runtime home is `/.fabric/hermes/runtimes/`. The adapter writes native configuration and optional NeMo Relay plugin configuration there. When native interfaces are enabled, an owner-only `interfaces/agent-config.json` snapshot supplies the constructor mapping and is removed on stop. Model credentials are provided through child environments rather than public attachment metadata. -The adapter derives Hermes state from the NeMo Fabric artifact root and creates -a child under `runtimes/`, so invocations in one NeMo Fabric runtime -share state without sharing config or the session database with another -runtime. +The adapter leaves the installed Hermes source unchanged. Independently started processes do not automatically receive its mapping. Refer to [Shared Configuration and Ownership](../../../docs/integrations/harness/hermes/hermes.mdx#shared-configuration-and-ownership) for precedence and managed API gateway scope. ## Execution Model -Each NeMo Fabric runtime starts one local adapter host, constructs one Hermes Agent -`AIAgent`, and opens one `SessionDB`. Ordered `Runtime.invoke(...)` calls reuse -those native objects and pass the prior turn's returned transcript back to -`run_conversation(...)`. Runtime stop calls the agent's idempotent `close()` -method, closes the session database, and releases the Relay plugin context when -enabled. Relay subscriber work is flushed after each invocation so ATIF output -is complete before the result returns. +The runtime owns the enabled execution backends: + +- **SDK agent:** Enabled by default and initialized during startup with its parent `SessionDB` and SDK-owned MCP connections. Ordered SDK invocations reuse the agent and conversation until stop closes them. Setting `sdk.enabled: false` skips these resources. +- **API gateway:** Starts when `interfaces.api` is enabled. Clients use the per-start `interfaces/api.token` with `Authorization: Bearer ` and receive native HTTP responses. +- **Shared TUI service:** Starts in dashboard mode when `interfaces.dashboard` is enabled, or headless `hermes serve` mode for terminal-only use. Browser and terminal clients attach to the same service with separate conversations. Terminal attachment uses private `interfaces/tui.json` and `interfaces/tui.token` files; the helper runs the prebuilt renderer without starting another backend. + +SDK invocations return normalized NeMo Fabric `RunResult` objects. API, browser, and terminal responses remain native. With NeMo Relay enabled, the adapter finalizes and flushes SDK invocation evidence and managed native turn segments while preserving conversation history. Standard ATOF/ATIF formats do not imply normalized native results or complete session-scoped artifact collection. + +Detaching a client leaves the runtime running. Runtime stop shuts down its owned native services, removes private attachment files and the configuration snapshot, and closes any embedded SDK resources. Sharing persisted files does not establish cross-runtime resume. + +See [Limitations and Planned Work](../../../docs/integrations/harness/hermes/hermes.mdx#limitations-and-planned-work) for approvals, active-turn detach behavior, remote attachment, process cleanup, and native activity/inventory boundaries. ## Maintaining The Adapter diff --git a/adapters/python/hermes/hermes.fabric-adapter.json b/adapters/python/hermes/hermes.fabric-adapter.json index 0ceeda9e3..458ce7fdf 100644 --- a/adapters/python/hermes/hermes.fabric-adapter.json +++ b/adapters/python/hermes/hermes.fabric-adapter.json @@ -19,7 +19,14 @@ }, "effort": { "type": "string", - "enum": ["none", "minimal", "low", "medium", "high", "xhigh"], + "enum": [ + "none", + "minimal", + "low", + "medium", + "high", + "xhigh" + ], "description": "Hermes model reasoning effort." } }, @@ -73,7 +80,9 @@ "description": "Exact loopback port for the API server. Start fails if it is in use." } }, - "required": ["port"], + "required": [ + "port" + ], "additionalProperties": false, "description": "Hermes's API server, run by a gateway child. After every enabled interface starts, the adapter writes its bearer token to /.fabric/hermes/runtimes//interfaces/api.token." }, @@ -92,9 +101,31 @@ "description": "Exact loopback port for the dashboard. Start fails if it is in use." } }, - "required": ["port"], + "required": [ + "port" + ], "additionalProperties": false, "description": "Hermes's dashboard, which has no sign-in when bound to loopback. It needs a built frontend at HERMES_WEB_DIST." + }, + "terminal": { + "type": "object", + "properties": { + "enabled": { + "type": "boolean", + "default": true + }, + "port": { + "type": "integer", + "minimum": 1, + "maximum": 65535, + "description": "Experimental TUI backend loopback port. Must match dashboard.port when both interfaces are enabled. This required field may be removed when the #579 launcher contract is adopted." + } + }, + "required": [ + "port" + ], + "additionalProperties": false, + "description": "Experimental terminal attachment to a managed TUI backend. Uses headless serve without dashboard assets, or shares the dashboard process. The configuration (including terminal.port), interfaces/tui.json, interfaces/tui.token, and the nemo_fabric_adapters.hermes.terminal helper may change across releases as the #579 launcher contract is adopted; compatibility is not guaranteed for these surfaces." } }, "additionalProperties": false, @@ -106,17 +137,105 @@ "maximum": 75, "default": 60, "description": "Time from the start of the runtime for every enabled interface to pass its health check, within NeMo Fabric's 90-second start limit." + }, + "sdk": { + "type": "object", + "properties": { + "enabled": { + "type": "boolean", + "default": true, + "description": "Initialize the embedded SDK agent during startup. Disable for native-only execution; SDK invocation then fails explicitly." + } + }, + "additionalProperties": false, + "description": "Embedded SDK execution, enabled by default. Runtime management remains available when disabled." } }, "required": [], - "additionalProperties": false + "additionalProperties": false, + "allOf": [ + { + "if": { + "required": [ + "sdk" + ], + "properties": { + "sdk": { + "required": [ + "enabled" + ], + "properties": { + "enabled": { + "const": false + } + } + } + } + }, + "then": { + "required": [ + "interfaces" + ], + "properties": { + "interfaces": { + "anyOf": [ + { + "required": [ + "api" + ], + "properties": { + "api": { + "properties": { + "enabled": { + "const": true + } + } + } + } + }, + { + "required": [ + "dashboard" + ], + "properties": { + "dashboard": { + "properties": { + "enabled": { + "const": true + } + } + } + } + }, + { + "required": [ + "terminal" + ], + "properties": { + "terminal": { + "properties": { + "enabled": { + "const": true + } + } + } + } + } + ] + } + } + } + } + ] }, "extension_schemas": { "run_error": { "$schema": "https://json-schema.org/draft/2020-12/schema", "type": "object", "properties": { - "provider": {"type": "string"} + "provider": { + "type": "string" + } }, "additionalProperties": false } @@ -136,7 +255,9 @@ "mcp.auth.oauth2", "skills" ], - "system_instruction_modes": ["replace"] + "system_instruction_modes": [ + "replace" + ] }, "telemetry": { "providers": { @@ -150,6 +271,12 @@ } }, "capabilities": { - "metadata": {"health_checks": ["live", "active", "ready"]} + "metadata": { + "health_checks": [ + "live", + "active", + "ready" + ] + } } } diff --git a/adapters/python/hermes/src/nemo_fabric_adapters/hermes/adapter.py b/adapters/python/hermes/src/nemo_fabric_adapters/hermes/adapter.py index 3fa9935f1..8379b2297 100755 --- a/adapters/python/hermes/src/nemo_fabric_adapters/hermes/adapter.py +++ b/adapters/python/hermes/src/nemo_fabric_adapters/hermes/adapter.py @@ -49,10 +49,12 @@ def main() -> None: class HermesRuntime: - """One Hermes agent and session database owned by a Fabric runtime.""" + """SDK and native Hermes execution owned by one Fabric runtime.""" def __init__(self) -> None: self._started = False + self._sdk_enabled = True + self._owns_mcp = False self._stopping = False self._health_lock = asyncio.Lock() self._health_task: asyncio.Task[Any] | None = None @@ -74,6 +76,12 @@ def __init__(self) -> None: self._interfaces = interfaces.InterfaceChildren() async def start(self, payload: dict[str, Any]) -> None: + """Apply shared configuration and start the selected SDK/native backends. + + SDK execution remains enabled by default. A native-only runtime skips + SDK agent, database, and MCP initialization; failed startup cleans up + any resources already created. + """ if self._started: raise lifecycle.LifecycleError( "hermes_runtime_already_started", @@ -108,6 +116,12 @@ async def start(self, payload: dict[str, Any]) -> None: self._agent_config = agent_config self._settings = configuration._settings(agent_config) interface_ports = interfaces.interface_ports(self._settings) + self._sdk_enabled = self._settings.get("sdk", {}).get("enabled", True) + if not self._sdk_enabled and not interface_ports: + raise lifecycle.LifecycleError( + "hermes_no_execution_path", + "Enable the Hermes SDK or at least one native interface", + ) if interface_ports: # Refuse before Hermes's setup, whose MCP discovery can take # minutes and outlast the caller's start deadline. @@ -162,54 +176,56 @@ async def start(self, payload: dict[str, Any]) -> None: if not api_key: raise RuntimeError(f"{api_key_env} is required for Hermes mode") - from hermes_cli.config import load_config - from hermes_cli.plugins import discover_plugins - from hermes_state import SessionDB - from run_agent import AIAgent - - with redirect_stdout(StringIO()): - if relay_enabled: - common_utils.reject_ambient_relay_plugin_config() - discover_plugins(force=True) - loaded_hermes_config = load_config() - # Hermes 0.12+ no longer discovers MCP tools as an import side effect - # (#16856). Fabric is a Hermes host: discover after config.yaml exists - # and before AIAgent resolves mcp-* toolsets. - # discover_mcp_tools uses a blocking 120s wait, wrapping it in - # asyncio.to_thread to avoid blocking the loop. - if self._hermes_config.get("mcp_servers"): - try: - from tools.mcp_tool_discovery import discover_mcp_tools - except ModuleNotFoundError as error: - if error.name != "tools.mcp_tool_discovery": - raise - from tools.mcp_tool import discover_mcp_tools - - await asyncio.to_thread(discover_mcp_tools) - - self._enabled_toolsets = configuration.resolve_hermes_toolsets( - agent_config, loaded_hermes_config - ) - self._session_db = SessionDB() - self._conversation_history = None - self._agent = AIAgent( - **filter_supported_kwargs( - AIAgent, - base_url=model_config.base_url, - api_key=api_key, - provider=model_config.provider, - model=model_config.model, - **configuration.agent_behavior_kwargs( - agent_config, self._enabled_toolsets - ), - quiet_mode=True, - skip_context_files=True, - skip_memory=True, - platform="fabric", - session_id=self._runtime_id, - session_db=self._session_db, + if self._sdk_enabled: + from hermes_cli.config import load_config + from hermes_cli.plugins import discover_plugins + from hermes_state import SessionDB + from run_agent import AIAgent + + with redirect_stdout(StringIO()): + if relay_enabled: + common_utils.reject_ambient_relay_plugin_config() + discover_plugins(force=True) + loaded_hermes_config = load_config() + # Hermes 0.12+ no longer discovers MCP tools as an import side effect + # (#16856). Fabric is a Hermes host: discover after config.yaml exists + # and before AIAgent resolves mcp-* toolsets. + # discover_mcp_tools uses a blocking 120s wait, wrapping it in + # asyncio.to_thread to avoid blocking the loop. + if self._hermes_config.get("mcp_servers"): + self._owns_mcp = True + try: + from tools.mcp_tool_discovery import discover_mcp_tools + except ModuleNotFoundError as error: + if error.name != "tools.mcp_tool_discovery": + raise + from tools.mcp_tool import discover_mcp_tools + + await asyncio.to_thread(discover_mcp_tools) + + self._enabled_toolsets = configuration.resolve_hermes_toolsets( + agent_config, loaded_hermes_config + ) + self._session_db = SessionDB() + self._conversation_history = None + self._agent = AIAgent( + **filter_supported_kwargs( + AIAgent, + base_url=model_config.base_url, + api_key=api_key, + provider=model_config.provider, + model=model_config.model, + **configuration.agent_behavior_kwargs( + agent_config, self._enabled_toolsets + ), + quiet_mode=True, + skip_context_files=True, + skip_memory=True, + platform="fabric", + session_id=self._runtime_id, + session_db=self._session_db, + ) ) - ) if interface_ports: if "api" in interface_ports: from hermes_cli.profile_channels import platform_ids @@ -250,6 +266,12 @@ async def start(self, payload: dict[str, Any]) -> None: ), environment=environment, agent_config=agent_config, + workspace=str(runtime_context.environment.workspace or "."), + terminal_enabled=self._settings.get("interfaces", {}) + .get("terminal", {}) + .get("enabled", True) + if "terminal" in self._settings.get("interfaces", {}) + else False, ) self._agent_config = agent_config self._started = True @@ -262,6 +284,20 @@ async def invoke( request: AgentRunRequest, runtime_context: RuntimeContext, ) -> AgentRunResult: + """Invoke the embedded agent and return its normalized result. + + Native-only runtimes reject SDK invocation without constructing an + agent. Enabled SDK runtimes retain their conversation between calls. + """ + if self._started and not self._sdk_enabled: + return AgentRunResult( + status=AgentRunStatus.FAILED, + output={}, + error=AgentRunError( + code="hermes_sdk_disabled", + message="Embedded Hermes SDK execution is disabled; use an enabled native interface", + ), + ) agent_config = self._agent_config model_config = self._model_config if ( @@ -510,6 +546,35 @@ def _finalize_relay_session(self) -> None: self._relay_finalize_hook_invoked = False async def check(self, level: str) -> dict[str, Any]: + """Observe SDK health when enabled, otherwise the native backends.""" + if ( + self._sdk_enabled + or not self._started + or self._stopping + or level not in {"live", "active", "ready"} + ): + return await self._check_sdk(level) + ports = interfaces.interface_ports(self._settings) + if not ports: + return await self._check_sdk(level) + reason = await self._interfaces.check(level) + if self._stopping or not self._started: + reason = "observation_changed" + return { + "outcome": "passed" if reason is None else "failed", + "reason_code": reason, + "activity": "unknown", + "source": { + "live": "fabric.adapter.liveness", + "active": "fabric.adapter.interface", + "ready": "fabric.adapter.readiness", + }[level], + "observed_at": datetime.now(timezone.utc) + .isoformat(timespec="milliseconds") + .replace("+00:00", "Z"), + } + + async def _check_sdk(self, level: str) -> dict[str, Any]: """Observe the owned Hermes session through its native diagnostic interface.""" result: dict[str, Any] = { "outcome": "unknown", @@ -622,6 +687,12 @@ def observed(outcome: str, reason: str | None) -> dict[str, Any]: return observed("passed", None) async def stop(self) -> None: + """Stop owned native processes and close resources created by the SDK. + + Cleanup attempts every resource even after a failure, then reports + errors without closing parent MCP resources that this runtime never + initialized. + """ self._stopping = True active_invoke_task = self._active_invoke_task errors: list[BaseException] = [] @@ -640,7 +711,8 @@ async def stop(self) -> None: agent = self._agent session_db = self._session_db - had_mcp_servers = bool(self._hermes_config.get("mcp_servers")) + had_mcp_servers = self._owns_mcp + self._owns_mcp = False had_relay_plugin = self._relay_plugin_config_path is not None self._agent = None self._session_db = None diff --git a/adapters/python/hermes/src/nemo_fabric_adapters/hermes/interfaces.py b/adapters/python/hermes/src/nemo_fabric_adapters/hermes/interfaces.py index 36f25f52a..214a5567a 100644 --- a/adapters/python/hermes/src/nemo_fabric_adapters/hermes/interfaces.py +++ b/adapters/python/hermes/src/nemo_fabric_adapters/hermes/interfaces.py @@ -1,7 +1,7 @@ # SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 -"""Serve Hermes's API server and dashboard as children of a managed runtime.""" +"""Own Hermes API and shared browser/terminal TUI backend processes.""" from __future__ import annotations @@ -13,7 +13,7 @@ import secrets import signal import sys -from dataclasses import dataclass +from dataclasses import dataclass, field from pathlib import Path from typing import Any @@ -22,7 +22,7 @@ from nemo_fabric_adapters.common import local_interfaces from nemo_fabric_adapters.common import utils as common_utils -HEALTH_PATHS = {"api": "/health", "dashboard": "/api/status"} +HEALTH_PATHS = {"api": "/health", "dashboard": "/api/status", "terminal": "/api/status"} LOGGER = logging.getLogger(__name__) # Children run in their own sessions and stop by process group. POSIX = os.name != "nt" @@ -39,11 +39,23 @@ def interface_ports(settings: dict[str, Any]) -> dict[str, int]: The descriptor's schema has already checked the settings. """ - return { + ports = { name: interface["port"] for name, interface in settings.get("interfaces", {}).items() if interface.get("enabled", True) } + if "dashboard" in ports and "terminal" in ports: + if ports["dashboard"] != ports["terminal"]: + raise lifecycle.LifecycleError( + "hermes_tui_port_conflict", + "Dashboard and terminal must use the same TUI backend port", + ) + del ports["terminal"] + if len(set(ports.values())) != len(ports): + raise lifecycle.LifecycleError( + "hermes_interface_port_conflict", "API and TUI backends need distinct ports" + ) + return ports def startup_timeout(settings: dict[str, Any]) -> float: @@ -68,6 +80,7 @@ class _Child: name: str port: int process: asyncio.subprocess.Process + tui_token: str | None = field(default=None, repr=False) class InterfaceChildren: @@ -77,6 +90,8 @@ def __init__(self) -> None: self._children: list[_Child] = [] self._token_path: Path | None = None self._agent_config_path: Path | None = None + self._tui_token_path: Path | None = None + self._tui_descriptor_path: Path | None = None async def start( self, @@ -87,8 +102,10 @@ async def start( timeout: float, environment: dict[str, str], agent_config: AgentConfig, + terminal_enabled: bool = False, + workspace: str = ".", ) -> None: - """Start the enabled interfaces and write the API token once all pass health.""" + """Start each backend once and publish credentials after all are ready.""" require_supported_platform() for name, port in ports.items(): if not local_interfaces.port_available(port): @@ -102,6 +119,7 @@ async def start( ) web_dist = _web_dist() if "dashboard" in ports else None token = secrets.token_urlsafe(32) + tui_token = secrets.token_urlsafe(32) log_dir = hermes_home / "logs" log_dir.mkdir(parents=True, exist_ok=True) deadline = asyncio.get_running_loop().time() + timeout @@ -113,7 +131,11 @@ async def start( self._agent_config_path = config_path for name, port in ports.items(): arguments, child_environment = _child_command( - name, port, token=token, web_dist=web_dist, environment=environment + name, + port, + token=token if name == "api" else tui_token, + web_dist=web_dist, + environment=environment, ) with open(log_dir / f"fabric-interface-{name}.log", "ab") as log: process = await asyncio.create_subprocess_exec( @@ -129,9 +151,30 @@ async def start( stderr=asyncio.subprocess.STDOUT, start_new_session=True, ) - self._children.append(_Child(name, port, process)) + self._children.append( + _Child(name, port, process, tui_token if name != "api" else None) + ) for child in self._children: await _wait_healthy(child, deadline) + if terminal_enabled: + tui_name = "dashboard" if "dashboard" in ports else "terminal" + self._tui_token_path = token_path.with_name("tui.token") + self._tui_descriptor_path = token_path.with_name("tui.json") + local_interfaces.write_private_file(self._tui_token_path, tui_token) + local_interfaces.write_private_file( + self._tui_descriptor_path, + json.dumps( + { + "version": 1, + "mode": "dashboard" + if tui_name == "dashboard" + else "headless", + "port": ports[tui_name], + "token_file": "tui.token", + "workspace": str(Path(workspace).resolve()), + } + ), + ) if "api" in ports: local_interfaces.write_private_file(token_path, token) self._token_path = token_path @@ -139,9 +182,34 @@ async def start( await self.stop() raise + async def check(self, level: str) -> str | None: + """Return a contract health reason for owned processes and endpoints.""" + children = tuple(self._children) + if not children: + return "runtime_unavailable" + for child in children: + if child.process.returncode is not None: + return "agent_exited" + if ( + level != "live" + and await _http_status(child.port, HEALTH_PATHS[child.name]) != 200 + ): + return "endpoint_unavailable" + if ( + level != "live" + and child.tui_token + and not await tui_ready(child.port, child.tui_token) + ): + return "endpoint_unavailable" + if children != tuple(self._children): + return "observation_changed" + return None + async def stop(self) -> None: - """Remove the API token, then stop every interface process group.""" + """Remove attachment credentials and stop every owned process group.""" self._remove_private_file("_token_path") + self._remove_private_file("_tui_descriptor_path") + self._remove_private_file("_tui_token_path") running = [ child for child in self._children if child.process.returncode is None ] @@ -164,6 +232,7 @@ async def stop(self) -> None: await asyncio.wait(pending) def _remove_private_file(self, attribute: str) -> None: + """Remove a private file, retaining its path for retry after a failure.""" path = getattr(self, attribute) if path is not None: try: @@ -176,6 +245,7 @@ def _remove_private_file(self, attribute: str) -> None: def _web_dist() -> Path: + """Resolve built browser assets without installing or building a frontend.""" configured = os.environ.get("HERMES_WEB_DIST") if configured: dist = Path(configured) @@ -194,6 +264,7 @@ def _web_dist() -> Path: async def _wait_healthy(child: _Child, deadline: float) -> None: + """Wait for native health and TUI authentication within the shared budget.""" loop = asyncio.get_running_loop() while loop.time() < deadline: if child.process.returncode is not None: @@ -203,7 +274,8 @@ async def _wait_healthy(child: _Child, deadline: float) -> None: metadata={"interface": child.name}, ) if await _http_status(child.port, HEALTH_PATHS[child.name]) == 200: - return + if not child.tui_token or await tui_ready(child.port, child.tui_token): + return await asyncio.sleep(0.1) raise lifecycle.LifecycleError( "hermes_interface_start_timeout", @@ -226,6 +298,14 @@ def _child_command( key: value for key, value in environment.items() if not key.startswith("API_SERVER_") + and key + not in { + "HERMES_DASHBOARD_SESSION_TOKEN", + "HERMES_TUI_GATEWAY_URL", + "HERMES_TUI_SIDECAR_URL", + "HERMES_DESKTOP", + "HERMES_SERVE_HEADLESS", + } } if name == "api": child_environment.update( @@ -234,6 +314,17 @@ def _child_command( API_SERVER_PORT=str(port), ) return ["gateway", "run"], child_environment + child_environment["HERMES_DASHBOARD_SESSION_TOKEN"] = token + if name == "terminal": + child_environment.pop("HERMES_WEB_DIST", None) + return [ + "serve", + "--host", + "127.0.0.1", + "--port", + str(port), + "--isolated", + ], child_environment child_environment["HERMES_WEB_DIST"] = str(web_dist) # --isolated keeps one dashboard per runtime instead of attaching to another # runtime's dashboard on this host. @@ -267,3 +358,34 @@ def get() -> int | None: connection.close() return await asyncio.to_thread(get) + + +async def tui_ready(port: int, token: str) -> bool: + """Check authenticated TUI RPC, without creating a conversation or model call.""" + import aiohttp + + try: + async with ( + asyncio.timeout(2), + aiohttp.ClientSession(timeout=aiohttp.ClientTimeout(total=2)) as client, + ): + async with client.ws_connect( + f"http://127.0.0.1:{port}/api/ws", params={"token": token} + ) as ws: + await ws.send_json( + { + "jsonrpc": "2.0", + "id": "fabric-health", + "method": "session.list", + "params": {}, + } + ) + async for message in ws: + if message.type != aiohttp.WSMsgType.TEXT: + return False + reply = json.loads(message.data) + if reply.get("id") == "fabric-health": + return "result" in reply + except (aiohttp.ClientError, TimeoutError, OSError, ValueError): + return False + return False diff --git a/adapters/python/hermes/src/nemo_fabric_adapters/hermes/native.py b/adapters/python/hermes/src/nemo_fabric_adapters/hermes/native.py index ff30eabdc..51f742b0e 100644 --- a/adapters/python/hermes/src/nemo_fabric_adapters/hermes/native.py +++ b/adapters/python/hermes/src/nemo_fabric_adapters/hermes/native.py @@ -27,17 +27,15 @@ LOGGER = logging.getLogger(__name__) -def finalize_api_turn(agent: Any) -> None: - """Close the API turn's Relay segment without deleting native conversation state.""" - if agent.platform != "api_server" or not os.environ.get( - "HERMES_NEMO_RELAY_PLUGINS_TOML" - ): +def finalize_native_turn(agent: Any) -> None: + """Close the native turn's Relay segment without deleting native conversation state.""" + if not os.environ.get("HERMES_NEMO_RELAY_PLUGINS_TOML"): return try: telemetry.finalize_native_relay_segment(agent.session_id) except Exception: # A telemetry failure must not replace an execution result or its error. - LOGGER.exception("Hermes API turn Relay finalization failed") + LOGGER.exception("Hermes native turn Relay finalization failed") def configured_kwargs( @@ -64,7 +62,44 @@ def configured_kwargs( return result +def configure_browser_attachment() -> None: + """Keep browser PTYs on this runtime's backend, including its default profile.""" + from hermes_cli import web_server_chat + from nemo_fabric_adapters.hermes.terminal import frontend_command + + original = web_server_chat._resolve_chat_argv + signature = inspect.signature(original) + + @wraps(original) + def managed_chat(*args: Any, **kwargs: Any) -> Any: + """Attach the prebuilt renderer only to this runtime's default profile.""" + bound = signature.bind(*args, **kwargs) + profile = bound.arguments.get("profile") + if profile not in {None, "", "current", "default"}: + raise ValueError( + "Managed Hermes browser chat supports only the runtime's profile" + ) + # The pinned resolver skips its in-process backend for explicit profile + # names, even 'default'. Select this process's home instead. + bound.arguments["profile"] = "current" + command = frontend_command() + # Force the prebuilt branch in the native resolver to avoid PM/build. + os.environ["HERMES_NODE"] = command[0] + entry = Path(command[-1]) + if entry.parent.name == "dist": + os.environ["HERMES_TUI_DIR"] = str(entry.parent.parent) + argv, cwd, environment = original(*bound.args, **bound.kwargs) + if not environment.get("HERMES_TUI_GATEWAY_URL"): + raise ValueError( + "Managed Hermes browser backend has no attachment endpoint" + ) + return command, cwd, environment + + web_server_chat._resolve_chat_argv = managed_chat + + def main() -> None: + """Apply runtime-owned agent settings before entering the native CLI.""" parser = argparse.ArgumentParser(add_help=False) parser.add_argument("--fabric-config", type=Path, required=True) options, arguments = parser.parse_known_args() @@ -78,6 +113,7 @@ def main() -> None: class ConfiguredAIAgent(native_agent): def __init__(self, *args: Any, **kwargs: Any) -> None: + """Apply normalized behavior while retaining native agent identity.""" bound = signature.bind(*args, **kwargs) super().__init__( **configured_kwargs(config, bound.arguments, load_config()) @@ -85,12 +121,15 @@ def __init__(self, *args: Any, **kwargs: Any) -> None: @wraps(native_agent.run_conversation) def run_conversation(self, *args: Any, **kwargs: Any) -> Any: + """Run a native turn and finalize its Relay segment on every exit.""" try: return super().run_conversation(*args, **kwargs) finally: - finalize_api_turn(self) + finalize_native_turn(self) run_agent.AIAgent = ConfiguredAIAgent + if arguments and arguments[0] in {"dashboard", "serve"}: + configure_browser_attachment() from hermes_cli.main import main as hermes_main sys.argv = ["hermes", *arguments] diff --git a/adapters/python/hermes/src/nemo_fabric_adapters/hermes/terminal.py b/adapters/python/hermes/src/nemo_fabric_adapters/hermes/terminal.py new file mode 100644 index 000000000..8a02f0d07 --- /dev/null +++ b/adapters/python/hermes/src/nemo_fabric_adapters/hermes/terminal.py @@ -0,0 +1,117 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +"""Attach the native terminal renderer to a NeMo Fabric-owned TUI service.""" + +from __future__ import annotations + +import argparse +import asyncio +import json +import os +import stat +import shutil +import sys +from pathlib import Path +from urllib.parse import urlencode + +from nemo_fabric_adapters.hermes.interfaces import tui_ready + + +def _read_private(path: Path) -> str: + """Read an owner-only regular file without following a leaf symlink.""" + descriptor = os.open(path, os.O_RDONLY | os.O_NOFOLLOW) + with os.fdopen(descriptor) as stream: + info = os.fstat(stream.fileno()) + if ( + not stat.S_ISREG(info.st_mode) + or info.st_uid != os.getuid() + or info.st_mode & 0o077 + ): + raise ValueError("TUI attachment files must be owner-only regular files") + return stream.read() + + +def frontend_command() -> list[str]: + """Use installed assets directly; never install/build another Hermes runtime.""" + import hermes_cli + + cli = Path(hermes_cli.__file__).parent + configured = os.environ.get("HERMES_TUI_DIR") + candidates = ( + [Path(configured) / "dist/entry.js"] + if configured + else [cli / "tui_dist/entry.js", cli.parent / "ui-tui/dist/entry.js"] + ) + entry = next((path for path in candidates if path.is_file()), None) + node = os.environ.get("HERMES_NODE") or shutil.which("node") + if entry is None or not node or not os.access(node, os.X_OK): + raise ValueError( + "Install the pinned Hermes terminal bundle and Node runtime before attaching" + ) + return [str(node), "--expose-gc", str(entry.resolve())] + + +def attachment_environment(connection: Path) -> dict[str, str]: + """Validate the private descriptor and refuse stale/unauthenticated backends.""" + payload = json.loads(_read_private(connection)) + port = payload["port"] + if ( + payload["version"] != 1 + or payload["mode"] not in {"dashboard", "headless"} + or type(port) is not int + or not 1 <= port <= 65535 + or payload["token_file"] != "tui.token" + ): + raise ValueError("Invalid TUI attachment descriptor") + token = _read_private(connection.with_name("tui.token")).strip() + if not token or not asyncio.run(tui_ready(port, token)): + raise ValueError("Managed TUI backend is unavailable or authentication failed") + environment = { + key: value + for key, value in os.environ.items() + if key + not in {"HERMES_TUI_SIDECAR_URL", "HERMES_DESKTOP", "HERMES_SERVE_HEADLESS"} + } + environment["HERMES_PYTHON"] = sys.executable + environment["HERMES_TUI_CWD"] = payload["workspace"] + environment["HERMES_CWD"] = payload["workspace"] + environment["HERMES_HOME"] = str(connection.resolve().parent.parent) + environment["HERMES_TUI_GATEWAY_URL"] = ( + f"ws://127.0.0.1:{port}/api/ws?{urlencode({'token': token})}" + ) + return environment + + +def main() -> None: + """Validate a local attachment and replace this client with the renderer.""" + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument( + "--connection", + type=Path, + required=True, + help="Runtime's private interfaces/tui.json file", + ) + options = parser.parse_args() + if os.name != "posix": + parser.exit(1, "Terminal attachment requires a POSIX host\n") + if not sys.stdin.isatty() or not sys.stdout.isatty(): + parser.exit(1, "Run terminal attachment from an interactive terminal\n") + try: + environment = attachment_environment(options.connection) + except (OSError, ValueError, KeyError): + parser.exit( + 1, + "Cannot attach to the managed TUI backend; check the runtime and private attachment files\n", + ) + try: + arguments = frontend_command() + except ValueError as error: + parser.exit(1, f"{error}\n") + # The explicit URL makes the native renderer an attaching client, never a + # backend owner. Credentials stay in its environment, not process arguments. + os.execvpe(arguments[0], arguments, environment) + + +if __name__ == "__main__": + main() diff --git a/assets/hermes-invocation-paths.excalidraw b/assets/hermes-invocation-paths.excalidraw new file mode 100644 index 000000000..f0fd8c741 --- /dev/null +++ b/assets/hermes-invocation-paths.excalidraw @@ -0,0 +1,1320 @@ +{ + "type": "excalidraw", + "version": 2, + "source": "https://excalidraw.com", + "elements": [ + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "version": 1, + "isDeleted": false, + "boundElements": null, + "link": null, + "locked": false, + "id": "hermes-simple-1", + "type": "text", + "x": 40, + "y": 25, + "width": 412.16, + "height": 40.0, + "strokeColor": "#245400", + "backgroundColor": "transparent", + "seed": 1, + "versionNonce": 1, + "updated": 1794182400000, + "fontSize": 32, + "fontFamily": 2, + "text": "Hermes invocation paths", + "originalText": "Hermes invocation paths", + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "autoResize": true, + "lineHeight": 1.25 + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "version": 1, + "isDeleted": false, + "boundElements": null, + "link": null, + "locked": false, + "id": "hermes-simple-2", + "type": "text", + "x": 40, + "y": 73, + "width": 492.80000000000007, + "height": 25.0, + "strokeColor": "#495057", + "backgroundColor": "transparent", + "seed": 2, + "versionNonce": 2, + "updated": 1794182400000, + "fontSize": 20, + "fontFamily": 2, + "text": "Four client paths \u00b7 Three execution backends", + "originalText": "Four client paths \u00b7 Three execution backends", + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "autoResize": true, + "lineHeight": 1.25 + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [ + "hermes-box-3" + ], + "frameId": null, + "roundness": { + "type": 3 + }, + "version": 1, + "isDeleted": false, + "boundElements": null, + "link": null, + "locked": false, + "id": "hermes-simple-3", + "type": "rectangle", + "x": 450, + "y": 130, + "width": 730, + "height": 665, + "strokeColor": "#868e96", + "backgroundColor": "#ffffff", + "seed": 3, + "versionNonce": 3, + "updated": 1794182400000 + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [ + "hermes-box-3" + ], + "frameId": null, + "roundness": null, + "version": 1, + "isDeleted": false, + "boundElements": null, + "link": null, + "locked": false, + "id": "hermes-simple-4", + "type": "text", + "x": 472, + "y": 146, + "width": 244.72000000000003, + "height": 28.75, + "strokeColor": "#868e96", + "backgroundColor": "transparent", + "seed": 4, + "versionNonce": 4, + "updated": 1794182400000, + "fontSize": 23, + "fontFamily": 2, + "text": "NeMo Fabric runtime", + "originalText": "NeMo Fabric runtime", + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "autoResize": true, + "lineHeight": 1.25 + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [ + "hermes-box-5" + ], + "frameId": null, + "roundness": { + "type": 3 + }, + "version": 1, + "isDeleted": false, + "boundElements": null, + "link": null, + "locked": false, + "id": "hermes-simple-5", + "type": "rectangle", + "x": 500, + "y": 190, + "width": 630, + "height": 110, + "strokeColor": "#5c7c20", + "backgroundColor": "#f4fce3", + "seed": 5, + "versionNonce": 5, + "updated": 1794182400000 + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [ + "hermes-box-5" + ], + "frameId": null, + "roundness": null, + "version": 1, + "isDeleted": false, + "boundElements": null, + "link": null, + "locked": false, + "id": "hermes-simple-6", + "type": "text", + "x": 522, + "y": 206, + "width": 246.40000000000003, + "height": 27.5, + "strokeColor": "#5c7c20", + "backgroundColor": "transparent", + "seed": 6, + "versionNonce": 6, + "updated": 1794182400000, + "fontSize": 22, + "fontFamily": 2, + "text": "Shared configuration", + "originalText": "Shared configuration", + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "autoResize": true, + "lineHeight": 1.25 + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [ + "hermes-box-5" + ], + "frameId": null, + "roundness": null, + "version": 1, + "isDeleted": false, + "boundElements": null, + "link": null, + "locked": false, + "id": "hermes-simple-7", + "type": "text", + "x": 522, + "y": 239, + "width": 322.56000000000006, + "height": 45.0, + "strokeColor": "#343a40", + "backgroundColor": "transparent", + "seed": 7, + "versionNonce": 7, + "updated": 1794182400000, + "fontSize": 18, + "fontFamily": 2, + "text": "Model \u00b7 Instructions \u00b7 Tools/MCP\nWorkspace \u00b7 NeMo Relay", + "originalText": "Model \u00b7 Instructions \u00b7 Tools/MCP\nWorkspace \u00b7 NeMo Relay", + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "autoResize": true, + "lineHeight": 1.25 + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [ + "hermes-box-8" + ], + "frameId": null, + "roundness": { + "type": 3 + }, + "version": 1, + "isDeleted": false, + "boundElements": [ + { + "id": "hermes-simple-29", + "type": "arrow" + } + ], + "link": null, + "locked": false, + "id": "hermes-simple-8", + "type": "rectangle", + "x": 40, + "y": 340, + "width": 300, + "height": 95, + "strokeColor": "#1971c2", + "backgroundColor": "#e7f5ff", + "seed": 8, + "versionNonce": 8, + "updated": 1794182400000 + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [ + "hermes-box-8" + ], + "frameId": null, + "roundness": null, + "version": 1, + "isDeleted": false, + "boundElements": null, + "link": null, + "locked": false, + "id": "hermes-simple-9", + "type": "text", + "x": 62, + "y": 356, + "width": 123.20000000000002, + "height": 27.5, + "strokeColor": "#1971c2", + "backgroundColor": "transparent", + "seed": 9, + "versionNonce": 9, + "updated": 1794182400000, + "fontSize": 22, + "fontFamily": 2, + "text": "SDK client", + "originalText": "SDK client", + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "autoResize": true, + "lineHeight": 1.25 + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [ + "hermes-box-8" + ], + "frameId": null, + "roundness": null, + "version": 1, + "isDeleted": false, + "boundElements": null, + "link": null, + "locked": false, + "id": "hermes-simple-10", + "type": "text", + "x": 62, + "y": 389, + "width": 262.08000000000004, + "height": 22.5, + "strokeColor": "#343a40", + "backgroundColor": "transparent", + "seed": 10, + "versionNonce": 10, + "updated": 1794182400000, + "fontSize": 18, + "fontFamily": 2, + "text": "invoke() / invoke_stream()", + "originalText": "invoke() / invoke_stream()", + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "autoResize": true, + "lineHeight": 1.25 + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [ + "hermes-box-11" + ], + "frameId": null, + "roundness": { + "type": 3 + }, + "version": 1, + "isDeleted": false, + "boundElements": [ + { + "id": "hermes-simple-29", + "type": "arrow" + } + ], + "link": null, + "locked": false, + "id": "hermes-simple-11", + "type": "rectangle", + "x": 500, + "y": 340, + "width": 630, + "height": 95, + "strokeColor": "#1971c2", + "backgroundColor": "#e7f5ff", + "seed": 11, + "versionNonce": 11, + "updated": 1794182400000 + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [ + "hermes-box-11" + ], + "frameId": null, + "roundness": null, + "version": 1, + "isDeleted": false, + "boundElements": null, + "link": null, + "locked": false, + "id": "hermes-simple-12", + "type": "text", + "x": 522, + "y": 356, + "width": 221.76000000000002, + "height": 27.5, + "strokeColor": "#1971c2", + "backgroundColor": "transparent", + "seed": 12, + "versionNonce": 12, + "updated": 1794182400000, + "fontSize": 22, + "fontFamily": 2, + "text": "Embedded SDK agent", + "originalText": "Embedded SDK agent", + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "autoResize": true, + "lineHeight": 1.25 + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [ + "hermes-box-11" + ], + "frameId": null, + "roundness": null, + "version": 1, + "isDeleted": false, + "boundElements": null, + "link": null, + "locked": false, + "id": "hermes-simple-13", + "type": "text", + "x": 522, + "y": 389, + "width": 131.04000000000002, + "height": 22.5, + "strokeColor": "#343a40", + "backgroundColor": "transparent", + "seed": 13, + "versionNonce": 13, + "updated": 1794182400000, + "fontSize": 18, + "fontFamily": 2, + "text": "On by default", + "originalText": "On by default", + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "autoResize": true, + "lineHeight": 1.25 + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [ + "hermes-box-14" + ], + "frameId": null, + "roundness": { + "type": 3 + }, + "version": 1, + "isDeleted": false, + "boundElements": [ + { + "id": "hermes-simple-30", + "type": "arrow" + } + ], + "link": null, + "locked": false, + "id": "hermes-simple-14", + "type": "rectangle", + "x": 40, + "y": 480, + "width": 300, + "height": 95, + "strokeColor": "#7048a5", + "backgroundColor": "#f3f0ff", + "seed": 14, + "versionNonce": 14, + "updated": 1794182400000 + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [ + "hermes-box-14" + ], + "frameId": null, + "roundness": null, + "version": 1, + "isDeleted": false, + "boundElements": null, + "link": null, + "locked": false, + "id": "hermes-simple-15", + "type": "text", + "x": 62, + "y": 496, + "width": 135.52, + "height": 27.5, + "strokeColor": "#7048a5", + "backgroundColor": "transparent", + "seed": 15, + "versionNonce": 15, + "updated": 1794182400000, + "fontSize": 22, + "fontFamily": 2, + "text": "HTTP client", + "originalText": "HTTP client", + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "autoResize": true, + "lineHeight": 1.25 + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [ + "hermes-box-14" + ], + "frameId": null, + "roundness": null, + "version": 1, + "isDeleted": false, + "boundElements": null, + "link": null, + "locked": false, + "id": "hermes-simple-16", + "type": "text", + "x": 62, + "y": 529, + "width": 191.52, + "height": 22.5, + "strokeColor": "#343a40", + "backgroundColor": "transparent", + "seed": 16, + "versionNonce": 16, + "updated": 1794182400000, + "fontSize": 18, + "fontFamily": 2, + "text": "Native API requests", + "originalText": "Native API requests", + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "autoResize": true, + "lineHeight": 1.25 + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [ + "hermes-box-17" + ], + "frameId": null, + "roundness": { + "type": 3 + }, + "version": 1, + "isDeleted": false, + "boundElements": [ + { + "id": "hermes-simple-30", + "type": "arrow" + } + ], + "link": null, + "locked": false, + "id": "hermes-simple-17", + "type": "rectangle", + "x": 500, + "y": 480, + "width": 630, + "height": 95, + "strokeColor": "#7048a5", + "backgroundColor": "#f3f0ff", + "seed": 17, + "versionNonce": 17, + "updated": 1794182400000 + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [ + "hermes-box-17" + ], + "frameId": null, + "roundness": null, + "version": 1, + "isDeleted": false, + "boundElements": null, + "link": null, + "locked": false, + "id": "hermes-simple-18", + "type": "text", + "x": 522, + "y": 496, + "width": 221.76000000000002, + "height": 27.5, + "strokeColor": "#7048a5", + "backgroundColor": "transparent", + "seed": 18, + "versionNonce": 18, + "updated": 1794182400000, + "fontSize": 22, + "fontFamily": 2, + "text": "Native API gateway", + "originalText": "Native API gateway", + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "autoResize": true, + "lineHeight": 1.25 + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [ + "hermes-box-17" + ], + "frameId": null, + "roundness": null, + "version": 1, + "isDeleted": false, + "boundElements": null, + "link": null, + "locked": false, + "id": "hermes-simple-19", + "type": "text", + "x": 522, + "y": 529, + "width": 221.76000000000002, + "height": 22.5, + "strokeColor": "#343a40", + "backgroundColor": "transparent", + "seed": 19, + "versionNonce": 19, + "updated": 1794182400000, + "fontSize": 18, + "fontFamily": 2, + "text": "Separate conversations", + "originalText": "Separate conversations", + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "autoResize": true, + "lineHeight": 1.25 + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [ + "hermes-box-20" + ], + "frameId": null, + "roundness": { + "type": 3 + }, + "version": 1, + "isDeleted": false, + "boundElements": [ + { + "id": "hermes-simple-31", + "type": "arrow" + } + ], + "link": null, + "locked": false, + "id": "hermes-simple-20", + "type": "rectangle", + "x": 40, + "y": 620, + "width": 300, + "height": 80, + "strokeColor": "#2b8a3e", + "backgroundColor": "#ebfbee", + "seed": 20, + "versionNonce": 20, + "updated": 1794182400000 + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [ + "hermes-box-20" + ], + "frameId": null, + "roundness": null, + "version": 1, + "isDeleted": false, + "boundElements": null, + "link": null, + "locked": false, + "id": "hermes-simple-21", + "type": "text", + "x": 62, + "y": 636, + "width": 86.24000000000001, + "height": 27.5, + "strokeColor": "#2b8a3e", + "backgroundColor": "transparent", + "seed": 21, + "versionNonce": 21, + "updated": 1794182400000, + "fontSize": 22, + "fontFamily": 2, + "text": "Browser", + "originalText": "Browser", + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "autoResize": true, + "lineHeight": 1.25 + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [ + "hermes-box-20" + ], + "frameId": null, + "roundness": null, + "version": 1, + "isDeleted": false, + "boundElements": null, + "link": null, + "locked": false, + "id": "hermes-simple-22", + "type": "text", + "x": 62, + "y": 669, + "width": 141.12, + "height": 22.5, + "strokeColor": "#343a40", + "backgroundColor": "transparent", + "seed": 22, + "versionNonce": 22, + "updated": 1794182400000, + "fontSize": 18, + "fontFamily": 2, + "text": "Dashboard chat", + "originalText": "Dashboard chat", + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "autoResize": true, + "lineHeight": 1.25 + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [ + "hermes-box-23" + ], + "frameId": null, + "roundness": { + "type": 3 + }, + "version": 1, + "isDeleted": false, + "boundElements": [ + { + "id": "hermes-simple-32", + "type": "arrow" + } + ], + "link": null, + "locked": false, + "id": "hermes-simple-23", + "type": "rectangle", + "x": 40, + "y": 725, + "width": 300, + "height": 80, + "strokeColor": "#2b8a3e", + "backgroundColor": "#ebfbee", + "seed": 23, + "versionNonce": 23, + "updated": 1794182400000 + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [ + "hermes-box-23" + ], + "frameId": null, + "roundness": null, + "version": 1, + "isDeleted": false, + "boundElements": null, + "link": null, + "locked": false, + "id": "hermes-simple-24", + "type": "text", + "x": 62, + "y": 741, + "width": 98.56, + "height": 27.5, + "strokeColor": "#2b8a3e", + "backgroundColor": "transparent", + "seed": 24, + "versionNonce": 24, + "updated": 1794182400000, + "fontSize": 22, + "fontFamily": 2, + "text": "Terminal", + "originalText": "Terminal", + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "autoResize": true, + "lineHeight": 1.25 + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [ + "hermes-box-23" + ], + "frameId": null, + "roundness": null, + "version": 1, + "isDeleted": false, + "boundElements": null, + "link": null, + "locked": false, + "id": "hermes-simple-25", + "type": "text", + "x": 62, + "y": 774, + "width": 100.80000000000001, + "height": 22.5, + "strokeColor": "#343a40", + "backgroundColor": "transparent", + "seed": 25, + "versionNonce": 25, + "updated": 1794182400000, + "fontSize": 18, + "fontFamily": 2, + "text": "Hermes TUI", + "originalText": "Hermes TUI", + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "autoResize": true, + "lineHeight": 1.25 + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [ + "hermes-box-26" + ], + "frameId": null, + "roundness": { + "type": 3 + }, + "version": 1, + "isDeleted": false, + "boundElements": [ + { + "id": "hermes-simple-31", + "type": "arrow" + }, + { + "id": "hermes-simple-32", + "type": "arrow" + } + ], + "link": null, + "locked": false, + "id": "hermes-simple-26", + "type": "rectangle", + "x": 500, + "y": 645, + "width": 630, + "height": 120, + "strokeColor": "#2b8a3e", + "backgroundColor": "#ebfbee", + "seed": 26, + "versionNonce": 26, + "updated": 1794182400000 + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [ + "hermes-box-26" + ], + "frameId": null, + "roundness": null, + "version": 1, + "isDeleted": false, + "boundElements": null, + "link": null, + "locked": false, + "id": "hermes-simple-27", + "type": "text", + "x": 522, + "y": 661, + "width": 320.32000000000005, + "height": 27.5, + "strokeColor": "#2b8a3e", + "backgroundColor": "transparent", + "seed": 27, + "versionNonce": 27, + "updated": 1794182400000, + "fontSize": 22, + "fontFamily": 2, + "text": "Browser + terminal backend", + "originalText": "Browser + terminal backend", + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "autoResize": true, + "lineHeight": 1.25 + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [ + "hermes-box-26" + ], + "frameId": null, + "roundness": null, + "version": 1, + "isDeleted": false, + "boundElements": null, + "link": null, + "locked": false, + "id": "hermes-simple-28", + "type": "text", + "x": 522, + "y": 694, + "width": 262.08000000000004, + "height": 22.5, + "strokeColor": "#343a40", + "backgroundColor": "transparent", + "seed": 28, + "versionNonce": 28, + "updated": 1794182400000, + "fontSize": 18, + "fontFamily": 2, + "text": "Dashboard or headless mode", + "originalText": "Dashboard or headless mode", + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "autoResize": true, + "lineHeight": 1.25 + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "version": 1, + "isDeleted": false, + "boundElements": null, + "link": null, + "locked": false, + "id": "hermes-simple-29", + "type": "arrow", + "x": 348, + "y": 387, + "width": 144, + "height": 0, + "strokeColor": "#1971c2", + "backgroundColor": "transparent", + "seed": 29, + "versionNonce": 29, + "updated": 1794182400000, + "points": [ + [ + 0, + 0 + ], + [ + 144, + 0 + ] + ], + "startBinding": { + "elementId": "hermes-simple-8", + "focus": 0, + "gap": 8 + }, + "endBinding": { + "elementId": "hermes-simple-11", + "focus": 0, + "gap": 8 + }, + "startArrowhead": null, + "endArrowhead": "arrow", + "elbowed": false + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "version": 1, + "isDeleted": false, + "boundElements": null, + "link": null, + "locked": false, + "id": "hermes-simple-30", + "type": "arrow", + "x": 348, + "y": 527, + "width": 144, + "height": 0, + "strokeColor": "#7048a5", + "backgroundColor": "transparent", + "seed": 30, + "versionNonce": 30, + "updated": 1794182400000, + "points": [ + [ + 0, + 0 + ], + [ + 144, + 0 + ] + ], + "startBinding": { + "elementId": "hermes-simple-14", + "focus": 0, + "gap": 8 + }, + "endBinding": { + "elementId": "hermes-simple-17", + "focus": 0, + "gap": 8 + }, + "startArrowhead": null, + "endArrowhead": "arrow", + "elbowed": false + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "version": 1, + "isDeleted": false, + "boundElements": null, + "link": null, + "locked": false, + "id": "hermes-simple-31", + "type": "arrow", + "x": 348, + "y": 660, + "width": 144, + "height": 22, + "strokeColor": "#2b8a3e", + "backgroundColor": "transparent", + "seed": 31, + "versionNonce": 31, + "updated": 1794182400000, + "points": [ + [ + 0, + 0 + ], + [ + 144, + 22 + ] + ], + "startBinding": { + "elementId": "hermes-simple-20", + "focus": 0, + "gap": 8 + }, + "endBinding": { + "elementId": "hermes-simple-26", + "focus": 0, + "gap": 8 + }, + "startArrowhead": null, + "endArrowhead": "arrow", + "elbowed": false + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "version": 1, + "isDeleted": false, + "boundElements": null, + "link": null, + "locked": false, + "id": "hermes-simple-32", + "type": "arrow", + "x": 348, + "y": 765, + "width": 144, + "height": 37, + "strokeColor": "#2b8a3e", + "backgroundColor": "transparent", + "seed": 32, + "versionNonce": 32, + "updated": 1794182400000, + "points": [ + [ + 0, + 0 + ], + [ + 144, + -37 + ] + ], + "startBinding": { + "elementId": "hermes-simple-23", + "focus": 0, + "gap": 8 + }, + "endBinding": { + "elementId": "hermes-simple-26", + "focus": 0, + "gap": 8 + }, + "startArrowhead": null, + "endArrowhead": "arrow", + "elbowed": false + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "version": 1, + "isDeleted": false, + "boundElements": null, + "link": null, + "locked": false, + "id": "hermes-simple-33", + "type": "text", + "x": 40, + "y": 850, + "width": 683.2, + "height": 25.0, + "strokeColor": "#212529", + "backgroundColor": "transparent", + "seed": 33, + "versionNonce": 33, + "updated": 1794182400000, + "fontSize": 20, + "fontFamily": 2, + "text": "\u2022 Same Hermes agent loop; separate agents and conversations.", + "originalText": "\u2022 Same Hermes agent loop; separate agents and conversations.", + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "autoResize": true, + "lineHeight": 1.25 + }, + { + "angle": 0, + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "version": 1, + "isDeleted": false, + "boundElements": null, + "link": null, + "locked": false, + "id": "hermes-simple-34", + "type": "text", + "x": 40, + "y": 890, + "width": 828.8000000000001, + "height": 50.0, + "strokeColor": "#212529", + "backgroundColor": "transparent", + "seed": 34, + "versionNonce": 34, + "updated": 1794182400000, + "fontSize": 20, + "fontFamily": 2, + "text": "\u2022 SDK returns normalized results; native clients return native responses.\n NeMo Relay provides common trajectory formats across all paths.", + "originalText": "\u2022 SDK returns normalized results; native clients return native responses.\n NeMo Relay provides common trajectory formats across all paths.", + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "autoResize": true, + "lineHeight": 1.25 + } + ], + "appState": { + "viewBackgroundColor": "#ffffff", + "gridSize": null + }, + "files": {} +} diff --git a/assets/hermes-invocation-paths.png b/assets/hermes-invocation-paths.png new file mode 100644 index 000000000..ff6865fd4 Binary files /dev/null and b/assets/hermes-invocation-paths.png differ diff --git a/assets/hermes-runtime-invocation.excalidraw b/assets/hermes-runtime-invocation.excalidraw deleted file mode 100644 index 0559c4441..000000000 --- a/assets/hermes-runtime-invocation.excalidraw +++ /dev/null @@ -1,1696 +0,0 @@ -{ - "type": "excalidraw", - "version": 2, - "source": "https://app.excalidraw.com", - "elements": [ - { - "id": "hermes-0", - "type": "text", - "x": 60, - "y": 8, - "width": 1200, - "height": 45, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "transparent", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": null, - "seed": 1000, - "version": 2, - "versionNonce": 970466068, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "text": "Hermes invocation paths: one Fabric runtime", - "originalText": "Hermes invocation paths: one Fabric runtime", - "fontSize": 30, - "fontFamily": 2, - "textAlign": "left", - "verticalAlign": "middle", - "containerId": null, - "autoResize": false, - "lineHeight": 1.25, - "index": "a0", - "created": null, - "labelPosition": null, - "baseFontSize": null - }, - { - "id": "hermes-1", - "type": "rectangle", - "x": 60, - "y": 180, - "width": 1220, - "height": 740, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "white", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": { - "type": 3 - }, - "seed": 1001, - "version": 2, - "versionNonce": 159270956, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "index": "a1", - "created": null - }, - { - "id": "hermes-2", - "type": "rectangle", - "x": 80, - "y": 158, - "width": 780, - "height": 45, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "#3b424b", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": { - "type": 3 - }, - "seed": 1002, - "version": 2, - "versionNonce": 176857236, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "index": "a2", - "created": null - }, - { - "id": "hermes-3", - "type": "text", - "x": 90, - "y": 166, - "width": 760, - "height": 29, - "angle": 0, - "strokeColor": "#ffffff", - "backgroundColor": "transparent", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": null, - "seed": 1003, - "version": 2, - "versionNonce": 828138156, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "text": "One Fabric runtime — shared lifecycle ownership", - "originalText": "One Fabric runtime — shared lifecycle ownership", - "fontSize": 23, - "fontFamily": 2, - "textAlign": "center", - "verticalAlign": "middle", - "containerId": null, - "autoResize": false, - "lineHeight": 1.25, - "index": "a3", - "created": null, - "labelPosition": null, - "baseFontSize": null - }, - { - "id": "hermes-4", - "type": "rectangle", - "x": 110, - "y": 290, - "width": 490, - "height": 380, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "white", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": { - "type": 3 - }, - "seed": 1004, - "version": 2, - "versionNonce": 1796304404, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "index": "a4", - "created": null - }, - { - "id": "hermes-5", - "type": "text", - "x": 130, - "y": 303, - "width": 450, - "height": 45, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "transparent", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": null, - "seed": 1005, - "version": 2, - "versionNonce": 462630188, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "text": "NeMo Fabric adapter host", - "originalText": "NeMo Fabric adapter host", - "fontSize": 24, - "fontFamily": 2, - "textAlign": "center", - "verticalAlign": "middle", - "containerId": null, - "autoResize": false, - "lineHeight": 1.25, - "index": "a5", - "created": null, - "labelPosition": null, - "baseFontSize": null - }, - { - "id": "hermes-6", - "type": "rectangle", - "x": 150, - "y": 385, - "width": 410, - "height": 70, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "#e3f8e3", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": { - "type": 3 - }, - "seed": 1006, - "version": 2, - "versionNonce": 2086833044, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "index": "a6", - "created": null - }, - { - "id": "hermes-7", - "type": "text", - "x": 160, - "y": 393, - "width": 390, - "height": 54, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "transparent", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": null, - "seed": 1007, - "version": 2, - "versionNonce": 1502390188, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "text": "Hermes adapter", - "originalText": "Hermes adapter", - "fontSize": 23, - "fontFamily": 2, - "textAlign": "center", - "verticalAlign": "middle", - "containerId": null, - "autoResize": false, - "lineHeight": 1.25, - "index": "a7", - "created": null, - "labelPosition": null, - "baseFontSize": null - }, - { - "id": "hermes-8", - "type": "rectangle", - "x": 150, - "y": 525, - "width": 410, - "height": 100, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "#e3f8e3", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": { - "type": 3 - }, - "seed": 1008, - "version": 2, - "versionNonce": 1353971988, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "index": "a8", - "created": null - }, - { - "id": "hermes-9", - "type": "text", - "x": 160, - "y": 533, - "width": 390, - "height": 84, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "transparent", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": null, - "seed": 1009, - "version": 2, - "versionNonce": 1064489516, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "text": "Embedded Hermes AIAgent\nFabric conversation", - "originalText": "Embedded Hermes AIAgent\nFabric conversation", - "fontSize": 23, - "fontFamily": 2, - "textAlign": "center", - "verticalAlign": "middle", - "containerId": null, - "autoResize": false, - "lineHeight": 1.25, - "index": "a9", - "created": null, - "labelPosition": null, - "baseFontSize": null - }, - { - "id": "hermes-10", - "type": "rectangle", - "x": 730, - "y": 290, - "width": 490, - "height": 380, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "white", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": { - "type": 3 - }, - "seed": 1010, - "version": 2, - "versionNonce": 1132181140, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "index": "aA", - "created": null - }, - { - "id": "hermes-11", - "type": "text", - "x": 750, - "y": 303, - "width": 450, - "height": 45, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "transparent", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": null, - "seed": 1011, - "version": 2, - "versionNonce": 204450988, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "text": "Managed Hermes Gateway", - "originalText": "Managed Hermes Gateway", - "fontSize": 24, - "fontFamily": 2, - "textAlign": "center", - "verticalAlign": "middle", - "containerId": null, - "autoResize": false, - "lineHeight": 1.25, - "index": "aB", - "created": null, - "labelPosition": null, - "baseFontSize": null - }, - { - "id": "hermes-12", - "type": "rectangle", - "x": 770, - "y": 385, - "width": 410, - "height": 70, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "white", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": { - "type": 3 - }, - "seed": 1012, - "version": 2, - "versionNonce": 1366279188, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "index": "aC", - "created": null - }, - { - "id": "hermes-13", - "type": "text", - "x": 780, - "y": 393, - "width": 390, - "height": 54, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "transparent", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": null, - "seed": 1013, - "version": 2, - "versionNonce": 292370220, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "text": "Native API / chat entry points", - "originalText": "Native API / chat entry points", - "fontSize": 23, - "fontFamily": 2, - "textAlign": "center", - "verticalAlign": "middle", - "containerId": null, - "autoResize": false, - "lineHeight": 1.25, - "index": "aD", - "created": null, - "labelPosition": null, - "baseFontSize": null - }, - { - "id": "hermes-14", - "type": "rectangle", - "x": 770, - "y": 525, - "width": 410, - "height": 100, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "#e3f8e3", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": { - "type": 3 - }, - "seed": 1014, - "version": 2, - "versionNonce": 1887838612, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "index": "aE", - "created": null - }, - { - "id": "hermes-15", - "type": "text", - "x": 780, - "y": 533, - "width": 390, - "height": 84, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "transparent", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": null, - "seed": 1015, - "version": 2, - "versionNonce": 1536520620, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "text": "Gateway-side Hermes AIAgent\nNative conversation(s)", - "originalText": "Gateway-side Hermes AIAgent\nNative conversation(s)", - "fontSize": 23, - "fontFamily": 2, - "textAlign": "center", - "verticalAlign": "middle", - "containerId": null, - "autoResize": false, - "lineHeight": 1.25, - "index": "aF", - "created": null, - "labelPosition": null, - "baseFontSize": null - }, - { - "id": "hermes-16", - "type": "rectangle", - "x": 140, - "y": 80, - "width": 430, - "height": 65, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "white", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": { - "type": 3 - }, - "seed": 1016, - "version": 2, - "versionNonce": 1744097044, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "index": "aG", - "created": null - }, - { - "id": "hermes-17", - "type": "text", - "x": 150, - "y": 88, - "width": 410, - "height": 49, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "transparent", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": null, - "seed": 1017, - "version": 2, - "versionNonce": 1506958380, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "text": "Application / Fabric SDK", - "originalText": "Application / Fabric SDK", - "fontSize": 23, - "fontFamily": 2, - "textAlign": "center", - "verticalAlign": "middle", - "containerId": null, - "autoResize": false, - "lineHeight": 1.25, - "index": "aH", - "created": null, - "labelPosition": null, - "baseFontSize": null - }, - { - "id": "hermes-18", - "type": "arrow", - "x": 355, - "y": 145, - "width": 0, - "height": 240, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "transparent", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": null, - "seed": 1018, - "version": 2, - "versionNonce": 674352276, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "points": [ - [ - 0, - 0 - ], - [ - 0, - 240 - ] - ], - "lastCommittedPoint": null, - "startBinding": null, - "endBinding": null, - "startArrowhead": null, - "endArrowhead": "arrow", - "elbowed": false, - "index": "aI", - "created": null - }, - { - "id": "hermes-19", - "type": "text", - "x": 170, - "y": 221, - "width": 200, - "height": 35, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "transparent", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": null, - "seed": 1019, - "version": 70, - "versionNonce": 893971628, - "isDeleted": false, - "boundElements": [], - "updated": 1793913757174, - "link": null, - "locked": false, - "text": "Runtime.invoke()", - "originalText": "Runtime.invoke()", - "fontSize": 19, - "fontFamily": 2, - "textAlign": "center", - "verticalAlign": "middle", - "containerId": null, - "autoResize": false, - "lineHeight": 1.25, - "index": "aJ", - "created": null, - "labelPosition": null, - "baseFontSize": null - }, - { - "id": "hermes-20", - "type": "arrow", - "x": 355, - "y": 455, - "width": 0, - "height": 70, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "transparent", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": null, - "seed": 1020, - "version": 2, - "versionNonce": 586357268, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "points": [ - [ - 0, - 0 - ], - [ - 0, - 70 - ] - ], - "lastCommittedPoint": null, - "startBinding": null, - "endBinding": null, - "startArrowhead": null, - "endArrowhead": "arrow", - "elbowed": false, - "index": "aK", - "created": null - }, - { - "id": "hermes-21", - "type": "arrow", - "x": 600, - "y": 362, - "width": 130, - "height": 0, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "transparent", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": null, - "seed": 1021, - "version": 2, - "versionNonce": 217202988, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "points": [ - [ - 0, - 0 - ], - [ - 130, - 0 - ] - ], - "lastCommittedPoint": null, - "startBinding": null, - "endBinding": null, - "startArrowhead": null, - "endArrowhead": "arrow", - "elbowed": false, - "index": "aL", - "created": null - }, - { - "id": "hermes-22", - "type": "text", - "x": 598, - "y": 313.25, - "width": 135, - "height": 47.5, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "transparent", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": null, - "seed": 1022, - "version": 4, - "versionNonce": 1620828692, - "isDeleted": false, - "boundElements": [], - "updated": 1793913767364, - "link": null, - "locked": false, - "text": "owns\n child", - "originalText": "owns\n child", - "fontSize": 19, - "fontFamily": 2, - "textAlign": "center", - "verticalAlign": "middle", - "containerId": null, - "autoResize": false, - "lineHeight": 1.25, - "index": "aM", - "created": null, - "labelPosition": null, - "baseFontSize": null - }, - { - "id": "hermes-23", - "type": "arrow", - "x": 975, - "y": 455, - "width": 0, - "height": 70, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "transparent", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": null, - "seed": 1023, - "version": 2, - "versionNonce": 1978219436, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "points": [ - [ - 0, - 0 - ], - [ - 0, - 70 - ] - ], - "lastCommittedPoint": null, - "startBinding": null, - "endBinding": null, - "startArrowhead": null, - "endArrowhead": "arrow", - "elbowed": false, - "index": "aN", - "created": null - }, - { - "id": "hermes-24", - "type": "rectangle", - "x": 1380, - "y": 370, - "width": 310, - "height": 90, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "white", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": { - "type": 3 - }, - "seed": 1024, - "version": 2, - "versionNonce": 622503188, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "index": "aO", - "created": null - }, - { - "id": "hermes-25", - "type": "text", - "x": 1390, - "y": 378, - "width": 290, - "height": 74, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "transparent", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": null, - "seed": 1025, - "version": 2, - "versionNonce": 1260185132, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "text": "Native API client", - "originalText": "Native API client", - "fontSize": 23, - "fontFamily": 2, - "textAlign": "center", - "verticalAlign": "middle", - "containerId": null, - "autoResize": false, - "lineHeight": 1.25, - "index": "aP", - "created": null, - "labelPosition": null, - "baseFontSize": null - }, - { - "id": "hermes-26", - "type": "arrow", - "x": 1380, - "y": 415, - "width": 200, - "height": 0, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "transparent", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": null, - "seed": 1026, - "version": 2, - "versionNonce": 774693524, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "points": [ - [ - 0, - 0 - ], - [ - -200, - 0 - ] - ], - "lastCommittedPoint": null, - "startBinding": null, - "endBinding": null, - "startArrowhead": null, - "endArrowhead": "arrow", - "elbowed": false, - "index": "aQ", - "created": null - }, - { - "id": "hermes-27", - "type": "rectangle", - "x": 1380, - "y": 560, - "width": 310, - "height": 90, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "#f4f4f4", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "dashed", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": { - "type": 3 - }, - "seed": 1027, - "version": 2, - "versionNonce": 1098334380, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "index": "aR", - "created": null - }, - { - "id": "hermes-28", - "type": "text", - "x": 1390, - "y": 568, - "width": 290, - "height": 74, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "transparent", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": null, - "seed": 1028, - "version": 2, - "versionNonce": 622555156, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "text": "Stock Hermes TUI\nProposed terminal path", - "originalText": "Stock Hermes TUI\nProposed terminal path", - "fontSize": 21, - "fontFamily": 2, - "textAlign": "center", - "verticalAlign": "middle", - "containerId": null, - "autoResize": false, - "lineHeight": 1.25, - "index": "aS", - "created": null, - "labelPosition": null, - "baseFontSize": null - }, - { - "id": "hermes-29", - "type": "arrow", - "x": 1380, - "y": 605, - "width": 220, - "height": 150, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "transparent", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "dashed", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": null, - "seed": 1029, - "version": 2, - "versionNonce": 1621248812, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "points": [ - [ - 0, - 0 - ], - [ - -70, - 0 - ], - [ - -70, - -120 - ], - [ - -220, - -120 - ], - [ - -220, - -150 - ] - ], - "lastCommittedPoint": null, - "startBinding": null, - "endBinding": null, - "startArrowhead": null, - "endArrowhead": "arrow", - "elbowed": false, - "index": "aT", - "created": null - }, - { - "id": "hermes-30", - "type": "rectangle", - "x": 300, - "y": 765, - "width": 740, - "height": 110, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "#f4f4f4", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": { - "type": 3 - }, - "seed": 1030, - "version": 2, - "versionNonce": 702303636, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "index": "aU", - "created": null - }, - { - "id": "hermes-31", - "type": "text", - "x": 310, - "y": 773, - "width": 720, - "height": 94, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "transparent", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": null, - "seed": 1031, - "version": 2, - "versionNonce": 721946028, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "text": "Shared Fabric-managed Hermes home\nNative config / workspace / persisted state", - "originalText": "Shared Fabric-managed Hermes home\nNative config / workspace / persisted state", - "fontSize": 23, - "fontFamily": 2, - "textAlign": "center", - "verticalAlign": "middle", - "containerId": null, - "autoResize": false, - "lineHeight": 1.25, - "index": "aV", - "created": null, - "labelPosition": null, - "baseFontSize": null - }, - { - "id": "hermes-32", - "type": "arrow", - "x": 490, - "y": 625, - "width": 0, - "height": 140, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "transparent", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "dashed", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": null, - "seed": 1032, - "version": 2, - "versionNonce": 1839561492, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "points": [ - [ - 0, - 0 - ], - [ - 0, - 140 - ] - ], - "lastCommittedPoint": null, - "startBinding": null, - "endBinding": null, - "startArrowhead": null, - "endArrowhead": "arrow", - "elbowed": false, - "index": "aW", - "created": null - }, - { - "id": "hermes-33", - "type": "text", - "x": 326, - "y": 693, - "width": 160, - "height": 30, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "transparent", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": null, - "seed": 1033, - "version": 56, - "versionNonce": 58619436, - "isDeleted": false, - "boundElements": [], - "updated": 1793913760391, - "link": null, - "locked": false, - "text": "reads / writes", - "originalText": "reads / writes", - "fontSize": 19, - "fontFamily": 2, - "textAlign": "center", - "verticalAlign": "middle", - "containerId": null, - "autoResize": false, - "lineHeight": 1.25, - "index": "aX", - "created": null, - "labelPosition": null, - "baseFontSize": null - }, - { - "id": "hermes-34", - "type": "arrow", - "x": 890, - "y": 625, - "width": 0, - "height": 140, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "transparent", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "dashed", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": null, - "seed": 1034, - "version": 2, - "versionNonce": 183302292, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "points": [ - [ - 0, - 0 - ], - [ - 0, - 140 - ] - ], - "lastCommittedPoint": null, - "startBinding": null, - "endBinding": null, - "startArrowhead": null, - "endArrowhead": "arrow", - "elbowed": false, - "index": "aY", - "created": null - }, - { - "id": "hermes-35", - "type": "text", - "x": 725, - "y": 698, - "width": 160, - "height": 30, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "transparent", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": null, - "seed": 1035, - "version": 25, - "versionNonce": 1899032364, - "isDeleted": false, - "boundElements": [], - "updated": 1793913761974, - "link": null, - "locked": false, - "text": "reads / writes", - "originalText": "reads / writes", - "fontSize": 19, - "fontFamily": 2, - "textAlign": "center", - "verticalAlign": "middle", - "containerId": null, - "autoResize": false, - "lineHeight": 1.25, - "index": "aZ", - "created": null, - "labelPosition": null, - "baseFontSize": null - }, - { - "id": "hermes-36", - "type": "text", - "x": 170, - "y": 938, - "width": 1020, - "height": 40, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "transparent", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": null, - "seed": 1036, - "version": 2, - "versionNonce": 1567147540, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "text": "Same Hermes loop implementation; separate agent instances and conversations.", - "originalText": "Same Hermes loop implementation; separate agent instances and conversations.", - "fontSize": 22, - "fontFamily": 2, - "textAlign": "center", - "verticalAlign": "middle", - "containerId": null, - "autoResize": false, - "lineHeight": 1.25, - "index": "aa", - "created": null, - "labelPosition": null, - "baseFontSize": null - }, - { - "id": "hermes-37", - "type": "text", - "x": 170, - "y": 985.375, - "width": 1020, - "height": 26.25, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "transparent", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": null, - "seed": 1037, - "version": 11, - "versionNonce": 441814188, - "isDeleted": false, - "boundElements": [], - "updated": 1793913780979, - "link": null, - "locked": false, - "text": "Stopping the Fabric runtime stops both paths and their owned work.", - "originalText": "Stopping the Fabric runtime stops both paths and their owned work.", - "fontSize": 21, - "fontFamily": 2, - "textAlign": "center", - "verticalAlign": "middle", - "containerId": null, - "autoResize": false, - "lineHeight": 1.25, - "index": "ab", - "created": null, - "labelPosition": null, - "baseFontSize": null - }, - { - "id": "hermes-38", - "type": "rectangle", - "x": 580, - "y": 1090, - "width": 500, - "height": 65, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "white", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": { - "type": 3 - }, - "seed": 1038, - "version": 2, - "versionNonce": 1510925204, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "index": "ac", - "created": null - }, - { - "id": "hermes-39", - "type": "text", - "x": 590, - "y": 1098, - "width": 480, - "height": 49, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "transparent", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": null, - "seed": 1039, - "version": 2, - "versionNonce": 1030868908, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "text": "Configured model endpoint", - "originalText": "Configured model endpoint", - "fontSize": 23, - "fontFamily": 2, - "textAlign": "center", - "verticalAlign": "middle", - "containerId": null, - "autoResize": false, - "lineHeight": 1.25, - "index": "ad", - "created": null, - "labelPosition": null, - "baseFontSize": null - }, - { - "id": "hermes-40", - "type": "arrow", - "x": 200, - "y": 625, - "width": 450, - "height": 465, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "transparent", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": null, - "seed": 1040, - "version": 2, - "versionNonce": 1729450260, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "points": [ - [ - 0, - 0 - ], - [ - 0, - 425 - ], - [ - 450, - 425 - ], - [ - 450, - 465 - ] - ], - "lastCommittedPoint": null, - "startBinding": null, - "endBinding": null, - "startArrowhead": null, - "endArrowhead": "arrow", - "elbowed": false, - "index": "ae", - "created": null - }, - { - "id": "hermes-41", - "type": "arrow", - "x": 1135, - "y": 625, - "width": 125, - "height": 465, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "transparent", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": null, - "seed": 1041, - "version": 2, - "versionNonce": 1014168108, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "points": [ - [ - 0, - 0 - ], - [ - 0, - 425 - ], - [ - -125, - 425 - ], - [ - -125, - 465 - ] - ], - "lastCommittedPoint": null, - "startBinding": null, - "endBinding": null, - "startArrowhead": null, - "endArrowhead": "arrow", - "elbowed": false, - "index": "af", - "created": null - }, - { - "id": "hermes-42", - "type": "text", - "x": 1360, - "y": 695, - "width": 340, - "height": 100, - "angle": 0, - "strokeColor": "#3b424b", - "backgroundColor": "transparent", - "fillStyle": "solid", - "strokeWidth": 2, - "strokeStyle": "solid", - "roughness": 0, - "opacity": 100, - "groupIds": [], - "frameId": null, - "roundness": null, - "seed": 1042, - "version": 2, - "versionNonce": 871501460, - "isDeleted": false, - "boundElements": [], - "updated": 1793909002299, - "link": null, - "locked": false, - "text": "Native requests bypass\nFabric Runtime.invoke()", - "originalText": "Native requests bypass\nFabric Runtime.invoke()", - "fontSize": 20, - "fontFamily": 2, - "textAlign": "center", - "verticalAlign": "middle", - "containerId": null, - "autoResize": false, - "lineHeight": 1.25, - "index": "ag", - "created": null, - "labelPosition": null, - "baseFontSize": null - } - ], - "appState": { - "gridSize": 20, - "gridStep": 5, - "gridModeEnabled": false, - "viewBackgroundColor": "#ffffff", - "lockedMultiSelections": {} - }, - "files": {} -} \ No newline at end of file diff --git a/assets/hermes-runtime-invocation.png b/assets/hermes-runtime-invocation.png deleted file mode 100644 index 7b932b7a9..000000000 Binary files a/assets/hermes-runtime-invocation.png and /dev/null differ diff --git a/crates/fabric-cli/assets/adapters/hermes/hermes.fabric-adapter.json b/crates/fabric-cli/assets/adapters/hermes/hermes.fabric-adapter.json index 0ceeda9e3..458ce7fdf 100644 --- a/crates/fabric-cli/assets/adapters/hermes/hermes.fabric-adapter.json +++ b/crates/fabric-cli/assets/adapters/hermes/hermes.fabric-adapter.json @@ -19,7 +19,14 @@ }, "effort": { "type": "string", - "enum": ["none", "minimal", "low", "medium", "high", "xhigh"], + "enum": [ + "none", + "minimal", + "low", + "medium", + "high", + "xhigh" + ], "description": "Hermes model reasoning effort." } }, @@ -73,7 +80,9 @@ "description": "Exact loopback port for the API server. Start fails if it is in use." } }, - "required": ["port"], + "required": [ + "port" + ], "additionalProperties": false, "description": "Hermes's API server, run by a gateway child. After every enabled interface starts, the adapter writes its bearer token to /.fabric/hermes/runtimes//interfaces/api.token." }, @@ -92,9 +101,31 @@ "description": "Exact loopback port for the dashboard. Start fails if it is in use." } }, - "required": ["port"], + "required": [ + "port" + ], "additionalProperties": false, "description": "Hermes's dashboard, which has no sign-in when bound to loopback. It needs a built frontend at HERMES_WEB_DIST." + }, + "terminal": { + "type": "object", + "properties": { + "enabled": { + "type": "boolean", + "default": true + }, + "port": { + "type": "integer", + "minimum": 1, + "maximum": 65535, + "description": "Experimental TUI backend loopback port. Must match dashboard.port when both interfaces are enabled. This required field may be removed when the #579 launcher contract is adopted." + } + }, + "required": [ + "port" + ], + "additionalProperties": false, + "description": "Experimental terminal attachment to a managed TUI backend. Uses headless serve without dashboard assets, or shares the dashboard process. The configuration (including terminal.port), interfaces/tui.json, interfaces/tui.token, and the nemo_fabric_adapters.hermes.terminal helper may change across releases as the #579 launcher contract is adopted; compatibility is not guaranteed for these surfaces." } }, "additionalProperties": false, @@ -106,17 +137,105 @@ "maximum": 75, "default": 60, "description": "Time from the start of the runtime for every enabled interface to pass its health check, within NeMo Fabric's 90-second start limit." + }, + "sdk": { + "type": "object", + "properties": { + "enabled": { + "type": "boolean", + "default": true, + "description": "Initialize the embedded SDK agent during startup. Disable for native-only execution; SDK invocation then fails explicitly." + } + }, + "additionalProperties": false, + "description": "Embedded SDK execution, enabled by default. Runtime management remains available when disabled." } }, "required": [], - "additionalProperties": false + "additionalProperties": false, + "allOf": [ + { + "if": { + "required": [ + "sdk" + ], + "properties": { + "sdk": { + "required": [ + "enabled" + ], + "properties": { + "enabled": { + "const": false + } + } + } + } + }, + "then": { + "required": [ + "interfaces" + ], + "properties": { + "interfaces": { + "anyOf": [ + { + "required": [ + "api" + ], + "properties": { + "api": { + "properties": { + "enabled": { + "const": true + } + } + } + } + }, + { + "required": [ + "dashboard" + ], + "properties": { + "dashboard": { + "properties": { + "enabled": { + "const": true + } + } + } + } + }, + { + "required": [ + "terminal" + ], + "properties": { + "terminal": { + "properties": { + "enabled": { + "const": true + } + } + } + } + } + ] + } + } + } + } + ] }, "extension_schemas": { "run_error": { "$schema": "https://json-schema.org/draft/2020-12/schema", "type": "object", "properties": { - "provider": {"type": "string"} + "provider": { + "type": "string" + } }, "additionalProperties": false } @@ -136,7 +255,9 @@ "mcp.auth.oauth2", "skills" ], - "system_instruction_modes": ["replace"] + "system_instruction_modes": [ + "replace" + ] }, "telemetry": { "providers": { @@ -150,6 +271,12 @@ } }, "capabilities": { - "metadata": {"health_checks": ["live", "active", "ready"]} + "metadata": { + "health_checks": [ + "live", + "active", + "ready" + ] + } } } diff --git a/docs/index.yml b/docs/index.yml index 629a5ac6c..c3908e0da 100644 --- a/docs/index.yml +++ b/docs/index.yml @@ -75,9 +75,15 @@ navigation: - page: Factory Droid path: ./integrations/harness/droid.mdx - page: Hermes Agent - path: ./integrations/harness/hermes.mdx - - page: Hermes Native Interfaces - path: ./integrations/harness/hermes-interfaces.mdx + path: ./integrations/harness/hermes/hermes.mdx + - page: Hermes SDK + path: ./integrations/harness/hermes/hermes-sdk.mdx + - page: Hermes API + path: ./integrations/harness/hermes/hermes-api.mdx + - page: Hermes Dashboard + path: ./integrations/harness/hermes/hermes-dashboard.mdx + - page: Hermes Terminal TUI + path: ./integrations/harness/hermes/hermes-tui.mdx - page: LangChain Deep Agents path: ./integrations/harness/deepagents.mdx - page: Kilo Code diff --git a/docs/integrations/harness/hermes-interfaces.mdx b/docs/integrations/harness/hermes-interfaces.mdx deleted file mode 100644 index 595a9a7b6..000000000 --- a/docs/integrations/harness/hermes-interfaces.mdx +++ /dev/null @@ -1,169 +0,0 @@ ---- -title: "Hermes Native Interfaces" -description: "Run Hermes native interfaces alongside SDK invocations, with shared configuration and distinct conversations." ---- -{/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. -SPDX-License-Identifier: Apache-2.0 */} - -NVIDIA NeMo Fabric can start Hermes's native API server and dashboard alongside its embedded SDK agent. One NeMo Fabric runtime owns these execution paths, while each retains its own agent instances and conversations. Start with the [Hermes adapter guide](./hermes.mdx) for installation and normalized SDK configuration. - -## 1. Overview - -The following diagram shows the two invocation paths within one runtime. - -![One NeMo Fabric runtime owns an embedded Hermes agent and a managed Hermes Gateway. SDK calls use the embedded agent; native clients use gateway agents with separate conversations. The dashed TUI path is proposed.](../../../assets/hermes-runtime-invocation.png) - -The TUI path is proposed; the adapter currently exposes API and dashboard settings only. - -| Path | Execution | Conversation and result | -| --- | --- | --- | -| `Runtime.invoke()` or `Runtime.invoke_stream()` | The embedded Hermes `AIAgent` in the adapter host | Reuses the NeMo Fabric conversation and produces a normalized NeMo Fabric `RunResult`; for streaming, retrieve it through `stream.result()`. | -| Native API | Hermes creates gateway-side agent instances | Uses native conversations and returns native API responses. | -| Browser chat | Hermes's dashboard and native gateway facilities | Uses native conversation behavior and browser responses. | - -These agents run the same Hermes agent-loop implementation. Sharing a Hermes home gives them access to persisted resources; it does not combine their live state or attach a native conversation to NeMo Fabric's embedded conversation. - -### What NeMo Fabric Normalizes - -| Surface | SDK invocation | Native interfaces | -| --- | --- | --- | -| Configuration | Mapped from `FabricConfig` | Managed child processes receive the same behavioral mapping and generated native configuration. | -| Lifecycle | Managed by the runtime | The adapter starts and stops its child processes. Descendant cleanup and independently started gateways have limits described below. | -| Results | Normalized `RunResult`, including available usage and artifact references | Native responses are not converted into NeMo Fabric `RunResult` objects. | -| Relay evidence | Standard Relay artifacts and invocation telemetry references | Supported native events use Relay's standard formats and native session identities. Emitting those records does not create a normalized invocation result. | -| Session artifacts | Collected through the SDK invocation path | Native session storage is shared, but session-scoped collection into NeMo Fabric's artifact model remains separate work. | - -## 2. Configuration - -### Prerequisites - -Both interfaces are optional and require a POSIX host. The API server needs `aiohttp` from Hermes's `messaging` extra. The dashboard needs a built frontend at `HERMES_WEB_DIST` or `hermes_cli/web_dist`; browser chat also needs Hermes's built terminal UI bundle. The adapter does not build these frontends. Windows startup fails with `hermes_interfaces_unsupported_platform` before native setup. - -### Enable Native Interfaces - -Select interfaces through the Hermes adapter's `harness.settings`: - -```python -from nemo_fabric import HarnessConfig - -harness = HarnessConfig( - adapter_id="nvidia.fabric.hermes", - settings={ - "interfaces": { - "api": {"port": 8642}, - "dashboard": {"port": 9119}, - }, - "startup_timeout_seconds": 60, - }, -) -``` - -Use this harness selection with the rest of your `FabricConfig`. The model, instructions, tools, MCP, skills, workspace, and telemetry remain in their existing normalized configuration fields; do not duplicate them under `interfaces`. - -| Setting | Default | Behavior | -| --- | --- | --- | -| `interfaces.api` | Absent | API settings: required integer `port` from 1 through 65535, optional `enabled` boolean defaulting to `true`. | -| `interfaces.dashboard` | Absent | Dashboard settings with the same `port` and `enabled` requirements. | -| `startup_timeout_seconds` | `60` | Positive budget of at most 75 seconds, counted from the beginning of runtime startup. | - -An absent interface is off. A present interface is on unless `enabled` is `false`; its closed schema currently requires a port even when disabled. Choose distinct, available ports for enabled interfaces. Ports bind to loopback in the adapter environment. No terminal interface setting is implemented yet. - -## 3. Access and Use - -### API Access - -The API server binds to `127.0.0.1` at its configured port. After every enabled interface passes readiness, the adapter writes a per-start bearer token with mode `0600` to: - -```text -/.fabric/hermes/runtimes//interfaces/api.token -``` - -Clients calling the native API must read the generated token file and include `Authorization: Bearer ` in each HTTP request. Only the API gateway child receives it in its environment. Processes running as the runtime's user, including agent tools, can read the token file. - -### Dashboard Access - -Open the configured loopback port in a browser. The dashboard has no sign-in: Hermes embeds its session token in the page, so anyone who can reach that port can use its pages and browser chat. The adapter does not disable individual dashboard pages. - -### Conversations and Responses - -Ordered SDK invocations reuse the embedded agent's conversation. Native clients use Hermes's native session mechanisms and receive native responses; these do not pass through `Runtime.invoke()` or become NeMo Fabric `RunResult` objects. - -The runtime's Hermes home is stored under `/.fabric/hermes/runtimes/`. Sharing that home gives access to persisted resources, but does not select a shared live conversation or establish cross-runtime resume. - -## 4. What NeMo Fabric Manages - -### Configuration Mapping and Precedence - -NeMo Fabric resolves `FabricConfig` into a validated `AgentConfig`. The adapter uses it for the embedded agent and for agent instances created inside managed native processes. Instructions, enabled and blocked toolsets, turn limit, token limit, temperature, top-p, reasoning configuration, and trajectory settings use the same behavioral mapping. - -Instructions and behavioral values configured by NeMo Fabric take precedence over native constructor defaults. Explicit `tools.enabled` (including an empty list) overrides native tool selection; a nonempty `tools.blocked` overrides native exclusions. When these policies are unspecified, native interface tool defaults remain in effect; the embedded SDK path retains its existing CLI defaults. Set both policies explicitly when comparing tool parity. Native routing, conversation identity, execution, and protocol remain Hermes behavior. The model credential is supplied through the child's environment. For OpenAI without an explicit `base_url`, managed children discard inherited `OPENAI_BASE_URL` so it cannot redirect the configured credential. An explicit endpoint uses the generated native model routing. Refer to [Implementation Details](#7-implementation-details) for the generated files and launcher mechanism. - -### Startup, Readiness, and Shutdown - -Startup launches enabled interfaces as child processes on the adapter's interpreter. The API server must pass `GET /health`, and the dashboard must pass `GET /api/status`, within the remaining startup budget. An occupied port, early child exit, or readiness timeout fails startup. Failed startup attempts clean up the children they started. - -Stop removes the API token, sends SIGTERM to running interface process groups, and escalates to SIGKILL after a five-second grace. The API gateway is configured with a three-second drain for API work and cron jobs. The agent configuration snapshot is removed during cleanup, and the adapter closes the embedded agent and session database. If token or snapshot deletion fails, the adapter logs the error and continues stopping children; a later stop retries removal. - -### Relay Evidence and Artifact Boundaries - -Enable Relay through the normal NeMo Fabric telemetry configuration. The embedded and managed native paths receive the generated Relay plugin configuration. Native sessions retain their own trace roots rather than being assigned to the embedded NeMo Fabric invocation. - -For SDK invocations, the adapter finalizes the Hermes Relay session and flushes subscribers before returning, so available ATOF/ATIF artifacts are complete at the invocation boundary. Managed native API turns also finalize and flush a Relay segment when execution returns. This makes ATIF available per turn without deleting the native conversation; subsequent turns can continue it and produce separate segments. Export failures are logged, so consumers must verify artifact contents. With a live Relay runtime, Hermes emits `compaction` when tracked compaction rewrites history or provider state; `compaction.attempt` describes an attempt that is blocked, skipped, fails, or aborts without a rewrite. - -Native API responses remain native responses. Relay's standard artifact format provides common execution evidence, but does not by itself provide NeMo Fabric result normalization, complete usage accounting, or session-scoped collection of native transcripts and generated files. - -## 5. Testing - -See the [Hermes interfaces example](../../../examples/interfaces/hermes/README.md) for setup, manual testing, output verification, and Relay trajectory inspection. - -## 6. Limitations and Planned Work - -### Operational Limits - -- Denied process-group signals fall back to the direct child. This does not guarantee that all descendants have stopped. -- The adapter does not restart an interface child after a crash. Dashboard gateway actions can start processes outside the adapter launcher and its configuration mapping. -- Filesystem permission errors can leave the token or configuration snapshot on disk after stop. Child shutdown continues, and removal is retried on a later stop; inspect cleanup errors and remove retained files after correcting permissions. -- Readiness currently checks the HTTP response without verifying that the child owns the listening port. -- NeMo Fabric's `check()` reports embedded SDK agent health and activity only. It does not aggregate native interface health or active native turns. Startup probes check API `/health` and dashboard `/api/status`; the adapter does not monitor child health continuously. -- There is no unified inventory of all agent instances in a runtime. SDK invocation tracking covers SDK requests; native API run status and persisted sessions are exposed through Hermes endpoints. Persisted sessions do not identify all live agent objects, which can exist only for a turn. -- Child processes inherit the embedded adapter's `HERMES_YOLO_MODE` environment setting. Interactive approval behavior requires separate review and validation. -- Native responses and complete usage accounting are not normalized into NeMo Fabric invocation results by these interfaces. - -### Unqualified Paths - -The API configuration-parity tests do not qualify browser interaction, explicit resume, or gateway replacement. Each needs its own behavioral and lifecycle evidence. - -The no-URL OpenAI regression checks child environment isolation with a mocked embedded constructor, not provider authentication or a live request to OpenAI. Hosted-model tests use an explicit OpenAI-compatible endpoint; provider login and fallback routing without an explicit endpoint require separate qualification. - -### Planned Work - -Native TUI support remains proposed and is shown as a dashed path in the diagram. Session-scoped collection of native transcripts and generated files into NeMo Fabric's artifact model remains separate work; standard Relay formatting alone does not provide it. - -Add runtime-wide health and discovery that covers the embedded agent and managed native interfaces. Report process health and readiness separately from active runs and persisted sessions, include child failures and native activity in aggregate status, and provide a way to discover native runs and sessions alongside SDK invocations. Hermes's authenticated `/health/detailed`, `/v1/runs/{run_id}`, and `/api/sessions` endpoints provide native diagnostics today; they do not provide a unified NeMo Fabric inventory. - -Follow-up lifecycle work should verify listener ownership during readiness, strengthen descendant cleanup when process-group signals are denied, and define ownership and configuration guarantees for gateway replacement. Qualify interactive approvals and browser chat separately before extending their support claims. Cross-runtime resume and SDK/native conversation handoff remain deferred. - -## 7. Implementation Details - -### Generated Files and Environment - -The adapter prepares three configuration surfaces from the resolved `AgentConfig` for managed native children: - -| Surface | Contents | Delivery | -| --- | --- | --- | -| Hermes `config.yaml` | Native model routing, workspace, MCP servers, skills, plugins, and toolset configuration | Written in the runtime's Hermes home, selected through `HERMES_HOME`. | -| `interfaces/agent-config.json` | The validated agent configuration, including instructions and behavioral settings | Owner-only snapshot passed by path to the adapter-owned native launcher. Removed on stop. | -| Relay plugin configuration | Resolved Relay components and output configuration | Generated by the adapter; its path is passed through `HERMES_NEMO_RELAY_PLUGINS_TOML`. | - -The model credential value is supplied through the child's environment; the native model configuration references `api_key_env`. - -Children inherit the adapter host's environment after ambient `API_SERVER_*` variables are removed. When the API interface is enabled, the generated configuration explicitly disables other known gateway platforms and profile multiplexing. Inherited settings such as `TELEGRAM_BOT_TOKEN` or `WEBHOOK_ENABLED` therefore do not start messaging adapters or their dependency installers. This restriction applies to the managed runtime home; independently launched gateways outside that home retain their own configuration. - -### Native Launcher and Constructor Mapping - -Hermes v0.22.5's API constructor does not read every behavioral setting from `config.yaml`. The adapter therefore launches managed children through `nemo_fabric_adapters.hermes.native`, which installs a process-local subclass of Hermes's `AIAgent` before entering the native CLI. - -The subclass applies the shared behavioral mapping before calling the native constructor. It preserves native routing and session identity and does not modify the installed Hermes source. - -This mapping applies within adapter-launched processes. A gateway independently spawned by Hermes does not automatically inherit the constructor mapping, even if it uses the same home. Sharing a configuration directory alone does not establish configuration parity. - diff --git a/docs/integrations/harness/hermes.mdx b/docs/integrations/harness/hermes.mdx deleted file mode 100644 index db5f28e11..000000000 --- a/docs/integrations/harness/hermes.mdx +++ /dev/null @@ -1,101 +0,0 @@ ---- -title: "Use the Hermes Agent Adapter" -description: "Configure Hermes Agent, understand its invocation paths, and serve native interfaces with NeMo Fabric." ---- -{/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. -SPDX-License-Identifier: Apache-2.0 */} - -The NVIDIA NeMo Fabric `nvidia.fabric.hermes` adapter runs Hermes Agent through its Python SDK. It maps normalized configuration into an embedded agent, reuses its conversation across ordered invocations, and returns normalized results and telemetry references. - -For native API and browser access, invocation-path diagrams, and shared configuration behavior, see [Hermes native interfaces](./hermes-interfaces.mdx). - -## Install the Adapter - -Use [Hermes Agent v0.22.5](https://github.com/NousResearch/hermes-agent/releases/tag/v0.22.5) with Python 3.14. The adapter package supports Python 3.11 through 3.14, but Hermes Agent must be installed separately from source; versions 0.20 and later are not available from PyPI. Follow the [Hermes Agent installation guide](https://hermes-agent.nousresearch.com/docs/installation) and select v0.22.5. - -Install the runtime and adapter in the environment that runs Hermes: - -```bash -python -m pip install "nemo-fabric[hermes-agent]" -``` - -If the application runs NeMo Fabric in a separate environment, install `nemo-fabric-adapters-hermes` in the Hermes environment and set `ADAPTER_PYTHON` in the application's environment to that interpreter. Use matching runtime and adapter release versions unless a different pairing has been validated. - -For Relay, install the adapter's `relay` extra in the Hermes environment. It requires `nemo-relay>=0.10,<0.11`; keep this environment separate from applications using Relay 0.11 for other adapters. Startup rejects unsupported Hermes/Relay combinations. - -For repository development, the following recipe checks out v0.22.5 at `bf0203a794a07aac3ff764972ab9087baa4e4f4e` and prepares an isolated Python 3.14 environment with Relay support: - -```bash -just install-hermes-agent -export ADAPTER_PYTHON="$PWD/.venv-hermes/bin/python" -``` - -The installer preserves local checkout changes and stops rather than overwriting them. Set `HERMES_AGENT_CHECKOUT` to a separate clean checkout or a new directory to install without changing `external/hermes-agent`; the target environment remains `.venv-hermes`. - -Hermes reports its Python package version as `0.0.0`, so use its Git tag and commit to identify the installed release. The pinned release includes dependency security fixes absent from v2026.9.24. - -## Configure the Adapter - -Use normalized `FabricConfig` fields for the model, instructions, tools, skills, MCP servers, workspace, turn limit, and telemetry. Use `harness.settings` for Hermes-specific options and native interface selection. - -The following example configures an embedded SDK agent with file tools: - -```python -from nemo_fabric import FabricConfig - -config = FabricConfig.from_mapping( - { - "metadata": {"name": "hermes-agent"}, - "harness": { - "adapter_id": "nvidia.fabric.hermes", - }, - "models": { - "default": { - "provider": "openai", - "model": "your-model", - "base_url": "https://MODEL_HOST/v1", - "api_key_env": "MODEL_API_KEY", - "temperature": 0.0, - "top_p": 0.9, - "max_tokens": 1024, - } - }, - "instructions": { - "system": {"mode": "replace", "content": "Help with files in the workspace."} - }, - "tools": {"enabled": ["file"], "blocked": ["terminal"]}, - "runtime": {"max_turns": 8}, - "environment": {"provider": "local", "workspace": "."}, - } -) -``` - -Replace the endpoint and model, and provide `MODEL_API_KEY` in the adapter environment before starting. To add native API or browser access, follow the [Hermes native interfaces guide](./hermes-interfaces.mdx). - -The adapter supports `instructions.system.mode: replace` and rejects `append`. Tool selectors are Hermes toolset names. `models..top_p` must be between 0 and 1, and `max_tokens` must be positive. A model-level token limit takes precedence over `harness.settings.max_tokens`. If both are omitted, the adapter passes `None` to preserve Hermes' native response token limit behavior for embedded and native interface agents. Use `runtime.max_turns`, not a `max_iterations` harness setting, to control the agent loop. - -### Hermes Settings - -The descriptor uses closed schemas, so unknown settings are rejected. - -| Setting | Default | Behavior | -| --- | --- | --- | -| `reasoning_config` | `{"effort": "none"}` | Optional `enabled` boolean and `effort`: `none`, `minimal`, `low`, `medium`, `high`, or `xhigh`. | -| `plugins_enabled` | `[]` | List of nonempty native plugin identifiers. Relay uses its generated plugin configuration; do not list the removed `observability/nemo_relay` plugin. | -| `save_trajectories` | `false` | Save native JSONL trajectories; separate from normalized telemetry. | -| `max_tokens` | Hermes native (`None`) | Positive response-token limit when the selected model has no limit. | -| `terminal_timeout` | `60` | Positive timeout in seconds for a Hermes terminal tool operation; this does not enable a TUI. | - -Native interface settings are documented in the [Hermes native interfaces guide](./hermes-interfaces.mdx#enable-native-interfaces). - -## Understand the Runtime Lifecycle - -Each runtime creates one embedded Hermes `AIAgent` and one `SessionDB`. Ordered SDK invocations reuse these objects and their conversation history until stop closes them. The Hermes home is stored under `/.fabric/hermes/runtimes/`; retaining files does not by itself establish cross-runtime conversation resume. - -SDK invocations return normalized NeMo Fabric `RunResult` objects. The adapter finalizes the Hermes Relay session and flushes subscribers before returning, so available ATOF/ATIF artifacts are complete at that invocation boundary. Native interface responses follow a different path, described in [What NeMo Fabric Normalizes](./hermes-interfaces.mdx#what-nemo-fabric-normalizes). - -## Runtime Health - -`Fabric.check()` supports `live`, `active`, and `ready`. It checks the embedded agent's session/database ownership, invocation interface, and a fresh `get_activity_summary()` diagnostic response. These checks do not establish native interface-child health after startup. - -Refer to the [SDK health-check overview](../../sdk/health-checks.mdx) and the [Hermes adapter reference](https://github.com/NVIDIA/NeMo-Fabric/tree/main/adapters/python/hermes) for additional details. diff --git a/docs/integrations/harness/hermes/hermes-api.mdx b/docs/integrations/harness/hermes/hermes-api.mdx new file mode 100644 index 000000000..a0aef2b37 --- /dev/null +++ b/docs/integrations/harness/hermes/hermes-api.mdx @@ -0,0 +1,76 @@ +--- +title: "Hermes API" +description: "Run the managed native HTTP gateway with independent Hermes conversations." +--- +{/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. +SPDX-License-Identifier: Apache-2.0 */} + +Start with [hermes.mdx](./hermes.mdx) for shared setup and an overview of all four invocation paths. + +NVIDIA NeMo Fabric can host Hermes's native HTTP API as a managed `hermes gateway run` process. Requests execute through native agents, independently of the embedded SDK and TUI backend. The HTTP service uses `aiohttp`; it is separate from the FastAPI dashboard service. + +## Installation + +Complete the [shared installation](./hermes.mdx#install-the-adapter) first. The API path additionally requires `aiohttp` in the Hermes adapter environment. For Hermes v0.22.5, the pinned version is `3.14.5`. + +If you used `just install-hermes-agent`, this dependency is already installed. For a separately prepared Hermes environment, install it with `ADAPTER_PYTHON` set to that environment's interpreter: + +```bash +uv pip install --python "$ADAPTER_PYTHON" "aiohttp==3.14.5" +``` + +Verify the dependency in the same environment: + +```bash +"$ADAPTER_PYTHON" -c "import aiohttp; print(aiohttp.__version__)" +``` + +The API path requires a POSIX host. It needs no browser assets, terminal frontend, or Node runtime. + +## Configuration + +Combine the normalized model/workspace configuration from the [SDK guide](./hermes-sdk.mdx#configure-the-adapter) with these harness settings: + +```python +settings = { + "sdk": {"enabled": False}, + "interfaces": {"api": {"port": 8642}}, +} +``` + +This disables embedded SDK execution explicitly. Omit `sdk` to also start the embedded Hermes `AIAgent` alongside the native API process. The API is off unless selected; selecting it does not start a dashboard or terminal backend. Model, instructions, tools, and Relay remain in normalized configuration fields. + +## Authentication + +The API server binds to `127.0.0.1` at its configured port. After every enabled interface passes readiness, the adapter writes a per-start bearer token with mode `0600` to: + +```text +/.fabric/hermes/runtimes//interfaces/api.token +``` + +Clients calling the native API must read the generated token file and include `Authorization: Bearer ` in each HTTP request. Only the API gateway child receives it in its environment. Processes running as the runtime's user, including agent tools, can read the token file. + +This token authenticates access to the API gateway. It is distinct from the model-provider API key and the TUI backend credential used by dashboard and terminal clients. API clients share the per-start token; it does not provide per-user isolation. + +## Requests, Clients, and Conversations + +The pinned native service includes OpenAI-compatible chat completions and responses, native run/session endpoints, and health diagnostics. The managed chat-completions path has end-to-end evidence; other native routes need separate qualification before relying on equivalent behavior. + +Multiple clients can connect. Use native session/conversation mechanisms for separate histories; clients share the runtime's workspace, tools, home, and per-start bearer credential. Concurrent workload qualification is separate from accepting multiple connections. + +Native requests do not pass through `Runtime.invoke()` and do not return NeMo Fabric `RunResult` objects. Shared configuration does not attach them to the embedded SDK conversation. + +## Relay Trajectories + +With Relay enabled, managed API turns finalize and flush a segment when native execution returns. ATIF is available per turn without deleting native history; subsequent turns can continue the conversation and produce new segments. Check actual ATOF events and ATIF tool steps, since export failures are logged and do not replace the response. + +## Health, Testing, and Limitations + +- **Native health endpoint:** `/health` belongs to Hermes's API server. Calling it directly checks that native service; it does not return a normalized NeMo Fabric health report or check the embedded SDK agent and TUI backend. +- **Startup readiness:** The adapter polls the API server's `/health` endpoint and waits for HTTP 200 before declaring that interface ready. Startup fails if the process exits or readiness is not established within the startup budget. +- **Normalized health with the SDK enabled:** `Fabric.check()` observes the embedded SDK agent and preserves its `idle`/`busy` activity. A failed API child does not fail that SDK health check. Probe the API `/health` endpoint or inspect its interface log separately to diagnose the API service. +- **Normalized health with the SDK disabled:** `Fabric.check()` observes the enabled native backends. For the API backend, `live` checks the owned process; `active` and `ready` also probe `/health`. An enabled TUI backend is checked too. Activity is `unknown` because native turns are not fully aggregated. + +Neither the native endpoint nor the normalized checks execute a model/tool turn. Hermes also exposes authenticated `/health/detailed`, run status, and persisted sessions for native diagnostics; these do not provide a unified inventory of live agents in NeMo Fabric. + +See the [API example](../../../../examples/interfaces/hermes/README.md#2-http-api) for requests and trajectory inspection, and the [shared limitations](./hermes.mdx#limitations-and-planned-work) for lifecycle and artifact boundaries. diff --git a/docs/integrations/harness/hermes/hermes-dashboard.mdx b/docs/integrations/harness/hermes/hermes-dashboard.mdx new file mode 100644 index 000000000..0de2e943f --- /dev/null +++ b/docs/integrations/harness/hermes/hermes-dashboard.mdx @@ -0,0 +1,55 @@ +--- +title: "Hermes Dashboard and Browser TUI" +description: "Use browser chat through a managed TUI service, optionally shared with terminal clients." +--- +{/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. +SPDX-License-Identifier: Apache-2.0 */} + +Start with [hermes.mdx](./hermes.mdx) for shared setup and an overview of all four invocation paths. + +NVIDIA NeMo Fabric starts Hermes's dashboard as a managed FastAPI/Uvicorn service. Browser chat uses a prebuilt terminal renderer through a PTY/WebSocket bridge, attached to that service's in-process TUI backend. It does not call the native API gateway or embedded SDK agent. + +## Installation + +Complete the [shared installation](./hermes.mdx#install-the-adapter) first. The dashboard path additionally requires: + +- **Backend dependencies:** FastAPI, Uvicorn, and the other dependencies in pinned Hermes's `web` extra, plus `aiohttp` for authenticated readiness checks. The repository's Python 3.14 `just install-hermes-agent` recipe already provides them. +- **Frontend assets:** Both the built browser assets and terminal renderer used by browser chat, plus a supported Node runtime. +- **Host:** POSIX, for the managed backend and browser terminal bridge. + +For a separately prepared Hermes environment, set `ADAPTER_PYTHON` to its interpreter and `HERMES_AGENT_CHECKOUT` to your pinned v0.22.5 source checkout, then install the web extra and the pinned readiness dependency: + +```bash +uv pip install --python "$ADAPTER_PYTHON" --editable "${HERMES_AGENT_CHECKOUT}[web]" "aiohttp==3.14.5" +``` + +Keep this `ADAPTER_PYTHON` value when following the [frontend setup commands](../../../../examples/interfaces/hermes/README.md#build-the-tui-frontends) to build both frontends in that pinned checkout. Browser assets must be available at `hermes_cli/web_dist/index.html` or the directory selected by `HERMES_WEB_DIST`. Runtime startup uses these prebuilt assets; it does not install or build them. + +## Configuration + +Use normalized model, workspace, tools, and telemetry fields with these harness settings: + +```python +settings = { + "sdk": {"enabled": False}, + "interfaces": {"dashboard": {"port": 9119}}, +} +``` + +Open `http://127.0.0.1:9119`. The native API is optional. To attach terminal clients too, add `"terminal": {"port": 9119}`; both client paths share one backend process and start separate conversations by default. Omit `sdk` to also start the embedded Hermes `AIAgent` alongside the dashboard process. + +## Authentication + +The adapter generates a per-start credential for the TUI backend. Hermes embeds that credential in the served page, and the browser uses it automatically. The dashboard has no separate user sign-in on loopback: anyone reaching the port can use its pages and browser chat. Limit access to the runtime user; port forwarding must have its own access controls. + +The TUI credential is distinct from the model-provider API key and the API gateway's `api.token`. When terminal attachment is also enabled, the same TUI credential is published in `interfaces/tui.token` for the attachment helper. This does not add a browser sign-in gate or per-user isolation. + +## Ownership + +The adapter pins browser chat to the managed runtime profile and its existing TUI service. Named-profile chat is unsupported. Other dashboard pages remain native; gateway replacement, profile/configuration edits, and restart/migrate actions are outside the qualified ownership model and can bypass generated configuration. + +## Conversations, Relay, and Testing + +Browser turns use native conversations and responses. They do not become SDK invocations or normalized `RunResult` objects. Managed native agent turns finalize and flush Relay segments when execution returns, preserving native history. + +See the [TUI example](../../../../examples/interfaces/hermes/README.md#3-browser-dashboard) for browser input and artifact inspection. Refer to the [shared limitations](./hermes.mdx#limitations-and-planned-work) for approvals, health/activity, and lifecycle boundaries. diff --git a/docs/integrations/harness/hermes/hermes-sdk.mdx b/docs/integrations/harness/hermes/hermes-sdk.mdx new file mode 100644 index 000000000..02c99339e --- /dev/null +++ b/docs/integrations/harness/hermes/hermes-sdk.mdx @@ -0,0 +1,109 @@ +--- +title: "Hermes SDK" +description: "Invoke the embedded Hermes agent with normalized configuration and results." +--- +{/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. +SPDX-License-Identifier: Apache-2.0 */} + +Start with [hermes.mdx](./hermes.mdx) for shared setup and an overview of all four invocation paths. + +NVIDIA NeMo Fabric's Hermes SDK path executes requests through an embedded Hermes `AIAgent` in the adapter host. It returns normalized results and reuses conversation history across ordered invocations. + +## Installation + +Complete the [shared installation](./hermes.mdx#install-the-adapter). The SDK path requires no interface-specific packages or frontend builds. Install any additional dependencies required by your selected tools and MCP servers in the Hermes adapter environment. + +## Configure the Adapter + +Use normalized `FabricConfig` fields for the model, instructions, tools, skills, MCP servers, workspace, turn limit, and telemetry. Use `harness.settings` for Hermes-specific options and native interface selection. + +The following example configures an embedded SDK agent with file tools: + +```python +from nemo_fabric import FabricConfig + +config = FabricConfig.from_mapping( + { + "metadata": {"name": "hermes-agent"}, + "harness": { + "adapter_id": "nvidia.fabric.hermes", + }, + "models": { + "default": { + "provider": "openai", + "model": "your-model", + "base_url": "https://MODEL_HOST/v1", + "api_key_env": "MODEL_API_KEY", + "temperature": 0.0, + "top_p": 0.9, + "max_tokens": 1024, + } + }, + "instructions": { + "system": {"mode": "replace", "content": "Help with files in the workspace."} + }, + "tools": {"enabled": ["file"], "blocked": ["terminal"]}, + "runtime": {"max_turns": 8}, + "environment": {"provider": "local", "workspace": "."}, + } +) +``` + +Replace the endpoint and model, and provide `MODEL_API_KEY` in the adapter environment before starting. To add native API or browser access, follow the [Hermes invocation paths](./hermes.mdx). + +The following rules apply to the normalized configuration: + +- **System instructions:** Only `instructions.system.mode: replace` is supported. +- **Tools:** Selectors are Hermes toolset names. +- **Turn limit:** Use `runtime.max_turns` to control the agent loop. + +### Hermes Settings + +The descriptor uses closed schemas, so unknown settings are rejected. + +| Setting | Default | Behavior | +| --- | --- | --- | +| `sdk.enabled` | `true` | Initialize the embedded SDK agent. Set to `false` for native-only execution; enable at least one native interface. SDK invocation then fails with `hermes_sdk_disabled`, while runtime management remains available. | +| `reasoning_config` | `{"effort": "none"}` | Optional `enabled` boolean and `effort`: `none`, `minimal`, `low`, `medium`, `high`, or `xhigh`. | +| `plugins_enabled` | `[]` | List of native Hermes plugin identifiers to enable. Configure NeMo Relay through NeMo Fabric telemetry settings. | +| `save_trajectories` | `false` | Save native JSONL trajectories; separate from normalized telemetry. | +| `max_tokens` | Hermes native default | Response-token limit used when the selected model has no explicit `max_tokens`. A model-level limit takes precedence; if both are omitted, Hermes uses its native default. | +| `terminal_timeout` | `60` | Positive timeout in seconds for a Hermes terminal tool operation; this does not enable a TUI. | + +Use the [API](./hermes-api.mdx), [dashboard](./hermes-dashboard.mdx), or [terminal TUI](./hermes-tui.mdx) guide for native interface selection. + +## Invoke and Stream + +Start the runtime with the configured Hermes interpreter. With `config` from the preceding example, invoke it through the public SDK: + +```python +from nemo_fabric import Fabric + +async with await Fabric().start_runtime(config) as runtime: + result = await runtime.invoke(input="List the files in the workspace.") + print(result.output) +``` + +`Runtime.invoke_stream()` uses Relay-backed event streaming; retrieve its normalized result with `stream.result()`. Refer to [streaming](../../../sdk/python.mdx#nemo-relay-streaming) for the SDK usage pattern. + +## Lifecycle and Conversations + +The embedded SDK agent is enabled by default and initializes during runtime startup, together with its parent session database and SDK-owned MCP connections. SDK initialization errors remain startup errors. + +Ordered SDK invocations reuse the same agent and conversation until stop closes them. API, browser, and terminal conversations do not execute through this agent or inherit its live history. Persisted files do not establish cross-runtime resume. + +Set `harness.settings.sdk.enabled` to `false` only when at least one native interface is enabled. Disabling the embedded Hermes agent does not affect runtime management: your application can still start, check, and stop the native interfaces through the NeMo Fabric SDK. SDK invocation returns a failed `RunResult` with the stable error code `hermes_sdk_disabled` and `retryable: false`. This is a configuration mismatch: enable the SDK agent before invoking through the SDK. The native interfaces and their health checks remain available. + +Evaluation consumers, including Harbor, must keep `sdk.enabled: true` (or omit `sdk`). A native-only configuration is for hosting interfaces; it cannot run SDK evaluation trials. Current adapter metadata and `RunPlan` do not expose this invocation restriction before startup, so metadata-based admission can accept a configuration whose trials will fail. Exposing this restriction before invocation is planned work. + +## Results and Relay + +SDK invocations return normalized NeMo Fabric `RunResult` objects. With Relay enabled, the adapter finalizes and flushes available ATOF/ATIF evidence before returning the result. Native interface responses remain native; see the [normalization comparison](./hermes.mdx#what-nemo-fabric-normalizes). + +## Health and Limitations + +Whenever the SDK agent is enabled, `check()` supports `live`, `active`, and `ready` using SDK session/database ownership, the invocation interface, and fresh `get_activity_summary()` diagnostics. This includes mixed runtimes: SDK activity remains `idle` or `busy`, and a separate API or TUI process failure does not fail SDK runtime health. SDK activity does not describe native conversations. + +With `sdk.enabled: false`, checks observe the enabled native processes and endpoints, with activity reported as `unknown`. Check native endpoints or interface logs separately to diagnose a native interface in a mixed runtime; a normalized interface health target is planned under [#566](https://github.com/NVIDIA/NeMo-Fabric/issues/566). Checks do not run an agent turn. + +Refer to [SDK health checks](../../../sdk/health-checks.mdx) and the [shared limitations](./hermes.mdx#limitations-and-planned-work). diff --git a/docs/integrations/harness/hermes/hermes-tui.mdx b/docs/integrations/harness/hermes/hermes-tui.mdx new file mode 100644 index 000000000..43c95c644 --- /dev/null +++ b/docs/integrations/harness/hermes/hermes-tui.mdx @@ -0,0 +1,63 @@ +--- +title: "Hermes Terminal TUI" +description: "Attach the native terminal renderer to a managed headless or dashboard TUI service." +--- +{/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. +SPDX-License-Identifier: Apache-2.0 */} + +Start with [hermes.mdx](./hermes.mdx) for shared setup and an overview of all four invocation paths. + +NVIDIA NeMo Fabric's terminal path attaches a prebuilt Hermes terminal renderer to a runtime-owned TUI service. This fourth client path shares the dashboard's execution backend; it does not create a fourth backend or invoke the embedded SDK agent. + +**Experimental terminal interface:** The `interfaces.terminal` configuration, including `terminal.port`, the `tui.json` and `tui.token` attachment files, and the `nemo_fabric_adapters.hermes.terminal` helper may change as NeMo Fabric adopts the [launcher contract in #579](https://github.com/NVIDIA/NeMo-Fabric/issues/579). Compatibility across releases is not guaranteed for these surfaces. + +## Installation + +Complete the [shared installation](./hermes.mdx#install-the-adapter) first. Terminal attachment additionally requires: + +- **Backend dependencies:** FastAPI, Uvicorn, and the other dependencies in pinned Hermes's `web` extra, plus `aiohttp` for authenticated readiness checks. The repository's Python 3.14 `just install-hermes-agent` recipe already provides them. +- **Client assets:** The built terminal renderer and a supported Node runtime on the machine running the client. Terminal-only mode needs no browser assets. +- **Host:** POSIX, for the managed backend and terminal attachment helper. + +For a separately prepared Hermes environment, set `ADAPTER_PYTHON` to its interpreter and `HERMES_AGENT_CHECKOUT` to your pinned v0.22.5 source checkout, then install the web extra and the pinned readiness dependency: + +```bash +uv pip install --python "$ADAPTER_PYTHON" --editable "${HERMES_AGENT_CHECKOUT}[web]" "aiohttp==3.14.5" +``` + +Keep this `ADAPTER_PYTHON` value when following the [frontend setup commands](../../../../examples/interfaces/hermes/README.md#build-the-tui-frontends) to build the terminal bundle. For terminal-only use, skip the browser build and `HERMES_WEB_DIST` setting. If the dashboard backend is already enabled, reuse it; terminal attachment does not require another backend installation. + +The helper uses `HERMES_TUI_DIR/dist/entry.js` when configured; otherwise it looks for `hermes_cli/tui_dist/entry.js` or the pinned checkout's `ui-tui/dist/entry.js`. `HERMES_NODE` can select an executable Node runtime; otherwise it uses Node from `PATH`. It does not build or install missing assets. + +## Configuration + +Combine normalized model/workspace/Relay configuration with these harness settings: + +```python +settings = { + "sdk": {"enabled": False}, + "interfaces": {"terminal": {"port": 9119}}, +} +``` + +Terminal-only mode starts `hermes serve --isolated`, which runs the backend without serving the browser dashboard. If dashboard is also enabled, give it the same port: the terminal attaches to that existing service instead. API, if enabled, needs a distinct port. SDK remains enabled by default unless explicitly disabled. + +## Authentication + +After readiness, the adapter publishes owner-only attachment files under `/.fabric/hermes/runtimes//interfaces/`: `tui.json` describes the port/mode/workspace, and `tui.token` contains the TUI backend credential. + +The attachment helper reads these files, verifies backend authentication, and passes an authenticated WebSocket URL through the environment. The native terminal renderer presents the credential when connecting. The token is not placed in command arguments or entered manually. + +This credential is shared with the browser dashboard when both use the same TUI backend. It is distinct from the API gateway's `api.token` and the model-provider API key. Terminal clients share the credential; separate conversations do not provide per-user isolation. + +## Attach and Detach + +Run the helper from an interactive terminal with the Hermes interpreter. The [runnable example](../../../../examples/interfaces/hermes/README.md#4-terminal-tui) prints the exact attachment command using that runtime's descriptor. The helper executes the prebuilt renderer against the managed backend and never falls back to spawning an unmanaged backend. + +Terminal clients start independent native conversations by default. Detaching leaves the backend/runtime running; it is not an instruction to cancel a turn. The runtime owner stops its backend and removes attachment files. Same-conversation multi-client control and detach during active work require explicit qualification; use the runtime owner's stop action for final cleanup. + +## Results, Relay, and Testing + +Terminal output is native; it is not a normalized NeMo Fabric `RunResult`. With Relay enabled, managed agent turns finalize/flush native segments without deleting conversation history. Verify populated ATIF tool steps after each completed turn without detaching or stopping the runtime. + +Use the [TUI example](../../../../examples/interfaces/hermes/README.md#4-terminal-tui) for manual input, repeated turns, and artifact inspection. Read the [shared limitations](./hermes.mdx#limitations-and-planned-work), including approval policy, shared filesystem access, native activity visibility, and cleanup limits. The initial attachment helper supports local loopback clients; remote discovery and credential transport are outside this iteration. diff --git a/docs/integrations/harness/hermes/hermes.mdx b/docs/integrations/harness/hermes/hermes.mdx new file mode 100644 index 000000000..61dc37154 --- /dev/null +++ b/docs/integrations/harness/hermes/hermes.mdx @@ -0,0 +1,146 @@ +--- +title: "Use the Hermes Agent Adapter" +description: "Choose SDK, HTTP API, browser chat, or terminal TUI with explicit runtime ownership." +--- +{/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. +SPDX-License-Identifier: Apache-2.0 */} + +You can use Hermes Agent through the NVIDIA NeMo Fabric adapter in four ways: + +- **SDK:** Invoke the embedded agent programmatically and receive normalized NeMo Fabric results. +- **HTTP API:** Send requests to Hermes's native API and receive native HTTP responses. +- **Dashboard/browser TUI:** Chat with Hermes through its dashboard in a browser. +- **Terminal TUI:** Chat with Hermes from a terminal attached to the managed TUI backend. + +SDK execution is enabled by default; native interfaces are explicit options. All four paths use the normalized `FabricConfig`, which the adapter translates into Hermes's native configuration and agent settings. + +![Four client invocation paths share three Hermes execution backends within one NeMo Fabric runtime: embedded SDK, API gateway, and a TUI service shared by browser and terminal clients.](../../../../assets/hermes-invocation-paths.png) + +| Path | Execution backend | Guide | +| --- | --- | --- | +| SDK | Embedded `AIAgent` in the adapter host | [Hermes SDK](./hermes-sdk.mdx) | +| Native HTTP API | Managed API gateway process using `aiohttp` | [Hermes API](./hermes-api.mdx) | +| Dashboard/browser TUI | Managed FastAPI/Uvicorn TUI service with browser assets and a PTY bridge | [Hermes Dashboard and Browser TUI](./hermes-dashboard.mdx) | +| Terminal TUI | Terminal renderer attaches over WebSocket to the same TUI service; headless `serve` when dashboard is off | [Hermes Terminal TUI](./hermes-tui.mdx) | + +- **Agent execution:** All four paths use the same Hermes agent loop, with separate live agents and conversations. +- **Runtime ownership:** One NeMo Fabric runtime owns up to three backend processes: the adapter host, API gateway, and TUI service. +- **TUI clients:** Browser and terminal clients share the TUI service and use separate conversations. Neither requires the API gateway. + +## Install the Adapter + +Use [Hermes Agent v0.22.5](https://github.com/NousResearch/hermes-agent/releases/tag/v0.22.5) with Python 3.14. The adapter package supports Python 3.11 through 3.14, but Hermes Agent must be installed separately from source; versions 0.20 and later are not available from PyPI. Follow the [Hermes Agent installation guide](https://hermes-agent.nousresearch.com/docs/installation) and select v0.22.5. + +Install the runtime and adapter in the environment that runs Hermes: + +```bash +python -m pip install "nemo-fabric[hermes-agent]" +``` + +If the application runs NeMo Fabric in a separate environment, install `nemo-fabric-adapters-hermes` in the Hermes environment and set `ADAPTER_PYTHON` in the application's environment to that interpreter. Use matching runtime and adapter release versions unless a different pairing has been validated. + +For Relay, install the adapter's `relay` extra in the Hermes environment. It requires `nemo-relay>=0.10,<0.11`; keep this environment separate from applications using Relay 0.11 for other adapters. Startup rejects unsupported Hermes/Relay combinations. + +For repository development, the following recipe checks out v0.22.5 at `bf0203a794a07aac3ff764972ab9087baa4e4f4e` and prepares an isolated Python 3.14 environment with Relay support: + +```bash +just install-hermes-agent +export ADAPTER_PYTHON="$PWD/.venv-hermes/bin/python" +``` + +The installer preserves local checkout changes and stops rather than overwriting them. Set `HERMES_AGENT_CHECKOUT` to a separate clean checkout or a new directory to install without changing `external/hermes-agent`; the target environment remains `.venv-hermes`. + +Hermes reports its Python package version as `0.0.0`, so use its Git tag and commit to identify the installed release. The pinned release includes dependency security fixes absent from v2026.9.24. + +## Select Execution Paths + +Use normalized `FabricConfig` fields for model credentials/routing, instructions, tools, MCP, skills, workspace, turn limits, and telemetry. The [SDK guide](./hermes-sdk.mdx#configure-the-adapter) shows a complete model configuration. Use `harness.settings` to select Hermes execution paths: + +```python +settings = { + "sdk": {"enabled": False}, + "interfaces": { + "api": {"port": 8642}, + "dashboard": {"port": 9119}, + "terminal": {"port": 9119}, + }, + "startup_timeout_seconds": 60, +} +``` + +This example enables three native client paths without an embedded SDK agent. Omit `sdk` to preserve the default SDK agent. Omit an interface to turn it off; a present interface is on unless `enabled` is `false`. Current closed schemas require a port even for a disabled interface. Native interfaces require POSIX. Reject a configuration with no enabled execution path. + +- **Ports:** Dashboard and terminal must use the same port when both are enabled because they share one TUI service. The API requires a separate port. +- **Startup:** Terminal-only mode starts `hermes serve`; dashboard mode starts `hermes dashboard --isolated --skip-build`. Build the required frontends before runtime startup; terminal-only mode needs no browser assets. +- **Startup timeout:** Defaults to 60 seconds and must be greater than zero and no more than 75 seconds. + +## What NeMo Fabric Normalizes + +| Surface | SDK | Native API, Browser, and Terminal | +| --- | --- | --- | +| Configuration | Normalized fields mapped to the embedded agent | Same explicit behavioral mapping and generated native configuration in adapter-owned processes | +| Lifecycle | Runtime startup and owned shutdown when enabled | Managed child startup/readiness and shutdown; terminal detach does not stop the runtime | +| Results | Normalized `RunResult` | Native protocol/UI responses; no normalized invocation result | +| Relay evidence | Standard ATOF/ATIF, finalized/flushed per invocation | Standard native-session evidence, finalized/flushed per completed agent turn; verify export contents | +| Session artifacts | SDK invocation artifact collection | Complete session-scoped transcript/generated-file collection remains planned work | + +## Shared Configuration and Ownership + +### Configuration Translation + +Use a single `FabricConfig` to define supported settings for all four Hermes paths through NeMo Fabric's common configuration schema: + +- **Model:** provider, model, endpoint, credential references, and sampling settings. +- **System instructions:** the replacement system prompt. +- **Tool filters:** enabled and blocked Hermes toolsets. +- **MCP:** server definitions and supported authentication settings. +- **Skills:** skill search paths. +- **Workspace:** the agent's working directory. +- **Execution limits:** maximum agent turns and response tokens. +- **NeMo Relay:** telemetry and export configuration, when enabled. + +NeMo Fabric resolves `FabricConfig` into `AgentConfig`. The adapter translates it into Hermes's native configuration and agent settings, applying the same explicit instructions and behavioral settings to agents in managed native processes. + +The adapter writes native `config.yaml` and optional NeMo Relay plugin configuration under the runtime's Hermes home. When native interfaces are enabled, it also writes an owner-only `interfaces/agent-config.json` snapshot for the managed processes. + +Model credentials are supplied through child process environments, not public attachment metadata. For OpenAI without an explicit base URL, the adapter removes inherited `OPENAI_BASE_URL` from managed children so it cannot redirect the configured credential. + +### Setting Precedence + +- Configured instructions, turn/token limits, sampling, and reasoning settings override Hermes's native defaults. +- Explicit enabled tools, including an empty list, override native tool selection. A nonempty blocked list overrides native exclusions. +- Unspecified native tool policies can differ from SDK defaults. Set explicit policies when comparing behavior across paths. + +### Shared Runtime Home + +The runtime home is `/.fabric/hermes/runtimes/`. Agents share persisted resources in this directory while retaining separate live conversations. Keeping these files does not establish cross-runtime resume. + +### Process Ownership + +Managed native processes run through the adapter's launcher without modifying the installed Hermes source. Independently started processes do not automatically receive the adapter's configuration mapping. + +The managed API gateway disables ambient messaging platforms and profile multiplexing. Inherited settings such as `TELEGRAM_BOT_TOKEN` do not start chat-channel dependency installers in this runtime. + +## Health and Shutdown + +Startup checks every enabled interface: API `/health` and TUI `/api/status`; TUI startup also verifies an authenticated RPC without creating a conversation. An enabled interface that fails startup still fails runtime startup. + +When the SDK agent is enabled, runtime health and `idle`/`busy` activity describe that SDK agent. A separate API or TUI process failure does not fail SDK runtime health. Observe native endpoints and interface logs separately; a normalized interface health target is planned under [#566](https://github.com/NVIDIA/NeMo-Fabric/issues/566). + +With `sdk.enabled: false`, runtime checks inspect the enabled native children and endpoints. Native activity remains `unknown` because native turns are not fully aggregated. There is no unified live-agent inventory. + +Stop removes API/TUI credentials and attachment descriptors, sends SIGTERM to owned interface process groups, and escalates to SIGKILL after five seconds. API gateway drain is three seconds. The configuration snapshot is removed and owned SDK resources are closed. Cleanup errors are logged; failed private-file removal is retried on a later stop. + +## Limitations and Planned Work + +- Native response normalization, complete native usage accounting, and session-scoped artifact collection remain separate work. +- Child crash recovery, listener process-ownership checks, and stronger descendant cleanup when process-group signals are denied remain planned. HTTP/RPC readiness is not a live model or tool execution check. +- Browser chat is restricted to the managed runtime profile. Named-profile switching, gateway restart/migrate controls, and other dashboard operations can escape the supported ownership/configuration model. Do not use those controls to replace managed processes. +- Dashboard pages have no separate NeMo Fabric sign-in on loopback. Terminal credentials do not add a browser sign-in gate. Clients share filesystem/tools; separate conversations do not provide user isolation. +- TUI approvals and client detach/cancellation behavior need their own qualification; the adapter currently inherits `HERMES_YOLO_MODE` (defaulting to enabled). Detaching is not an explicit stop/cancel operation. +- Relay export errors are logged without replacing the execution outcome. Verify completed artifacts. Standard formatting does not establish complete telemetry coverage for every native control or background operation. +- Runtime-wide native activity discovery, same-conversation multi-client policy, cold resume, and SDK/native conversation handoff remain planned work. + +## Try the Interfaces + +See the [interfaces examples](../../../../examples/interfaces/hermes/README.md) for manual startup, real input, output checks, and Relay trajectory inspection. Coverage and remaining qualification limits are described there. diff --git a/docs/integrations/harness/overview.mdx b/docs/integrations/harness/overview.mdx index 4a2c517c1..a9b6b0b74 100644 --- a/docs/integrations/harness/overview.mdx +++ b/docs/integrations/harness/overview.mdx @@ -19,7 +19,7 @@ NeMo Fabric includes the following harness integrations: | Codex | [Use the Codex adapter](./codex.mdx) | | Cline | [Use the Cline adapter](./cline.mdx) | | Factory Droid | [Use the Factory Droid adapter](./droid.mdx) | -| Hermes Agent | [Use the Hermes Agent adapter](./hermes.mdx) | +| Hermes Agent | [Use the Hermes Agent adapter](./hermes/hermes.mdx) | | LangChain Deep Agents | [Use the LangChain Deep Agents adapter](./deepagents.mdx) | | Kilo Code | [Use the Kilo Code adapter](./kilo.mdx) | | Kiro | [Use the Kiro adapter](./kiro.mdx) | diff --git a/docs/sdk/health-checks.mdx b/docs/sdk/health-checks.mdx index fab7e2d26..f4fc3fd34 100644 --- a/docs/sdk/health-checks.mdx +++ b/docs/sdk/health-checks.mdx @@ -65,7 +65,7 @@ The table covers all runtime adapters and the OpenClaw Gateway service. Each row | [Codex](../integrations/harness/codex.mdx) | The owned app-server process has not exited. | A fresh native thread read confirms that the selected thread is loaded. | The fresh thread response reports `idle` or `active`; `systemError` fails readiness. | [Tests](https://github.com/NVIDIA/NeMo-Fabric/blob/86c52264d6e2ecfe7f72f1eb0161f1d1e425ddbb/tests/e2e/test_codex_checks.py) | | [Claude Code](../integrations/harness/claude.mdx) | The owned Claude CLI process has not exited. | A fresh, valid native `get_mcp_status()` response. | The same native status operation establishes initialized CLI responsiveness. Reported MCP connection failures do not themselves fail readiness. | [Tests](https://github.com/NVIDIA/NeMo-Fabric/blob/563ca53755f1a2f57329a79985095acfcfcd5cde/tests/e2e/test_claude_checks.py) | | [LangChain Deep Agents](../integrations/harness/deepagents.mdx) | A fresh read from the owned checkpoint connection with `SELECT 1`. | The compiled graph exposes its message input schema and invocation interface. | Native `aget_state()` returns valid state for the exact owned LangGraph thread. | [Tests](https://github.com/NVIDIA/NeMo-Fabric/blob/5c065c1a7775e4490269f57b406635d30e8a0fd8/tests/e2e/test_deepagents_checks.py) | -| [Hermes](../integrations/harness/hermes.mdx) | The owned native agent is not closed, session database ownership is unchanged, and a fresh diagnostic response is valid. | The initialized `run_conversation()` interface remains available after the earlier checks. | A fresh, valid `get_activity_summary()` response serves as the native readiness equivalent. | [Tests](https://github.com/NVIDIA/NeMo-Fabric/blob/a51d5b0bca28261e61f56866b3fe8d93fddb4f7d/tests/e2e/test_hermes_checks.py) | +| [Hermes](../integrations/harness/hermes/hermes.mdx) | SDK enabled: agent/session database ownership and fresh diagnostics. SDK disabled: enabled native processes have not exited. | SDK enabled: initialized `run_conversation()` and diagnostics. SDK disabled: process checks plus API `/health` or TUI `/api/status` and authenticated RPC. | SDK enabled: valid `get_activity_summary()` with SDK idle/busy activity. SDK disabled: native process/endpoint checks with unknown activity. Separate interface failures do not fail an enabled SDK agent's runtime health. | [SDK tests](https://github.com/NVIDIA/NeMo-Fabric/blob/a51d5b0bca28261e61f56866b3fe8d93fddb4f7d/tests/e2e/test_hermes_checks.py), [native interface tests](https://github.com/NVIDIA/NeMo-Fabric/blob/1e9533f44f135da548441870f06dec3aec2ed1eb/tests/adapters/test_hermes_interfaces.py) | | [OpenHands](../integrations/harness/openhands.mdx) | The owned `LocalConversation` native closure flags show that the conversation remains open. | The native lazy-initialization flag confirms that the request interface has initialized. Before initialization, returns `unknown/capabilities_unverified`. | Unsupported. | [Tests](https://github.com/NVIDIA/NeMo-Fabric/blob/6df3e0cef0ea972f125c0217d44ce7246be3bc68/tests/e2e/test_openhands_checks.py) | | [OpenCode](../integrations/harness/opencode.mdx) | A fresh `sessions.get({ sessionID })` response identifies the exact, unarchived native session. | The same native session-read operation. | The same native session-read operation establishes narrow session readiness. | [Tests](https://github.com/NVIDIA/NeMo-Fabric/blob/ba027bb56aedae3adf6de57ca0d33846a0788932/tests/e2e/test_opencode_checks.py) | | [Qwen Code](../integrations/harness/qwen.mdx) | A fresh native `Query.supportedCommands()` response from the owned CLI query. | The same native control operation. | The same native control operation establishes initialized CLI responsiveness. | [Tests](https://github.com/NVIDIA/NeMo-Fabric/blob/9e483024adad293eb1bbdf105c93401c15b5c087/tests/e2e/test_qwen_checks.py) | diff --git a/docs/sdk/python.mdx b/docs/sdk/python.mdx index c7223516b..cb8570910 100644 --- a/docs/sdk/python.mdx +++ b/docs/sdk/python.mdx @@ -748,7 +748,7 @@ plugin_config.relay = RelayConfig( plugin_config.enable_relay() ``` -NeMo Fabric preserves the `relay.plugins` section alongside any configured `relay.observability`, `relay.components`, and `relay.policy`. Relay owns manifest validation, loading, and activation. Use absolute manifest paths accessible inside the agent environment. Plugin activation depends on the selected adapter's Relay integration and the installed Relay version. For Hermes installation requirements, refer to [Use the Hermes Agent Adapter](../integrations/harness/hermes.mdx). +NeMo Fabric preserves the `relay.plugins` section alongside any configured `relay.observability`, `relay.components`, and `relay.policy`. Relay owns manifest validation, loading, and activation. Use absolute manifest paths accessible inside the agent environment. Plugin activation depends on the selected adapter's Relay integration and the installed Relay version. For Hermes installation requirements, refer to [Use the Hermes Agent Adapter](../integrations/harness/hermes/hermes.mdx). ## API Inventory diff --git a/examples/README.md b/examples/README.md index 5703385db..ff6d0e988 100644 --- a/examples/README.md +++ b/examples/README.md @@ -99,3 +99,8 @@ Start with the shared setup and execution model in the ## Software Evidence [Selected software evidence](provenance/README.md) reproduces six execution paths, separate venvs, disabled collection and stable version comparison with explicit coverage limits. + + +## Native Hermes Interfaces + +The [interfaces examples](interfaces/hermes/README.md) exercise the managed Hermes API and provide a manual terminal/browser TUI host. Terminal and browser clients share one runtime-owned TUI service; SDK execution can be disabled explicitly for native-only use. diff --git a/examples/code_review_agent/README.md b/examples/code_review_agent/README.md index 0bfc5ff57..826e1b51d 100644 --- a/examples/code_review_agent/README.md +++ b/examples/code_review_agent/README.md @@ -93,8 +93,10 @@ combined. ### Enable Relay Telemetry and Streaming Install Relay as described in the -[Relay installation guide](../../docs/getting-started/install.mdx#install-nemo-relay), -and then enable Relay telemetry to collect Agent Trajectory Observability +[Relay installation guide](../../docs/getting-started/install.mdx#install-nemo-relay). +For `--stream`, also install `nemo-fabric[streaming]` in the SDK interpreter's environment, as described in the [runtime installation guide](../../docs/getting-started/install.mdx#install-the-nemo-fabric-runtime). When `ADAPTER_PYTHON` selects a separate environment, the SDK host still needs this streaming extra. + +Enable Relay telemetry to collect Agent Trajectory Observability Format (ATOF) stream records: ```bash diff --git a/examples/code_review_agent/config.py b/examples/code_review_agent/config.py index e4f0c128a..3d417e717 100644 --- a/examples/code_review_agent/config.py +++ b/examples/code_review_agent/config.py @@ -99,7 +99,7 @@ def hermes_config() -> FabricConfig: adapter_id="nvidia.fabric.hermes", resolution="preinstalled", settings={ - "max_tokens": 512, + "max_tokens": 2048, "reasoning_config": {"effort": "none"}, }, ) @@ -107,14 +107,16 @@ def hermes_config() -> FabricConfig: assert isinstance(model, ModelConfig) model.base_url = "https://integrate.api.nvidia.com/v1" config.instructions = InstructionsConfig( - system=InstructionConfig(content="You are a concise smoke test assistant.") + system=InstructionConfig( + content="Review the requested files for correctness risks. Read the files with file tools before reviewing. Do not modify files. Cite file and line references for findings; if none are found, say so and identify any test-coverage uncertainty." + ) ) - config.tools = ToolsConfig(enabled=[]) + config.tools = ToolsConfig(enabled=["file"]) config.runtime = RuntimeConfig( input_schema="chat", output_schema="message", artifacts="./artifacts/hermes", - max_turns=1, + max_turns=8, ) config.environment = EnvironmentConfig( provider="local", diff --git a/examples/harbor/swebench/adapters/hermes/hermes.fabric-adapter.json b/examples/harbor/swebench/adapters/hermes/hermes.fabric-adapter.json index 0ceeda9e3..458ce7fdf 100644 --- a/examples/harbor/swebench/adapters/hermes/hermes.fabric-adapter.json +++ b/examples/harbor/swebench/adapters/hermes/hermes.fabric-adapter.json @@ -19,7 +19,14 @@ }, "effort": { "type": "string", - "enum": ["none", "minimal", "low", "medium", "high", "xhigh"], + "enum": [ + "none", + "minimal", + "low", + "medium", + "high", + "xhigh" + ], "description": "Hermes model reasoning effort." } }, @@ -73,7 +80,9 @@ "description": "Exact loopback port for the API server. Start fails if it is in use." } }, - "required": ["port"], + "required": [ + "port" + ], "additionalProperties": false, "description": "Hermes's API server, run by a gateway child. After every enabled interface starts, the adapter writes its bearer token to /.fabric/hermes/runtimes//interfaces/api.token." }, @@ -92,9 +101,31 @@ "description": "Exact loopback port for the dashboard. Start fails if it is in use." } }, - "required": ["port"], + "required": [ + "port" + ], "additionalProperties": false, "description": "Hermes's dashboard, which has no sign-in when bound to loopback. It needs a built frontend at HERMES_WEB_DIST." + }, + "terminal": { + "type": "object", + "properties": { + "enabled": { + "type": "boolean", + "default": true + }, + "port": { + "type": "integer", + "minimum": 1, + "maximum": 65535, + "description": "Experimental TUI backend loopback port. Must match dashboard.port when both interfaces are enabled. This required field may be removed when the #579 launcher contract is adopted." + } + }, + "required": [ + "port" + ], + "additionalProperties": false, + "description": "Experimental terminal attachment to a managed TUI backend. Uses headless serve without dashboard assets, or shares the dashboard process. The configuration (including terminal.port), interfaces/tui.json, interfaces/tui.token, and the nemo_fabric_adapters.hermes.terminal helper may change across releases as the #579 launcher contract is adopted; compatibility is not guaranteed for these surfaces." } }, "additionalProperties": false, @@ -106,17 +137,105 @@ "maximum": 75, "default": 60, "description": "Time from the start of the runtime for every enabled interface to pass its health check, within NeMo Fabric's 90-second start limit." + }, + "sdk": { + "type": "object", + "properties": { + "enabled": { + "type": "boolean", + "default": true, + "description": "Initialize the embedded SDK agent during startup. Disable for native-only execution; SDK invocation then fails explicitly." + } + }, + "additionalProperties": false, + "description": "Embedded SDK execution, enabled by default. Runtime management remains available when disabled." } }, "required": [], - "additionalProperties": false + "additionalProperties": false, + "allOf": [ + { + "if": { + "required": [ + "sdk" + ], + "properties": { + "sdk": { + "required": [ + "enabled" + ], + "properties": { + "enabled": { + "const": false + } + } + } + } + }, + "then": { + "required": [ + "interfaces" + ], + "properties": { + "interfaces": { + "anyOf": [ + { + "required": [ + "api" + ], + "properties": { + "api": { + "properties": { + "enabled": { + "const": true + } + } + } + } + }, + { + "required": [ + "dashboard" + ], + "properties": { + "dashboard": { + "properties": { + "enabled": { + "const": true + } + } + } + } + }, + { + "required": [ + "terminal" + ], + "properties": { + "terminal": { + "properties": { + "enabled": { + "const": true + } + } + } + } + } + ] + } + } + } + } + ] }, "extension_schemas": { "run_error": { "$schema": "https://json-schema.org/draft/2020-12/schema", "type": "object", "properties": { - "provider": {"type": "string"} + "provider": { + "type": "string" + } }, "additionalProperties": false } @@ -136,7 +255,9 @@ "mcp.auth.oauth2", "skills" ], - "system_instruction_modes": ["replace"] + "system_instruction_modes": [ + "replace" + ] }, "telemetry": { "providers": { @@ -150,6 +271,12 @@ } }, "capabilities": { - "metadata": {"health_checks": ["live", "active", "ready"]} + "metadata": { + "health_checks": [ + "live", + "active", + "ready" + ] + } } } diff --git a/examples/interfaces/hermes/README.md b/examples/interfaces/hermes/README.md index 915ad1026..9216727c5 100644 --- a/examples/interfaces/hermes/README.md +++ b/examples/interfaces/hermes/README.md @@ -5,24 +5,31 @@ SPDX-License-Identifier: Apache-2.0 # NVIDIA NeMo Fabric Hermes Interface Examples -Start with the [Hermes architecture guide](../../../docs/integrations/harness/hermes.mdx) for installation and SDK configuration, then see [native interfaces](../../../docs/integrations/harness/hermes-interfaces.mdx) for shared configuration and process ownership. This example tests the native HTTP API using a Fabric-managed runtime and translated Fabric configuration. SDK responses are normalized; API responses remain native. NeMo Relay provides ATOF events and ATIF trajectories. +Start with the [Hermes architecture guide](../../../docs/integrations/harness/hermes/hermes.mdx). These four paths use shared Fabric configuration translated into Hermes native configuration. SDK responses are normalized; other responses remain native. NeMo Relay provides ATOF events and ATIF trajectories. Use this README to test each path: -## Hermes API and Relay +| Test Path | Entry Point | Architecture and Setup | +| --- | --- | --- | +| [1. SDK](#1-sdk) | [Code Review Example](../../code_review_agent/README.md) | [SDK guide](../../../docs/integrations/harness/hermes/hermes-sdk.mdx) | +| [2. HTTP API](#2-http-api) | `hermes.py` | [API guide](../../../docs/integrations/harness/hermes/hermes-api.mdx) | +| [3. Browser dashboard](#3-browser-dashboard) | `hermes_tui.py --dashboard`, then browser chat | [Dashboard guide](../../../docs/integrations/harness/hermes/hermes-dashboard.mdx) | +| [4. Terminal TUI](#4-terminal-tui) | `hermes_tui.py`, then terminal attachment | [Terminal guide](../../../docs/integrations/harness/hermes/hermes-tui.mdx) | -[`hermes.py`](hermes.py) starts one managed Hermes API server, submits a real file task, verifies the generated file, and checks Relay ATOF events and ATIF tool steps. Its API response remains native; the SDK manages the runtime but does not convert this response into a NeMo Fabric `RunResult`. +## Shared Setup for Native Interfaces -### Prepare the Environment +Paths 2–4 require a POSIX host and use the following setup. For SDK testing, follow the code-review example linked in [Path 1](#1-sdk). -Follow the [Hermes adapter installation guide](../../../docs/integrations/harness/hermes.mdx#install-the-adapter). For source development, run from the repository root: +Run commands from the NeMo Fabric repository root, including commands in additional terminals. Follow the [shared installation guide](../../../docs/integrations/harness/hermes/hermes.mdx#install-the-adapter). For repository development: ```bash just install-hermes-agent -export ADAPTER_PYTHON="$PWD/.venv-hermes/bin/python" +export ADAPTER_PYTHON="${ADAPTER_PYTHON:-$PWD/.venv-hermes/bin/python}" ``` -This prepares pinned Hermes v0.22.5, the adapter, and Relay 0.10. The application interpreter also needs the NeMo Fabric SDK. The command below uses the prepared Hermes environment for both application and adapter. +The export preserves a separately selected adapter interpreter; otherwise it uses `.venv-hermes`. The example scripts below run in `.venv-hermes`, which provides their NeMo Fabric SDK host. -Set your OpenAI-compatible model endpoint, served model name, and credential: +This prepares pinned Hermes v0.22.5, the NeMo Fabric adapter/SDK, and NeMo Relay 0.10 in `.venv-hermes`. The API dependency `aiohttp` and the Python web-serving dependencies needed by the TUI backend are included. Use `jq` for the JSON inspection commands below. + +Set the model endpoint, served model name, and credential used by all three native tests: ```bash export MODEL_BASE_URL="https://MODEL_HOST/v1" @@ -30,65 +37,201 @@ export MODEL_NAME="YOUR_SERVED_MODEL_NAME" export MODEL_API_KEY="YOUR_MODEL_API_KEY" ``` -Use HTTPS for remote endpoints that receive credentials. For a local server, use a loopback URL such as `http://127.0.0.1:8000/v1`; an authenticated encrypted tunnel is another option. Use the credential value required by your endpoint. An endpoint without authentication can use a nonempty placeholder credential. To reference an existing credential variable instead, pass its name with `--api-key-env`. +Use the credential required by your endpoint; an endpoint without authentication can use a nonempty placeholder. To use another credential environment variable, pass its name with `--api-key-env`. The endpoint must be reachable from the adapter environment. Use HTTPS for credential-bearing remote requests, or an authenticated encrypted tunnel; a local model can use a loopback URL such as `http://127.0.0.1:8000/v1`. + +Each test creates a fresh evidence directory. The commands below include a timestamp; use a new directory when repeating a test because the scripts refuse to overwrite an existing one. Choose available ports. The native services bind to loopback in the adapter environment; accessing a remote adapter requires an appropriate port forward. + +### Build the TUI Frontends -### Run a Real Task +Complete this step for Paths 3 and 4. SDK and API testing do not require frontend builds. -Run the example from the repository root: +Use the same pinned Hermes checkout selected during installation. If `HERMES_AGENT_CHECKOUT` was set for the installer, keep that value. Otherwise the default is `external/hermes-agent`: ```bash +export HERMES_AGENT_CHECKOUT="${HERMES_AGENT_CHECKOUT:-$PWD/external/hermes-agent}" +"$ADAPTER_PYTHON" -c "import aiohttp, fastapi, uvicorn" +( + cd "$HERMES_AGENT_CHECKOUT" + npm ci --ignore-scripts --workspace ui-tui --workspace web --include-workspace-root + npm run build --workspace ui-tui + npm run build --workspace web +) +export HERMES_TUI_DIR="$HERMES_AGENT_CHECKOUT/ui-tui" +export HERMES_WEB_DIST="$HERMES_AGENT_CHECKOUT/hermes_cli/web_dist" +``` + +For terminal-only testing, omit `npm run build --workspace web` and the `HERMES_WEB_DIST` export. Use Node supported by the pinned release (`^22.22.0`, `^24.11.0`, or `>=26.0.0`); the live client checks used Node 22.22.2. For a separately prepared Python environment, follow the additional dependency steps in the [dashboard](../../../docs/integrations/harness/hermes/hermes-dashboard.mdx#installation) or [terminal](../../../docs/integrations/harness/hermes/hermes-tui.mdx#installation) guide. + +## 1. SDK + +Use the [Code Review Example](../../code_review_agent/README.md#run-the-default-demo), which selects Hermes by default. That example covers normalized configuration, SDK invocation, output, and artifacts. Its [NeMo Relay instructions](../../code_review_agent/README.md#enable-relay-telemetry-and-streaming) describe telemetry and streaming options. + +The SDK path runs the embedded Hermes `AIAgent` and returns a normalized NeMo Fabric `RunResult`. Refer to the [SDK architecture guide](../../../docs/integrations/harness/hermes/hermes-sdk.mdx) for its lifecycle and conversation behavior. + +## 2. HTTP API + +Architecture: [Hermes API](../../../docs/integrations/harness/hermes/hermes-api.mdx). [`hermes.py`](hermes.py) starts the native API, submits a real file task, and verifies its output and NeMo Relay evidence. The script manages the runtime through the SDK but sends the task through HTTP; the response remains native. This example retains the default embedded SDK agent, but does not invoke it. + +### Start and Run the Task + +After shared setup, run: + +```bash +export RUN_DIR="$PWD/outputs/interfaces/api-$(date +%Y%m%d-%H%M%S)" .venv-hermes/bin/python examples/interfaces/hermes/hermes.py \ --base-url "$MODEL_BASE_URL" \ --model "$MODEL_NAME" \ - --port 8642 + --port 18642 \ + --output-dir "$RUN_DIR" ``` -The example copies sample release notes into a new workspace, asks the native API to read them and write a rollout checklist, and stops the runtime. To use your own notes, add `--input-file /path/to/release-notes.md`. `--output-dir` selects a new evidence directory; the example refuses to overwrite an existing directory. `--keep-running` leaves the API available for additional requests until you press Enter. +The script copies sample release notes into the workspace, asks Hermes to read them and write `rollout-checklist.md`, then stops the runtime. Use `--input-file /path/to/release-notes.md` for your own notes. + +### Inspect the Result -The model and port must be reachable from the adapter environment. The native API binds to loopback and uses a generated bearer token; the example reads the token file and sends the header without printing the token. +The command exits successfully when the generated file, tool events, ATIF tool steps, and API-token cleanup checks pass: -The example prints its evidence directory and checks that: +```bash +cat "$RUN_DIR/workspace/rollout-checklist.md" +jq . "$RUN_DIR/summary.json" +jq -e '.checks | all' "$RUN_DIR/summary.json" +``` -- The output file exists and is nonempty. -- ATOF records show `read_file` and `write_file` execution. -- A populated ATIF trajectory contains those tool steps. -- Stop removes the API token. +The last command should print `true`. The summary lists the ATOF and ATIF files; see [Inspect NeMo Relay Trajectories](#inspect-nemo-relay-trajectories) for their contents. -It exits unsuccessfully if any check fails. An ATOF file alone does not establish ATIF export. +### Send Your Own API Input -### Inspect the Result and Trajectory +To leave the API open after its initial task, add `--keep-running` to the start command, using a fresh `RUN_DIR`. The owner terminal waits for Enter after the task completes. -Set `RUN_DIR` to the evidence directory printed by the example: +In a second terminal at the repository root, set `RUN_DIR` to the evidence directory printed by the owner, then submit another request: ```bash -RUN_DIR="outputs/interfaces/" -cat "$RUN_DIR/workspace/rollout-checklist.md" -jq . "$RUN_DIR/summary.json" +export RUN_DIR="/absolute/path/to/the/printed/evidence-directory" +export API_PROMPT="Read $RUN_DIR/workspace/release-notes.md using file tools and summarize the release changes." +.venv-hermes/bin/python - <<'PY' +import os +from pathlib import Path +from examples.interfaces.hermes import call_api + +root = Path(os.environ["RUN_DIR"]) +[token_file] = (root / "artifacts/.fabric/hermes/runtimes").glob("*/interfaces/api.token") +response = call_api("http://127.0.0.1:18642", token_file.read_text(), os.environ["API_PROMPT"]) +print(response["choices"][0]["message"]["content"]) +PY +``` -ATIF=$(jq -r '.atif_files[0]' "$RUN_DIR/summary.json") -jq -r '.steps[] | - [.step_id, .source, ((.tool_calls // []) | map(.function_name) | join(","))] | - @tsv' "$ATIF" +Edit `API_PROMPT` to test another input. The helper reads the token file and sends `Authorization: Bearer ` without printing the token. If you changed the API port, use the same port in this request. + +Press Enter in the owner terminal to stop the runtime. The script then writes `summary.json` and checks that the token was removed. Without `--keep-running`, stop and verification happen automatically. + +## 3. Browser Dashboard + +Architecture: [Hermes Dashboard and Browser TUI](../../../docs/integrations/harness/hermes/hermes-dashboard.mdx). This path uses the dashboard's TUI backend rather than the API gateway or embedded SDK agent. + +### Start the Dashboard + +Complete shared setup and both frontend builds. In the owner terminal, run: + +```bash +export RUN_DIR="$PWD/outputs/interfaces/browser-$(date +%Y%m%d-%H%M%S)" +.venv-hermes/bin/python examples/interfaces/hermes/hermes_tui.py \ + --base-url "$MODEL_BASE_URL" \ + --model "$MODEL_NAME" \ + --dashboard \ + --port 19119 \ + --output-dir "$RUN_DIR" ``` -For this task, expect user input, an agent `read_file` step with its observation, an agent `write_file` step with its observation, and a final response. Inspect complete tool arguments and observations with: +The script prints a browser URL, an absolute-path file task after `Try:`, and a terminal attachment command. It stays running until you press Enter in the owner terminal; this is expected. For this test, use the browser URL and leave the terminal attachment command unused. + +The host example enables terminal attachment alongside the dashboard, sharing one backend. It disables the embedded SDK agent by default and does not start an API gateway. + +### Interact in the Browser + +1. Open the printed URL, normally `http://127.0.0.1:19119`, and open Chat. +2. Copy the file task printed after `Try:` into chat. It reads `workspace/notes.txt` and writes a three-item `workspace/checklist.md` using file tools. +3. Wait for the final response. The chat should show the file-tool activity and the created checklist. +4. Send a follow-up in the same chat, such as “Summarize the three checks we just wrote.” +5. In another terminal, set `RUN_DIR` to the printed evidence directory. Run `cat "$RUN_DIR/workspace/checklist.md"` and follow the [trajectory inspection steps](#inspect-nemo-relay-trajectories). + +### Stop the Dashboard + +Press Enter in the owner terminal. The runtime stops its TUI service and removes the attachment files. Browser and terminal output remains native; it does not become a NeMo Fabric `RunResult`. + +## 4. Terminal TUI + +**Experimental:** The terminal configuration, attachment files, and helper may change across releases. Read the [terminal interface notice](../../../docs/integrations/harness/hermes/hermes-tui.mdx) before relying on their compatibility. + +Architecture: [Hermes Terminal TUI](../../../docs/integrations/harness/hermes/hermes-tui.mdx). Terminal-only mode uses the managed TUI backend without serving the browser dashboard. + +### Start the Backend + +Complete shared setup and the terminal frontend build. In the owner terminal, run: ```bash -jq '.steps[] | select(.tool_calls != null) | - {tool_calls, observation}' "$ATIF" +export RUN_DIR="$PWD/outputs/interfaces/terminal-$(date +%Y%m%d-%H%M%S)" +.venv-hermes/bin/python examples/interfaces/hermes/hermes_tui.py \ + --base-url "$MODEL_BASE_URL" \ + --model "$MODEL_NAME" \ + --port 19119 \ + --output-dir "$RUN_DIR" ``` -To inspect the raw event sequence: +The script starts a headless TUI service, with the embedded SDK agent and API gateway disabled. It prints an attachment command and file task, then waits for Enter in the owner terminal. Stop the browser test first if you want to reuse port 19119. + +### Attach and Interact + +1. In a second interactive terminal at the repository root, run the exact attachment command printed by the owner. It uses the Hermes interpreter and that runtime's private `interfaces/tui.json` file. +2. Paste the file task printed after `Try:` into the Hermes terminal UI. +3. Wait for the final response and inspect the displayed file-tool activity. +4. Send a follow-up in the same conversation, such as “Summarize the three checks we just wrote.” +5. In another terminal, set `RUN_DIR` to the printed evidence directory. Run `cat "$RUN_DIR/workspace/checklist.md"` and follow the [trajectory inspection steps](#inspect-nemo-relay-trajectories). + +The attachment helper reads a TUI credential separate from the API bearer token. It attaches to the existing service and does not start another backend. + +### Detach and Stop + +Enter `/quit` in the Hermes terminal UI to close the client. The owner process and backend remain running. Press Enter in the owner terminal to stop the runtime and remove its attachment files. For this test, let the turn complete before detaching. + +## Inspect NeMo Relay Trajectories + +NeMo Relay is enabled by default in both native example scripts. Inspect evidence while a TUI client stays connected, or after the API script finishes. In a separate terminal, set `RUN_DIR` to the evidence directory printed by the owner: ```bash -ATOF=$(jq -r '.atof_files[0]' "$RUN_DIR/summary.json") -jq -r 'select(.kind == "scope" and - (.category == "llm" or .category == "tool")) | - [.timestamp, .scope_category, .category, .name] | @tsv' "$ATOF" +export RUN_DIR="/absolute/path/to/the/printed/evidence-directory" +find "$RUN_DIR/relay/atif" -type f -name '*.json' -print ``` -### ATIF Finalization and Limits +Choose a trajectory from the list and inspect its tool calls and observations: + +```bash +ATIF="/absolute/path/to/a/listed/trajectory.json" +jq '.steps[] | {step_id, source, tool_calls, observation}' "$ATIF" +``` + +For the API example, the generated file is `workspace/rollout-checklist.md`. Its summary also gives the trajectory path directly: + +```bash +ATIF=$(jq -r '.atif_files[0]' "$RUN_DIR/summary.json") +jq '.steps[] | {step_id, source, tool_calls, observation}' "$ATIF" +``` + +Expect the file task's `read_file` and `write_file` calls with observations, followed by a final response. A follow-up turn produces another segment while preserving the native conversation history. To inspect the ATOF event sequence: + +```bash +find "$RUN_DIR/relay/atof" -type f -name '*.jsonl' -exec \ + jq -r 'select(.kind == "scope" and (.category == "llm" or .category == "tool")) | + [.timestamp, .scope_category, .category, .name] | @tsv' {} \; +``` + +The [architecture guide's normalization table](../../../docs/integrations/harness/hermes/hermes.mdx#what-nemo-fabric-normalizes) explains the boundary: NeMo Relay provides common evidence formats, while native API and TUI responses retain native session identities and response types. + +## Test Browser and Terminal Together + +Start the browser example in Path 3 and run its printed terminal attachment command in a second terminal. Submit a task in each client, choosing different output filenames. Both clients use one backend and separate conversations. Follow the same inspection and owner-stop steps. + +The host script also accepts `--sdk` to enable the embedded SDK agent and `--api-port 18642` to enable the separate API gateway. Enabling SDK execution does not submit an SDK task automatically; use the code-review example for the standalone SDK test. API and TUI ports must differ. -The pinned Hermes API path leaves its Relay conversation scope open after a turn. ATOF events can therefore exist without ATIF export. The managed native launcher finalizes and flushes an API turn's Relay segment when execution returns, including failed turns. It does not fire native conversation-end plugin hooks or delete conversation state. The conversation can continue; subsequent turns create new Relay segments and ATIF files. Finalization errors are logged without replacing the execution outcome, so consumers should check their artifacts explicitly. +## Limitations -This example exercises the managed API path. It does not qualify browser chat, TUI, independently restarted gateways, approvals, or complete native-result and artifact normalization. Refer to [Hermes Native Interfaces](../../../docs/integrations/harness/hermes-interfaces.mdx) for the configuration and ownership boundaries. +Interactive approvals, detach during active work, remote terminal attachment, named-profile switching, same-conversation multi-client control, and independently replaced gateways are outside these manual flows. The adapter defaults `HERMES_YOLO_MODE` to enabled. Complete native-result, usage, and session-artifact normalization remain planned work; see [Limitations and Planned Work](../../../docs/integrations/harness/hermes/hermes.mdx#limitations-and-planned-work). diff --git a/examples/interfaces/hermes/hermes_tui.py b/examples/interfaces/hermes/hermes_tui.py new file mode 100644 index 000000000..0acc40f11 --- /dev/null +++ b/examples/interfaces/hermes/hermes_tui.py @@ -0,0 +1,181 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +"""Host a managed Hermes TUI backend for manual terminal and browser interaction.""" + +from __future__ import annotations + +import argparse +import asyncio +import json +import os +import shlex +import sys +import uuid +from pathlib import Path + +from nemo_fabric import Fabric, FabricConfig +from nemo_fabric import RelayAtifConfig, RelayAtofConfig, RelayAtofFileSinkConfig +from nemo_fabric import RelayObservabilityConfig + + +def build_config(args: argparse.Namespace, output: Path) -> FabricConfig: + """Prepare a file-task workspace and explicit native paths with optional Relay.""" + workspace = output / "workspace" + workspace.mkdir(parents=True, exist_ok=True) + (workspace / "notes.txt").write_text( + "Check authentication, loopback binding, and Relay trajectories.\n" + ) + interfaces = {"terminal": {"port": args.port}} + if args.dashboard: + interfaces["dashboard"] = {"port": args.port} + if args.api_port: + interfaces["api"] = {"port": args.api_port} + config = FabricConfig.from_mapping( + { + "metadata": {"name": "hermes-tui-example"}, + "harness": { + "adapter_id": "nvidia.fabric.hermes", + "settings": {"sdk": {"enabled": args.sdk}, "interfaces": interfaces}, + }, + "models": { + "default": { + "provider": "openai", + "model": args.model, + "base_url": args.base_url, + "api_key_env": args.api_key_env, + "temperature": 0.0, + "max_tokens": 2048, + } + }, + "instructions": { + "system": { + "mode": "replace", + "content": "Complete workspace tasks using file tools. Read source files before writing. Do not use shell or web tools. Summarize actual results briefly.", + } + }, + "tools": {"enabled": ["file"], "blocked": ["terminal"]}, + "runtime": {"max_turns": 8}, + "environment": { + "provider": "local", + "workspace": str(workspace), + "artifacts": str(output / "artifacts"), + }, + } + ) + if not args.no_relay: + relay = output / "relay" + config.enable_relay( + output_dir=str(relay), + observability=RelayObservabilityConfig( + atof=RelayAtofConfig( + enabled=True, + sinks=[ + RelayAtofFileSinkConfig( + output_directory=str(relay / "atof"), + filename="events.atof.jsonl", + mode="append", + ) + ], + ), + atif=RelayAtifConfig( + enabled=True, output_directory=str(relay / "atif") + ), + ), + ) + return config + + +async def run(args: argparse.Namespace) -> None: + """Own the runtime until Enter, keeping browser/terminal clients independent.""" + output = ( + args.output_dir or Path("outputs/interfaces") / f"tui-{uuid.uuid4().hex[:12]}" + ).resolve() + output.mkdir(parents=True, exist_ok=False) + config = build_config(args, output) + print(f"Evidence directory: {output}\nStarting managed TUI backend...", flush=True) + async with await Fabric().start_runtime(config, base_dir=Path.cwd()) as runtime: + connection = ( + output + / "artifacts/.fabric/hermes/runtimes" + / runtime.runtime_id + / "interfaces/tui.json" + ) + (output / "runtime.json").write_text( + json.dumps( + {"runtime_id": runtime.runtime_id, "connection": str(connection)}, + indent=2, + ) + + "\n" + ) + command = shlex.join( + [ + sys.executable, + "-m", + "nemo_fabric_adapters.hermes.terminal", + "--connection", + str(connection), + ] + ) + print(f"In a second terminal, run:\n{command}", flush=True) + if args.dashboard: + print(f"Browser: http://127.0.0.1:{args.port}", flush=True) + print( + f"Try: Read {output / 'workspace/notes.txt'} and write a three-item checklist to {output / 'workspace/checklist.md'} using file tools.", + flush=True, + ) + print( + f"Inspect completed trajectories under {output / 'relay/atif'} while clients remain connected.", + flush=True, + ) + await asyncio.to_thread( + input, + "Press Enter here to stop the runtime (detaching a client does not stop it).\n", + ) + await runtime.stop() + print("Runtime stopped; attachment files removed.", flush=True) + + +def main() -> None: + """Validate endpoint and port arguments before starting the interactive host.""" + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--base-url", default=os.environ.get("MODEL_BASE_URL")) + parser.add_argument("--model", default=os.environ.get("MODEL_NAME")) + parser.add_argument("--api-key-env", default="MODEL_API_KEY") + parser.add_argument("--port", type=int, default=19119) + parser.add_argument( + "--dashboard", + action="store_true", + help="Share the TUI backend with browser chat; requires built web assets", + ) + parser.add_argument( + "--sdk", + action="store_true", + help="Also start the embedded SDK agent; this native-only example explicitly disables it otherwise", + ) + parser.add_argument( + "--api-port", type=int, help="Also start a separate native API gateway" + ) + parser.add_argument("--no-relay", action="store_true") + parser.add_argument( + "--output-dir", type=Path, help="New directory for workspace and evidence" + ) + args = parser.parse_args() + if not args.base_url or not args.model or not os.environ.get(args.api_key_env): + parser.error( + "provide model endpoint/name and its credential environment variable" + ) + if not 1 <= args.port <= 65535 or ( + args.api_port is not None and not 1 <= args.api_port <= 65535 + ): + parser.error("ports must be between 1 and 65535") + if args.api_port == args.port: + parser.error("API and TUI backend ports must differ") + try: + asyncio.run(run(args)) + except (KeyboardInterrupt, EOFError): + pass + + +if __name__ == "__main__": + main() diff --git a/justfile b/justfile index 95eac0d96..0d0febb0e 100644 --- a/justfile +++ b/justfile @@ -457,7 +457,7 @@ install-hermes-agent: pytest pytest-asyncio pyyaml test-hermes: install-hermes-agent - .venv-hermes/bin/python -m pytest tests/adapters/test_hermes_adapter.py tests/adapters/test_hermes_config_builder.py tests/adapters/test_hermes_native_config.py tests/adapters/test_hermes_interfaces.py tests/adapters/test_hermes_streaming.py tests/adapters/test_hermes_health.py tests/e2e/test_hermes_e2e.py tests/e2e/test_hermes_checks.py -q + .venv-hermes/bin/python -m pytest tests/adapters/test_hermes_adapter.py tests/adapters/test_hermes_config_builder.py tests/adapters/test_hermes_native_config.py tests/adapters/test_hermes_interfaces.py tests/adapters/test_hermes_terminal.py tests/adapters/test_hermes_streaming.py tests/adapters/test_hermes_health.py tests/e2e/test_hermes_e2e.py tests/e2e/test_hermes_checks.py -q # Build the TypeScript contract and adapter packages using their locked dependencies. build-typescript: install-typescript diff --git a/sdk/python/nemo-fabric-adapter-catalog/src/nemo_fabric_adapter_catalog/catalog.json b/sdk/python/nemo-fabric-adapter-catalog/src/nemo_fabric_adapter_catalog/catalog.json index f23440d47..ee36b2836 100644 --- a/sdk/python/nemo-fabric-adapter-catalog/src/nemo_fabric_adapter_catalog/catalog.json +++ b/sdk/python/nemo-fabric-adapter-catalog/src/nemo_fabric_adapter_catalog/catalog.json @@ -842,6 +842,80 @@ "settings_schema": { "$schema": "https://json-schema.org/draft/2020-12/schema", "additionalProperties": false, + "allOf": [ + { + "if": { + "properties": { + "sdk": { + "properties": { + "enabled": { + "const": false + } + }, + "required": [ + "enabled" + ] + } + }, + "required": [ + "sdk" + ] + }, + "then": { + "properties": { + "interfaces": { + "anyOf": [ + { + "properties": { + "api": { + "properties": { + "enabled": { + "const": true + } + } + } + }, + "required": [ + "api" + ] + }, + { + "properties": { + "dashboard": { + "properties": { + "enabled": { + "const": true + } + } + } + }, + "required": [ + "dashboard" + ] + }, + { + "properties": { + "terminal": { + "properties": { + "enabled": { + "const": true + } + } + } + }, + "required": [ + "terminal" + ] + } + ] + } + }, + "required": [ + "interfaces" + ] + } + } + ], "properties": { "interfaces": { "additionalProperties": false, @@ -888,6 +962,26 @@ "port" ], "type": "object" + }, + "terminal": { + "additionalProperties": false, + "description": "Experimental terminal attachment to a managed TUI backend. Uses headless serve without dashboard assets, or shares the dashboard process. The configuration (including terminal.port), interfaces/tui.json, interfaces/tui.token, and the nemo_fabric_adapters.hermes.terminal helper may change across releases as the #579 launcher contract is adopted; compatibility is not guaranteed for these surfaces.", + "properties": { + "enabled": { + "default": true, + "type": "boolean" + }, + "port": { + "description": "Experimental TUI backend loopback port. Must match dashboard.port when both interfaces are enabled. This required field may be removed when the #579 launcher contract is adopted.", + "maximum": 65535, + "minimum": 1, + "type": "integer" + } + }, + "required": [ + "port" + ], + "type": "object" } }, "type": "object" @@ -938,6 +1032,18 @@ "description": "Whether Hermes saves its native conversation trajectories as JSONL.", "type": "boolean" }, + "sdk": { + "additionalProperties": false, + "description": "Embedded SDK execution, enabled by default. Runtime management remains available when disabled.", + "properties": { + "enabled": { + "default": true, + "description": "Initialize the embedded SDK agent during startup. Disable for native-only execution; SDK invocation then fails explicitly.", + "type": "boolean" + } + }, + "type": "object" + }, "startup_timeout_seconds": { "default": 60, "description": "Time from the start of the runtime for every enabled interface to pass its health check, within NeMo Fabric's 90-second start limit.", @@ -972,7 +1078,7 @@ "name": "nemo-fabric-adapters-hermes", "version": "0.7.0" }, - "sha256": "3370179a56673b6261c38653c42f049970eae696f9e88464c1757e8dc15348f0", + "sha256": "6b75b6803c8a4fac6cac5b96aa68b32e7a1b058a4c33f24041d22124b9caae85", "source": "adapters/python/hermes/hermes.fabric-adapter.json" }, "nvidia.fabric.kilo": { diff --git a/skills/nemo-fabric-integrate/BENCHMARK.md b/skills/nemo-fabric-integrate/BENCHMARK.md index 9b2fbb0aa..8111f17e0 100644 --- a/skills/nemo-fabric-integrate/BENCHMARK.md +++ b/skills/nemo-fabric-integrate/BENCHMARK.md @@ -10,8 +10,8 @@ Recommended for publication based on the completed evaluation evidence in this r - Skill: `nemo-fabric-integrate` - Validation status: `passed` -- Evaluation date: 2026-10-29 -- Evaluator version: `1.6.4` +- Evaluation date: 2026-11-09 +- Evaluator version: `1.6.9` - Agents: Claude Code (`aws/anthropic/bedrock-claude-opus-5-5`), Codex (`openai/openai/gpt-6.1-sol`) - Tasks: 6 evaluation tasks (4 positive, 2 negative) - Dataset digest: `sha256:4e12e11bb429ad195c00ceab9cd53bbf6351f32173f03b0ae827531ba2441a9e` (skill-evaluator-dataset-snapshot/1) @@ -28,11 +28,11 @@ SkillSpector provider, model, analyzer revision, and determinism evidence retain | Skill | Provider | Adapter | Analyzer | Model | Revision | Determinism | | --- | --- | --- | --- | --- | --- | --- | -| nemo-fabric-integrate | nv_inference | openai | mcp_tool_poisoning | `aws/anthropic/bedrock-claude-opus-5-5` | `2.13.2` | nondeterministic / provider_defaults | -| nemo-fabric-integrate | nv_inference | openai | semantic_developer_intent | `aws/anthropic/bedrock-claude-opus-5-5` | `2.13.2` | nondeterministic / provider_defaults | -| nemo-fabric-integrate | nv_inference | openai | semantic_quality_policy | `aws/anthropic/bedrock-claude-opus-5-5` | `2.13.2` | nondeterministic / provider_defaults | -| nemo-fabric-integrate | nv_inference | openai | semantic_security_discovery | `aws/anthropic/bedrock-claude-opus-5-5` | `2.13.2` | nondeterministic / provider_defaults | -| nemo-fabric-integrate | nv_inference | openai | meta_analyzer | `aws/anthropic/bedrock-claude-opus-5-5` | `2.13.2` | nondeterministic / provider_defaults | +| nemo-fabric-integrate | nv_inference | openai | mcp_tool_poisoning | `aws/anthropic/bedrock-claude-opus-5-5` | `2.13.4` | nondeterministic / provider_defaults | +| nemo-fabric-integrate | nv_inference | openai | semantic_developer_intent | `aws/anthropic/bedrock-claude-opus-5-5` | `2.13.4` | nondeterministic / provider_defaults | +| nemo-fabric-integrate | nv_inference | openai | semantic_quality_policy | `aws/anthropic/bedrock-claude-opus-5-5` | `2.13.4` | nondeterministic / provider_defaults | +| nemo-fabric-integrate | nv_inference | openai | semantic_security_discovery | `aws/anthropic/bedrock-claude-opus-5-5` | `2.13.4` | nondeterministic / provider_defaults | +| nemo-fabric-integrate | nv_inference | openai | meta_analyzer | `aws/anthropic/bedrock-claude-opus-5-5` | `2.13.4` | nondeterministic / provider_defaults | - Controls for `nemo-fabric-integrate`: temperature=not set (source=provider_default, forwarded=no); seed=not set (source=unset, forwarded=no); reasoning_effort=not set (source=provider_default, forwarded=no) @@ -50,12 +50,12 @@ The three-tier evaluation checks whether the skill: | Measure | Claude Code (Baseline → Skill Uplift) | Codex (Baseline → Skill Uplift) | |---|---:|---:| -| Overall | 71.7% — baseline ran, but no comparable score was available; uplift unavailable | 74.6% — baseline ran, but no comparable score was available; uplift unavailable | -| Security | 100.0% → 66.7% (-33.3 points) | 83.3% → 83.3% (±0.0 points) | -| Correctness | 33.3% → 90.0% (+56.7 points) | 40.0% → 63.3% (+23.3 points) | -| Discoverability | 88.8% — baseline ran, but no comparable score was available; uplift unavailable | 88.8% — baseline ran, but no comparable score was available; uplift unavailable | -| Effectiveness | 33.5% → 35.9% (+2.4 points) | 43.1% → 63.1% (+20.0 points) | -| Efficiency | 77.0% — baseline ran, but no comparable score was available; uplift unavailable | 74.6% — baseline ran, but no comparable score was available; uplift unavailable | +| Overall | 84.7% — baseline ran, but no comparable score was available; uplift unavailable | 81.1% — baseline ran, but no comparable score was available; uplift unavailable | +| Security | 100.0% → 100.0% (±0.0 points) | 100.0% → 91.7% (-8.3 points) | +| Correctness | 50.0% → 90.0% (+40.0 points) | 23.3% → 76.7% (+53.4 points) | +| Discoverability | 90.0% — baseline ran, but no comparable score was available; uplift unavailable | 90.0% — baseline ran, but no comparable score was available; uplift unavailable | +| Effectiveness | 34.7% → 71.0% (+36.3 points) | 30.6% → 69.1% (+38.5 points) | +| Efficiency | 72.7% — baseline ran, but no comparable score was available; uplift unavailable | 77.9% — baseline ran, but no comparable score was available; uplift unavailable | **How to read this table:** baseline is the same task attempted without the target skill. Scores are rounded to one decimal; threshold-adjacent values use additional precision so their displayed band matches the verdict. Uplift is derived from those displayed scores and shown in percentage points. @@ -70,9 +70,9 @@ Shared-source comparisons use the same signals and weights on both conditions. T | Agent | Dimension | Basis | Baseline → With skill (uplift) | Explanation | | --- | --- | --- | --- | --- | | claude-code | Discoverability | Dimension comparison | N/A | Discoverability is not applicable to the no-skill baseline; there is no installed skill to discover. | -| claude-code | Efficiency | Token-only comparison | 88.3% → 84.4% (-3.9 points) | Only Token Efficiency is comparable; the full dimension scores use different evaluator sources. | +| claude-code | Efficiency | Token-only comparison | 88.9% → 82.7% (-6.2 points) | Only Token Efficiency is comparable; the full dimension scores use different evaluator sources. | | codex | Discoverability | Dimension comparison | N/A | Discoverability is not applicable to the no-skill baseline; there is no installed skill to discover. | -| codex | Efficiency | Token-only comparison | 90.1% → 86.1% (-4.0 points) | Only Token Efficiency is comparable; the full dimension scores use different evaluator sources. | +| codex | Efficiency | Token-only comparison | 91.8% → 87.1% (-4.7 points) | Only Token Efficiency is comparable; the full dimension scores use different evaluator sources. | ## Token Usage @@ -80,21 +80,21 @@ Actual Tier 3 execution usage is reported for every observed agent/case pair and | Agent | Dataset case | With skill | Without skill | Delta | Change | Coverage | |---|---|---:|---:|---:|---:|---| -| claude-code | All cases | 4,723,729 | 2,341,658 | +2,382,071 | +101.73% | skill 6/6; base 6/6 | -| claude-code | nemo-fabric-integrate-001-python-service | 172,735 | 263,508 | -90,773 | -34.45% | skill 1/1; base 1/1 | -| claude-code | nemo-fabric-integrate-002-evaluation-platform | 1,445,584 | 144,301 | +1,301,283 | +901.78% | skill 1/1; base 1/1 | -| claude-code | nemo-fabric-integrate-003-adapter-contribution-negative | 359,448 | 144,194 | +215,254 | +149.28% | skill 1/1; base 1/1 | -| claude-code | nemo-fabric-integrate-004-overview-negative | 81,130 | 70,846 | +10,284 | +14.52% | skill 1/1; base 1/1 | -| claude-code | nemo-fabric-integrate-005-streaming-boundary | 546,936 | 181,713 | +365,223 | +200.99% | skill 1/1; base 1/1 | -| claude-code | nemo-fabric-integrate-006-system-instruction-modes | 2,117,896 | 1,537,096 | +580,800 | +37.79% | skill 1/1; base 1/1 | -| codex | All cases | 1,469,746 | 336,141 | +1,133,605 | +337.24% | skill 6/6; base 6/6 | -| codex | nemo-fabric-integrate-001-python-service | 103,401 | 54,999 | +48,402 | +88.01% | skill 1/1; base 1/1 | -| codex | nemo-fabric-integrate-002-evaluation-platform | 1,112,277 | 53,702 | +1,058,575 | +1971.20% | skill 1/1; base 1/1 | -| codex | nemo-fabric-integrate-003-adapter-contribution-negative | 57,523 | 40,960 | +16,563 | +40.44% | skill 1/1; base 1/1 | -| codex | nemo-fabric-integrate-004-overview-negative | 35,356 | 21,644 | +13,712 | +63.35% | skill 1/1; base 1/1 | -| codex | nemo-fabric-integrate-005-streaming-boundary | 65,253 | 83,151 | -17,898 | -21.52% | skill 1/1; base 1/1 | -| codex | nemo-fabric-integrate-006-system-instruction-modes | 95,936 | 81,685 | +14,251 | +17.45% | skill 1/1; base 1/1 | -| ALL AGENTS | Dataset aggregate | 6,193,475 | 2,677,799 | +3,515,676 | +131.29% | skill 12/12; base 12/12 | +| claude-code | All cases | 4,971,446 | 3,061,512 | +1,909,934 | +62.39% | skill 6/6; base 6/6 | +| claude-code | nemo-fabric-integrate-001-python-service | 2,125,082 | 185,699 | +1,939,383 | +1044.37% | skill 1/1; base 1/1 | +| claude-code | nemo-fabric-integrate-002-evaluation-platform | 1,795,901 | 990,792 | +805,109 | +81.26% | skill 1/1; base 1/1 | +| claude-code | nemo-fabric-integrate-003-adapter-contribution-negative | 170,494 | 143,890 | +26,604 | +18.49% | skill 1/1; base 1/1 | +| claude-code | nemo-fabric-integrate-004-overview-negative | 81,684 | 70,790 | +10,894 | +15.39% | skill 1/1; base 1/1 | +| claude-code | nemo-fabric-integrate-005-streaming-boundary | 438,619 | 181,633 | +256,986 | +141.49% | skill 1/1; base 1/1 | +| claude-code | nemo-fabric-integrate-006-system-instruction-modes | 359,666 | 1,488,708 | -1,129,042 | -75.84% | skill 1/1; base 1/1 | +| codex | All cases | 1,404,040 | 337,630 | +1,066,410 | +315.85% | skill 6/6; base 6/6 | +| codex | nemo-fabric-integrate-001-python-service | 107,526 | 55,016 | +52,510 | +95.44% | skill 1/1; base 1/1 | +| codex | nemo-fabric-integrate-002-evaluation-platform | 1,053,130 | 92,700 | +960,430 | +1036.06% | skill 1/1; base 1/1 | +| codex | nemo-fabric-integrate-003-adapter-contribution-negative | 57,465 | 54,610 | +2,855 | +5.23% | skill 1/1; base 1/1 | +| codex | nemo-fabric-integrate-004-overview-negative | 36,591 | 26,109 | +10,482 | +40.15% | skill 1/1; base 1/1 | +| codex | nemo-fabric-integrate-005-streaming-boundary | 69,949 | 53,874 | +16,075 | +29.84% | skill 1/1; base 1/1 | +| codex | nemo-fabric-integrate-006-system-instruction-modes | 79,379 | 55,321 | +24,058 | +43.49% | skill 1/1; base 1/1 | +| ALL AGENTS | Dataset aggregate | 6,375,486 | 3,399,142 | +2,976,344 | +87.56% | skill 12/12; base 12/12 | Prompt tokens include cached reads, so total tokens are `prompt + completion` (cached is not added twice). The Efficiency score uses `(prompt - cached) + completion`. Change is `100 * (with - without) / without`; negative means fewer tokens. N/A comparisons include their reason in the coverage column; coverage is never estimated. @@ -103,7 +103,7 @@ Prompt tokens include cached reads, so total tokens are `prompt + completion` (c | Tier | Purpose | Status | Evidence | |---|---|---|---| | Tier 1 | Static validation | **PASSED WITH OBSERVATIONS** | 11 validator(s); 12 finding(s) | -| Tier 2 | Semantic deduplication | **PASSED WITH OBSERVATIONS** | 2 validator(s); 1 finding(s) | +| Tier 2 | Semantic deduplication | **PASSED** | 2 validator(s); 0 finding(s) | | Tier 3 | Live agent evaluation | **PASS** | 2 agent(s); 6 task(s) | ## Findings and Observations @@ -111,12 +111,12 @@ Prompt tokens include cached reads, so total tokens are `prompt + completion` (c
Show detailed findings and successful checks -- **CRITICAL** CONTENT_DEDUP/llm_error: LLM analysis failed for a content cluster (`skills/nemo-fabric-integrate`) - **MEDIUM** QUALITY/quality_correctness: SKILL_SPEC recommended field missing: 'metadata.tags' (`skills/nemo-fabric-integrate/SKILL.md`) - **MEDIUM** QUALITY/quality_reliability: MCP skill lacks connection/error guidance (`skills/nemo-fabric-integrate/SKILL.md`) -- **MEDIUM** QUALITY/quality_efficiency: Large skill (6600 tokens, recommended max <5000). Per agentskills.io, SKILL.md should be concise (~500 lines) — large skill bodies increase token cost after invocation; long or unfocused top-level descriptions can degrade agent routing accuracy (`skills/nemo-fabric-integrate/SKILL.md`) +- **MEDIUM** QUALITY/quality_efficiency: Large skill (6761 tokens, recommended max <5000). Per agentskills.io, SKILL.md should be concise (~500 lines) — large skill bodies increase token cost after invocation; long or unfocused top-level descriptions can degrade agent routing accuracy (`skills/nemo-fabric-integrate/SKILL.md`) - **MEDIUM** SCHEMA/body_recommended_section: Missing recommended section: '## Instructions' (`skills/nemo-fabric-integrate/SKILL.md`) -- 8 additional finding(s) are available in the full evaluation artifacts. +- **MEDIUM** SCHEMA/body_recommended_section: Missing recommended section: '## Examples' (`skills/nemo-fabric-integrate/SKILL.md`) +- 7 additional finding(s) are available in the full evaluation artifacts.
diff --git a/skills/nemo-fabric-integrate/SKILL.md b/skills/nemo-fabric-integrate/SKILL.md index 53d2e3060..d25b951df 100644 --- a/skills/nemo-fabric-integrate/SKILL.md +++ b/skills/nemo-fabric-integrate/SKILL.md @@ -50,7 +50,7 @@ runtime assumptions but never installs harnesses or credentials at run time. `nemo-fabric[claude]`, `nemo-fabric[codex]`, or `nemo-fabric[deepagents]`. - Install Hermes Agent separately from the Fabric adapter. Follow the - [Hermes integration guide](https://github.com/NVIDIA/NeMo-Fabric/blob/main/docs/integrations/harness/hermes.mdx) + [Hermes integration guide](https://github.com/NVIDIA/NeMo-Fabric/blob/main/docs/integrations/harness/hermes/hermes.mdx) for the current compatible interpreter, source checkout, Relay dependencies, and adapter installation. The `nemo-fabric[hermes-agent]` extra does not install Hermes Agent. @@ -173,6 +173,8 @@ construction. ## Choose A Lifecycle +For Hermes, embedded SDK execution is enabled and eager by default. Set `harness.settings.sdk.enabled=False` only for native-only runtimes with an enabled API, dashboard, or terminal interface; management through the SDK remains available, but invocation fails explicitly. Browser and terminal clients share one managed TUI backend. Native responses remain native even when Relay emits standard ATOF/ATIF; do not treat them as SDK `RunResult` objects. Follow the [Hermes invocation guides](https://github.com/NVIDIA/NeMo-Fabric/blob/main/docs/integrations/harness/hermes/hermes.mdx) for interface-specific installation and attachment. + For Deep Agents, mini-SWE-agent, and the LangGraph custom-agent example, pass the same UUID string through `RunRequest.relay_session_root` on each conversation turn to group Relay trajectories under one session. Core forwards the typed field as `AgentRunRequest.relay_session_root`; context keys do not control Relay propagation. An unusable UUID preserves per-request behavior. The remaining adapters do not consume this field. Pick the smallest lifecycle the consumer needs: diff --git a/skills/nemo-fabric-integrate/skill-card.md b/skills/nemo-fabric-integrate/skill-card.md index 41c00c446..2d35465e9 100644 --- a/skills/nemo-fabric-integrate/skill-card.md +++ b/skills/nemo-fabric-integrate/skill-card.md @@ -9,7 +9,7 @@ NVIDIA
### License/Terms of Use:
Apache 2.0
## Use Case:
-Developers and engineers use this skill to integrate NVIDIA NeMo Fabric into their own applications, services, evaluation harnesses, or platforms through the public Python SDK, mapping their configuration to a typed FabricConfig, selecting a runtime lifecycle, validating with plan and doctor, and consuming normalized results, artifacts, and telemetry.
+Developers and engineers use this skill to integrate NVIDIA NeMo Fabric into their own applications, services, evaluation harnesses, or platforms through the public typed Python SDK, mapping consumer configuration into an in-memory FabricConfig, selecting a runtime lifecycle, validating with plan and doctor, and handling normalized results, artifacts, telemetry, and errors.
### Deployment Geography for Use:
Global
@@ -25,23 +25,19 @@ Risk: Review before execution as proposals could introduce incorrect or misleadi Mitigation: Review and scan skill before deployment.
## Reference(s):
-- [Mapping Consumer Config To FabricConfig](references/config-mapping.md)
-- [Results, Evidence, And Errors](references/results-and-errors.md)
-- [SDK API Inventory](references/sdk-api-inventory.md)
-- [NeMo Fabric Python SDK](https://github.com/NVIDIA/NeMo-Fabric/blob/main/docs/sdk/python.mdx)
-- [NeMo Fabric Installation Guide](https://github.com/NVIDIA/NeMo-Fabric/blob/main/docs/getting-started/install.mdx)
-- [NeMo Fabric Overview](https://github.com/NVIDIA/NeMo-Fabric/blob/main/docs/about-nemo-fabric/overview.mdx)
-- [Hermes Integration Guide](https://github.com/NVIDIA/NeMo-Fabric/blob/main/docs/integrations/harness/hermes.mdx)
-- [System Stats](https://github.com/NVIDIA/NeMo-Fabric/blob/main/docs/sdk/system-stats.mdx)
-- [nemo_fabric.client API Reference](https://github.com/NVIDIA/NeMo-Fabric/blob/main/docs/reference/api/python-library-reference/nemo_fabric.client.md)
-- [nemo_fabric.runtime API Reference](https://github.com/NVIDIA/NeMo-Fabric/blob/main/docs/reference/api/python-library-reference/nemo_fabric.runtime.md)
-- [NeMo Fabric Adapter Catalog](https://github.com/NVIDIA/NeMo-Fabric/blob/main/sdk/python/nemo-fabric-adapter-catalog/README.md)
-- [code_review_agent Example](https://github.com/NVIDIA/NeMo-Fabric/tree/main/examples/code_review_agent)
+- [config-mapping.md](references/config-mapping.md)
+- [results-and-errors.md](references/results-and-errors.md)
+- [sdk-api-inventory.md](references/sdk-api-inventory.md)
+- [NeMo Fabric installation guide](https://github.com/NVIDIA/NeMo-Fabric/blob/main/docs/getting-started/install.mdx)
+- [NeMo Fabric Python SDK guide](https://github.com/NVIDIA/NeMo-Fabric/blob/main/docs/sdk/python.mdx)
+- [Hermes integration guide](https://github.com/NVIDIA/NeMo-Fabric/blob/main/docs/integrations/harness/hermes/hermes.mdx)
+- [NeMo Fabric adapter catalog guide](https://github.com/NVIDIA/NeMo-Fabric/blob/main/sdk/python/nemo-fabric-adapter-catalog/README.md)
+- [code_review_agent example](https://github.com/NVIDIA/NeMo-Fabric/tree/main/examples/code_review_agent)
## Skill Output:
-**Output Type(s):** [Code, Configuration instructions, Shell commands]
-**Output Format:** [Markdown with inline Python and shell code blocks]
+**Output Type(s):** [Code, Configuration instructions, Analysis]
+**Output Format:** [Markdown with inline Python code blocks]
**Output Parameters:** [1D]
**Other Properties Related to Output:** [None]
@@ -52,39 +48,72 @@ Mitigation: Review and scan skill before deployment.
## Evaluation Tasks:
-6 evaluation tasks (4 positive, 2 negative), 1 attempt per task, each run in an isolated sandbox pod in a `k8s-sandbox` environment; evaluator version `1.6.4`, evaluated 2026-10-29. Overall verdict: PASS — recommended for publication.
+6 evaluation tasks (4 positive, 2 negative), 1 attempt per task, each attempt run in its own isolated sandbox pod (`k8s-sandbox` environment); each task was also attempted without the skill as a baseline. Evaluator version `1.6.9`, evaluation date 2026-11-09.
## Evaluation Metrics Used:
Reported benchmark dimensions:
-- Security: Is it safe to use? Scored from the `security` signal (100%).
-- Correctness: Is the answer correct? Scored from the `accuracy` signal (100%).
-- Discoverability: Was the right skill loaded when needed? Scored from the `skill_execution` signal (100%).
-- Effectiveness: Did the skill help complete the task? Equal-weight mean of `goal_accuracy` and `behavior_check`.
-- Efficiency: Did it avoid wasted tool calls and token usage? 50% `skill_efficiency` (tool-call productivity) and 50% `token_efficiency`.
+- Security: Is it safe to use? Scored from `security` (100%).
+- Correctness: Is the answer correct? Scored from `accuracy` (100%).
+- Discoverability: Was the right skill loaded when needed? Scored from `skill_execution` (100%).
+- Effectiveness: Did the skill help complete the task? Scored from `goal_accuracy` (50%) + `behavior_check` (50%).
+- Efficiency: Did it avoid wasted tool calls and token usage? Scored from `skill_efficiency` (50%) + `token_efficiency` (50%).
Underlying evaluation signals used in this run:
-- `security`: Unsafe operations, secret leakage, and unauthorized access.
-- `skill_execution`: Whether the expected skill was selected, decoys were avoided, and the workflow executed.
-- `skill_efficiency`: Tool-call productivity (legacy wire id; routing is scored under Discoverability).
-- `accuracy`: Final-answer correctness against the reference answer.
-- `goal_accuracy`: Whether the user's goal was achieved.
-- `behavior_check`: Whether the expected workflow behavior was followed.
-- `token_efficiency`: Actual uncached prompt plus completion usage (50% of Efficiency).
+- `security`: Security: unsafe operations, secret leakage, and unauthorized access.
+- `skill_execution`: Skill Execution: whether the expected skill was selected, decoys were avoided, and the workflow executed.
+- `skill_efficiency`: Tool Productivity: tool-call productivity (legacy wire id; routing is scored under Discoverability).
+- `accuracy`: Accuracy: final-answer correctness against the reference answer.
+- `goal_accuracy`: Goal Accuracy: whether the user's goal was achieved.
+- `behavior_check`: Behavior Check: whether the expected workflow behavior was followed.
+- `token_efficiency`: Token Efficiency: actual uncached prompt plus completion usage (50% of Efficiency).
## Evaluation Results:
+> ✅ **Overall verdict: PASS — Recommended for publication** + +### Publication Recommendation + +Recommended for publication based on the completed evaluation evidence in this report. + +### Results at a Glance + | Measure | Claude Code (Baseline → Skill Uplift) | Codex (Baseline → Skill Uplift) | |---|---:|---:| -| Overall | 71.7% — baseline ran, but no comparable score was available; uplift unavailable | 74.6% — baseline ran, but no comparable score was available; uplift unavailable | -| Security | 100.0% → 66.7% (-33.3 points) | 83.3% → 83.3% (±0.0 points) | -| Correctness | 33.3% → 90.0% (+56.7 points) | 40.0% → 63.3% (+23.3 points) | -| Discoverability | 88.8% — baseline ran, but no comparable score was available; uplift unavailable | 88.8% — baseline ran, but no comparable score was available; uplift unavailable | -| Effectiveness | 33.5% → 35.9% (+2.4 points) | 43.1% → 63.1% (+20.0 points) | -| Efficiency | 77.0% — baseline ran, but no comparable score was available; uplift unavailable | 74.6% — baseline ran, but no comparable score was available; uplift unavailable | +| Overall | 84.7% — baseline ran, but no comparable score was available; uplift unavailable | 81.1% — baseline ran, but no comparable score was available; uplift unavailable | +| Security | 100.0% → 100.0% (±0.0 points) | 100.0% → 91.7% (-8.3 points) | +| Correctness | 50.0% → 90.0% (+40.0 points) | 23.3% → 76.7% (+53.4 points) | +| Discoverability | 90.0% — baseline ran, but no comparable score was available; uplift unavailable | 90.0% — baseline ran, but no comparable score was available; uplift unavailable | +| Effectiveness | 34.7% → 71.0% (+36.3 points) | 30.6% → 69.1% (+38.5 points) | +| Efficiency | 72.7% — baseline ran, but no comparable score was available; uplift unavailable | 77.9% — baseline ran, but no comparable score was available; uplift unavailable | + +**How to read this table:** baseline is the same task attempted without the target skill. Scores are rounded to one decimal; threshold-adjacent values use additional precision so their displayed band matches the verdict. Uplift is derived from those displayed scores and shown in percentage points. + +Example: `47.0% → 92.0% (+45.0 points)` means the skill-assisted run scored 92.0%, 45.0 percentage points above its 47.0% no-skill baseline. + +A partial dimension was calculated from only the available configured signals; review the detailed report before relying on it. + +#### Comparison details + +Shared-source comparisons use the same signals and weights on both conditions. They do not replace the full dimension scores above or change the publication verdict. + +| Agent | Dimension | Basis | Baseline → With skill (uplift) | Explanation | +| --- | --- | --- | --- | --- | +| claude-code | Discoverability | Dimension comparison | N/A | Discoverability is not applicable to the no-skill baseline; there is no installed skill to discover. | +| claude-code | Efficiency | Token-only comparison | 88.9% → 82.7% (-6.2 points) | Only Token Efficiency is comparable; the full dimension scores use different evaluator sources. | +| codex | Discoverability | Dimension comparison | N/A | Discoverability is not applicable to the no-skill baseline; there is no installed skill to discover. | +| codex | Efficiency | Token-only comparison | 91.8% → 87.1% (-4.7 points) | Only Token Efficiency is comparable; the full dimension scores use different evaluator sources. | + +### Tier Status + +| Tier | Purpose | Status | Evidence | +|---|---|---|---| +| Tier 1 | Static validation | **PASSED WITH OBSERVATIONS** | 11 validator(s); 12 finding(s) | +| Tier 2 | Semantic deduplication | **PASSED** | 2 validator(s); 0 finding(s) | +| Tier 3 | Live agent evaluation | **PASS** | 2 agent(s); 6 task(s) | ## Skill Version(s):
-0.7.0 (source: Cargo.toml workspace version; pyproject.toml reports placeholder 0.0.0; git SHA 886e692, committed 2026-10-29)
+0.0.0 (source: pyproject.toml; git SHA c5d9a5a, committed 2026-11-08)
## Ethical Considerations:
NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal team to ensure this skill meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
diff --git a/skills/nemo-fabric-integrate/skill.oms.sig b/skills/nemo-fabric-integrate/skill.oms.sig index 18877c5e2..7ce58b5b4 100644 --- a/skills/nemo-fabric-integrate/skill.oms.sig +++ b/skills/nemo-fabric-integrate/skill.oms.sig @@ -1 +1 @@ -{"mediaType":"application/vnd.dev.sigstore.bundle.v0.3+json","verificationMaterial":{"x509CertificateChain":{"certificates":[{"rawBytes":"MIICgzCCAgmgAwIBAgIUKIyS7SxNteQIiWzK1dWj85E6520wCgYIKoZIzj0EAwMwVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwHhcNMjYwNDAxMDAwMDAwWhcNMjgwNDIyMTUzMzA5WjBUMQswCQYDVQQGEwJVUzEbMBkGA1UECgwSTlZJRElBIENvcnBvcmF0aW9uMSgwJgYDVQQDDB9OVklESUEgQWdlbnQgU2tpbGxzIFNpZ25pbmcgMDAxMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAEYoRM9bQl/dGlwSRNi6bTpIJUXH8Nv9GciP6LSflJYYMLCc296kpyuTSsk5ddbAWiDcFX3C/ydX3jwc+qCLYP6uHy9XphyLjOQ27Yb2J6rBLVtRBS1mgGco/Gr7fL6ODco4GaMIGXMB0GA1UdDgQWBBRQ/5ZW3nJ6lmo9SVk7I15o7UGmpTAfBgNVHSMEGDAWgBRPGpILxMBBleJSsBGjrMKsby1CgjAMBgNVHRMBAf8EAjAAMA4GA1UdDwEB/wQEAwIHgDA3BggrBgEFBQcBAQQrMCkwJwYIKwYBBQUHMAGGG2h0dHA6Ly9vY3NwLm5kaXMubnZpZGlhLmNvbTAKBggqhkjOPQQDAwNoADBlAjAUygu/GiOCIXrgGr4SmLgeEVDcEitfFUv7ALbvLVGVyMysB3mxmO/uInZfXzWcJZsCMQDxuoxj4ZmO30jhkPIcCxGFCOvnUsnfU3TfGcouYm4M6iRpbKvtVnHPiy4bi6pcKf0="},{"rawBytes":"MIICiDCCAg6gAwIBAgIUZsIuSv9NkpJCNqtYEfCouVv5BzowCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAASI72cR3ctKGg4VWnB3bNja6g1Z2PnOmFEopkPof+QeIcPk9rT+g9MjJnq51EQXL93a7C2GJ9J985G4o2V85VD7wJ1RaXhluHW2rf3y8bQGeAYaKMr5s/hUgn+M3/9WlWejgaAwgZ0wHQYDVR0OBBYEFE8akgvEwEGV4lKwEaOswqxvLUKCMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgEGMDcGCCsGAQUFBwEBBCswKTAnBggrBgEFBQcwAYYbaHR0cDovL29jc3AubmRpcy5udmlkaWEuY29tMAoGCCqGSM49BAMDA2gAMGUCMQCeIMMfAbyzPDacw2MxG+Yt1cikrJX/DVxiGfXuHmkkXn6VgSzE79+lkqDErpVO2gYCMCNEColOyvUvkzZGUEI1hQ3PfMgi3FIo9tHoBKMw4/wGBLFpu/0ubtmbBXM6/UMOEw=="},{"rawBytes":"MIICRTCCAcygAwIBAgIUeJdY3rV86EdvFmG7L8LJBsyQFYkwCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTB2MBAGByqGSM49AgEGBSuBBAAiA2IABAYpiXCDjJ9NT2eSDhyHJVSw1Tbze18cGG2F/578oWvHxg23eQAhNRYdq88i1iOshZSO6C29doKui5Xpmo/7Ctw9Sx4PP2RzOmIuOLCuTdNtKcTRwi4GEsd5BAFvWj42M6NjMGEwHQYDVR0OBBYEFItnoAjjfuCEUvzyvWyI2vOGvwPjMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEGMAoGCCqGSM49BAMDA2cAMGQCMCwtAjWLaNwgGWNCgdyNoTyvNhqWRECRJV2r3+7w8g0PL6NHLOsbkgE09BH95h8XlgIwTaQmbbUh2ChAJ5TA1wRiVDnCcvbzHlZl2jM2FcwQQZlk19LOAbyGMRixbu2Ww/rj"}]},"tlogEntries":[]},"dsseEnvelope":{"payload":"ewogICJfdHlwZSI6ICJodHRwczovL2luLXRvdG8uaW8vU3RhdGVtZW50L3YxIiwKICAic3ViamVjdCI6IFsKICAgIHsKICAgICAgIm5hbWUiOiAibmVtby1mYWJyaWMtaW50ZWdyYXRlIiwKICAgICAgImRpZ2VzdCI6IHsKICAgICAgICAic2hhMjU2IjogImMwZWNkMzViMmJlMWMyOGE3NzhlNjBmZWIyZWMwZmI5ZDAyYjkzOGI5YjNkODk1ZjY5ZGZhNzVjOTI2ODIyZDMiCiAgICAgIH0KICAgIH0KICBdLAogICJwcmVkaWNhdGVUeXBlIjogImh0dHBzOi8vbW9kZWxfc2lnbmluZy9zaWduYXR1cmUvdjEuMCIsCiAgInByZWRpY2F0ZSI6IHsKICAgICJzZXJpYWxpemF0aW9uIjogewogICAgICAiYWxsb3dfc3ltbGlua3MiOiBmYWxzZSwKICAgICAgIm1ldGhvZCI6ICJmaWxlcyIsCiAgICAgICJoYXNoX3R5cGUiOiAic2hhMjU2IiwKICAgICAgImlnbm9yZV9wYXRocyI6IFsKICAgICAgICAiLmdpdGh1YiIsCiAgICAgICAgIi5naXQiLAogICAgICAgICIuZ2l0YXR0cmlidXRlcyIsCiAgICAgICAgIi5naXRpZ25vcmUiCiAgICAgIF0KICAgIH0sCiAgICAicmVzb3VyY2VzIjogWwogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiZWE3MDM0ZWM3MDRiNTNkZWVjOWFmNDU4MDMzYWY2MTAxMWQ3NDVjMGUzYmJkMDBhOTE5ZmJkOWQ2YzkxMzcwZCIsCiAgICAgICAgIm5hbWUiOiAiQkVOQ0hNQVJLLm1kIgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiZjA3NDk1OTgzMjc0MjY5ZTAxMzhiYTJkNzU5N2IxNGU0YmQ4YzAzMjYwOTVhYmI1MjAyNTU1ZjZjNmUyYjY3ZiIsCiAgICAgICAgIm5hbWUiOiAiU0tJTEwubWQiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgImRpZ2VzdCI6ICI4MzM0OWJhMGY2YzYyNDhiY2YzMTBlMzAyMmM3OTc2YzllNzVkMmY4OWM1NjdhZjBiYTBlYjU1YzBiMGJmYWRjIiwKICAgICAgICAibmFtZSI6ICJldmFscy9ldmFscy5qc29uIgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiZWQ0ODIwODNjMzA5OGZkM2E2NDdkZWNmMmVjZDNkYjk3NjBjNTFlOTQxNzM2MmIwNWUzNGM3ZTExM2Q2ZDk1ZiIsCiAgICAgICAgIm5hbWUiOiAicmVmZXJlbmNlcy9jb25maWctbWFwcGluZy5tZCIKICAgICAgfSwKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAiZGlnZXN0IjogIjM3MzNiNThjMTMwMDZhMDA4ZDk2YjdhNDY1Yjg3ODQ1Zjg2YmJiMjcwMDlmMzkzMzk0ZWJlNWNhMjZlZDY1NTMiLAogICAgICAgICJuYW1lIjogInJlZmVyZW5jZXMvcmVzdWx0cy1hbmQtZXJyb3JzLm1kIgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiOTRhZTFiMWFiMjZhMGFiYTQ1YjViYzE3M2IyZDNhZWQwNTU4ZDdmYTU1MTVkNmMzN2VkMjQ4ODY4NjUyM2RjMyIsCiAgICAgICAgIm5hbWUiOiAicmVmZXJlbmNlcy9zZGstYXBpLWludmVudG9yeS5tZCIKICAgICAgfSwKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAiZGlnZXN0IjogImMxNzc2YTI2ZTA1MWVkNzc4ZDIwNDgxNzE2ZWRjZThkZDMxZmE1YzIwYmIyYjI4Mjg5MmUyNjY5ZDk2Y2RmOTAiLAogICAgICAgICJuYW1lIjogInNraWxsLWNhcmQubWQiCiAgICAgIH0KICAgIF0KICB9Cn0=","payloadType":"application/vnd.in-toto+json","signatures":[{"sig":"MGUCMBQ+8DWpNnFkoGrCj28lYdIf7GE1FaR2gddfGb0hBPVtRiafDn/tcfkliao4p4n5eAIxAKnMVBHDCFtusLjv/aOJGxC9qUxmHIZxYU/G0xOij58yTHDaNOmHyan1glCtH9SfxA==","keyid":""}]}} \ No newline at end of file +{"mediaType":"application/vnd.dev.sigstore.bundle.v0.3+json","verificationMaterial":{"x509CertificateChain":{"certificates":[{"rawBytes":"MIICgzCCAgmgAwIBAgIUKIyS7SxNteQIiWzK1dWj85E6520wCgYIKoZIzj0EAwMwVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwHhcNMjYwNDAxMDAwMDAwWhcNMjgwNDIyMTUzMzA5WjBUMQswCQYDVQQGEwJVUzEbMBkGA1UECgwSTlZJRElBIENvcnBvcmF0aW9uMSgwJgYDVQQDDB9OVklESUEgQWdlbnQgU2tpbGxzIFNpZ25pbmcgMDAxMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAEYoRM9bQl/dGlwSRNi6bTpIJUXH8Nv9GciP6LSflJYYMLCc296kpyuTSsk5ddbAWiDcFX3C/ydX3jwc+qCLYP6uHy9XphyLjOQ27Yb2J6rBLVtRBS1mgGco/Gr7fL6ODco4GaMIGXMB0GA1UdDgQWBBRQ/5ZW3nJ6lmo9SVk7I15o7UGmpTAfBgNVHSMEGDAWgBRPGpILxMBBleJSsBGjrMKsby1CgjAMBgNVHRMBAf8EAjAAMA4GA1UdDwEB/wQEAwIHgDA3BggrBgEFBQcBAQQrMCkwJwYIKwYBBQUHMAGGG2h0dHA6Ly9vY3NwLm5kaXMubnZpZGlhLmNvbTAKBggqhkjOPQQDAwNoADBlAjAUygu/GiOCIXrgGr4SmLgeEVDcEitfFUv7ALbvLVGVyMysB3mxmO/uInZfXzWcJZsCMQDxuoxj4ZmO30jhkPIcCxGFCOvnUsnfU3TfGcouYm4M6iRpbKvtVnHPiy4bi6pcKf0="},{"rawBytes":"MIICiDCCAg6gAwIBAgIUZsIuSv9NkpJCNqtYEfCouVv5BzowCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAASI72cR3ctKGg4VWnB3bNja6g1Z2PnOmFEopkPof+QeIcPk9rT+g9MjJnq51EQXL93a7C2GJ9J985G4o2V85VD7wJ1RaXhluHW2rf3y8bQGeAYaKMr5s/hUgn+M3/9WlWejgaAwgZ0wHQYDVR0OBBYEFE8akgvEwEGV4lKwEaOswqxvLUKCMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgEGMDcGCCsGAQUFBwEBBCswKTAnBggrBgEFBQcwAYYbaHR0cDovL29jc3AubmRpcy5udmlkaWEuY29tMAoGCCqGSM49BAMDA2gAMGUCMQCeIMMfAbyzPDacw2MxG+Yt1cikrJX/DVxiGfXuHmkkXn6VgSzE79+lkqDErpVO2gYCMCNEColOyvUvkzZGUEI1hQ3PfMgi3FIo9tHoBKMw4/wGBLFpu/0ubtmbBXM6/UMOEw=="},{"rawBytes":"MIICRTCCAcygAwIBAgIUeJdY3rV86EdvFmG7L8LJBsyQFYkwCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTB2MBAGByqGSM49AgEGBSuBBAAiA2IABAYpiXCDjJ9NT2eSDhyHJVSw1Tbze18cGG2F/578oWvHxg23eQAhNRYdq88i1iOshZSO6C29doKui5Xpmo/7Ctw9Sx4PP2RzOmIuOLCuTdNtKcTRwi4GEsd5BAFvWj42M6NjMGEwHQYDVR0OBBYEFItnoAjjfuCEUvzyvWyI2vOGvwPjMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEGMAoGCCqGSM49BAMDA2cAMGQCMCwtAjWLaNwgGWNCgdyNoTyvNhqWRECRJV2r3+7w8g0PL6NHLOsbkgE09BH95h8XlgIwTaQmbbUh2ChAJ5TA1wRiVDnCcvbzHlZl2jM2FcwQQZlk19LOAbyGMRixbu2Ww/rj"}]},"tlogEntries":[]},"dsseEnvelope":{"payload":"ewogICJfdHlwZSI6ICJodHRwczovL2luLXRvdG8uaW8vU3RhdGVtZW50L3YxIiwKICAic3ViamVjdCI6IFsKICAgIHsKICAgICAgIm5hbWUiOiAibmVtby1mYWJyaWMtaW50ZWdyYXRlIiwKICAgICAgImRpZ2VzdCI6IHsKICAgICAgICAic2hhMjU2IjogImYzZTEzZDgwNGZlZmEzZDJhYjI5ODFmYTMxZDgzM2Y5MTgzODE0ZTBlMThlNWFmN2FmMWFmYTNmMGFjZjk1YTQiCiAgICAgIH0KICAgIH0KICBdLAogICJwcmVkaWNhdGVUeXBlIjogImh0dHBzOi8vbW9kZWxfc2lnbmluZy9zaWduYXR1cmUvdjEuMCIsCiAgInByZWRpY2F0ZSI6IHsKICAgICJzZXJpYWxpemF0aW9uIjogewogICAgICAiYWxsb3dfc3ltbGlua3MiOiBmYWxzZSwKICAgICAgImlnbm9yZV9wYXRocyI6IFsKICAgICAgICAiLmdpdGF0dHJpYnV0ZXMiLAogICAgICAgICIuZ2l0IiwKICAgICAgICAiLmdpdGh1YiIsCiAgICAgICAgIi5naXRpZ25vcmUiCiAgICAgIF0sCiAgICAgICJoYXNoX3R5cGUiOiAic2hhMjU2IiwKICAgICAgIm1ldGhvZCI6ICJmaWxlcyIKICAgIH0sCiAgICAicmVzb3VyY2VzIjogWwogICAgICB7CiAgICAgICAgIm5hbWUiOiAiQkVOQ0hNQVJLLm1kIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgImRpZ2VzdCI6ICJiY2Q3YTRiOTA0MmZhNzI0MjdjZWVjNzAyMDcxNTlhYzkzNDEwNTRiODU2Yjc1M2U2MjY2ZTUwYzc2NTIwZmE2IgogICAgICB9LAogICAgICB7CiAgICAgICAgIm5hbWUiOiAiU0tJTEwubWQiLAogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAiZGlnZXN0IjogImQ1ZDcxZTIxYjc3OGRmZWNjZDdmOGFhMTE5ZmJhNjJhZDUyYzFhYWM5ZTk2MGMwZjExNGFiNGViOWZhMTY2YWUiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAibmFtZSI6ICJldmFscy9ldmFscy5qc29uIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgImRpZ2VzdCI6ICI4MzM0OWJhMGY2YzYyNDhiY2YzMTBlMzAyMmM3OTc2YzllNzVkMmY4OWM1NjdhZjBiYTBlYjU1YzBiMGJmYWRjIgogICAgICB9LAogICAgICB7CiAgICAgICAgIm5hbWUiOiAicmVmZXJlbmNlcy9jb25maWctbWFwcGluZy5tZCIsCiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiZWQ0ODIwODNjMzA5OGZkM2E2NDdkZWNmMmVjZDNkYjk3NjBjNTFlOTQxNzM2MmIwNWUzNGM3ZTExM2Q2ZDk1ZiIKICAgICAgfSwKICAgICAgewogICAgICAgICJuYW1lIjogInJlZmVyZW5jZXMvcmVzdWx0cy1hbmQtZXJyb3JzLm1kIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgImRpZ2VzdCI6ICIzNzMzYjU4YzEzMDA2YTAwOGQ5NmI3YTQ2NWI4Nzg0NWY4NmJiYjI3MDA5ZjM5MzM5NGViZTVjYTI2ZWQ2NTUzIgogICAgICB9LAogICAgICB7CiAgICAgICAgIm5hbWUiOiAicmVmZXJlbmNlcy9zZGstYXBpLWludmVudG9yeS5tZCIsCiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiOTRhZTFiMWFiMjZhMGFiYTQ1YjViYzE3M2IyZDNhZWQwNTU4ZDdmYTU1MTVkNmMzN2VkMjQ4ODY4NjUyM2RjMyIKICAgICAgfSwKICAgICAgewogICAgICAgICJuYW1lIjogInNraWxsLWNhcmQubWQiLAogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAiZGlnZXN0IjogIjE1NzA1OGUxZjYzMTQzNWYyZjIxOTQ3YWZkY2ExZWNmYTEzMzlhYWRjN2NjOGUwMzVmZWI0ODU3NzkxZTExNjAiCiAgICAgIH0KICAgIF0KICB9Cn0=","payloadType":"application/vnd.in-toto+json","signatures":[{"sig":"MGUCMAzbCslFOInAL9V6wFfOMthxOWvlSQCk1FT2mfeJZg5xcKV80pAsPYy5u1STBBOL1QIxAPlPLZTW575CK0jxkN6min38q08NH3cmnEn9tSOrmzd5naixDla62v37XTDGsTj4YQ==","keyid":""}]}} \ No newline at end of file diff --git a/tests/adapters/test_hermes_health.py b/tests/adapters/test_hermes_health.py index 35d366d7f..ec871238d 100644 --- a/tests/adapters/test_hermes_health.py +++ b/tests/adapters/test_hermes_health.py @@ -5,7 +5,7 @@ import asyncio import threading -from unittest.mock import MagicMock +from unittest.mock import AsyncMock, MagicMock import pytest from nemo_fabric_adapter_contract.models import AdapterCheckResult, AgentConfig @@ -73,6 +73,28 @@ async def test_hermes_health_uses_fresh_native_diagnostics(hermes_health, level) assert runtime._conversation_history is None +@pytest.mark.parametrize("level", ["live", "active", "ready"]) +@pytest.mark.parametrize("busy", [False, True]) +async def test_mixed_runtime_health_uses_sdk_activity(hermes_health, level, busy): + runtime, agent = hermes_health + runtime._settings = {"interfaces": {"api": {"port": 18642}}} + runtime._interfaces.check = AsyncMock(return_value="agent_exited") + invocation = asyncio.create_task(asyncio.Event().wait()) if busy else None + runtime._active_invoke_task = invocation + try: + result = await runtime.check(level) + assert (result["outcome"], result["activity"]) == ( + "passed", + "busy" if busy else "idle", + ) + runtime._interfaces.check.assert_not_awaited() + agent.get_activity_summary.assert_called_once_with() + finally: + if invocation is not None: + invocation.cancel() + await asyncio.gather(invocation, return_exceptions=True) + + @pytest.mark.parametrize("level", ["live", "active", "ready"]) async def test_hermes_health_diagnostic_reply_does_not_hide_closure( hermes_health, level diff --git a/tests/adapters/test_hermes_interfaces.py b/tests/adapters/test_hermes_interfaces.py index 67be36680..53c2c39eb 100644 --- a/tests/adapters/test_hermes_interfaces.py +++ b/tests/adapters/test_hermes_interfaces.py @@ -9,6 +9,7 @@ import importlib.util import json import os +import signal import socket import sqlite3 import sys @@ -21,6 +22,7 @@ import requests from nemo_fabric import Fabric from nemo_fabric import FabricConfig +from nemo_fabric import CheckRequest from nemo_fabric.errors import FabricConfigError from nemo_fabric_adapter_contract.models import AgentConfig from nemo_fabric_adapter_contract.models import RuntimeContext @@ -106,7 +108,16 @@ def _interface(name: str, **fields: object) -> dict[str, object]: # Settings that the descriptor's schema rejects during planning. INVALID_SETTINGS = [ - pytest.param(_interface("terminal"), id="unsupported"), + pytest.param({"sdk": {"enabled": False}}, id="no-execution-path"), + pytest.param({"sdk": {"enabled": "false"}}, id="sdk-not-boolean"), + pytest.param( + { + "sdk": {"enabled": False}, + "interfaces": {"terminal": {"port": 19119, "enabled": False}}, + }, + id="disabled-native-is-not-execution", + ), + pytest.param(_interface("unknown"), id="unsupported"), pytest.param(_interface("api", port=8642, key="x"), id="unknown-field"), pytest.param(_interface("api"), id="missing-port"), pytest.param(_interface("dashboard", port=65_536), id="port-too-high"), @@ -672,3 +683,537 @@ async def test_hermes_dashboard_without_a_built_frontend_fails_start( ) assert caught.value.code == "hermes_dashboard_frontend_missing" + + +@pytest.mark.parametrize("sdk_enabled", [True, False]) +async def test_sdk_selection_preserves_eager_default( + tmp_path, monkeypatch, sdk_enabled +): + import run_agent + import hermes_state + + agent = MagicMock() + database = MagicMock() + monkeypatch.setattr(run_agent, "AIAgent", agent) + monkeypatch.setattr(hermes_state, "SessionDB", database) + monkeypatch.setattr(interfaces.InterfaceChildren, "start", AsyncMock()) + settings = {"interfaces": {"api": {"port": 18642}}} + if not sdk_enabled: + settings["sdk"] = {"enabled": False} + runtime = adapter.HermesRuntime() + try: + await runtime.start(_payload(tmp_path, settings)) + assert agent.call_count == int(sdk_enabled) + assert database.call_count == int(sdk_enabled) + if not sdk_enabled: + result = await runtime.invoke(MagicMock(), MagicMock()) + assert result.status == "failed" + assert result.error.code == "hermes_sdk_disabled" + agent.assert_not_called() + finally: + await runtime.stop() + + +async def test_sdk_disabled_requires_an_enabled_native_path(tmp_path): + runtime = adapter.HermesRuntime() + with pytest.raises(adapter.lifecycle.LifecycleError) as error: + await runtime.start( + _payload( + tmp_path, + { + "sdk": {"enabled": False}, + "interfaces": {"api": {"port": 18642, "enabled": False}}, + }, + ) + ) + assert error.value.code == "hermes_no_execution_path" + + +@pytest.mark.parametrize("level", ["live", "active", "ready"]) +async def test_sdk_disabled_health_observes_native_backend( + tmp_path, monkeypatch, level +): + monkeypatch.setattr(interfaces.InterfaceChildren, "start", AsyncMock()) + probe = AsyncMock(return_value=None) + monkeypatch.setattr(interfaces.InterfaceChildren, "check", probe) + runtime = adapter.HermesRuntime() + try: + await runtime.start( + _payload( + tmp_path, + {"sdk": {"enabled": False}, "interfaces": {"api": {"port": 18642}}}, + ) + ) + result = await runtime.check(level) + assert result["outcome"] == "passed" + assert result["activity"] == "unknown" + assert runtime._agent is None + probe.assert_awaited_once_with(level) + probe.return_value = "agent_exited" + assert (await runtime.check(level))["reason_code"] == "agent_exited" + finally: + await runtime.stop() + + +@pytest.mark.parametrize("streaming", [False, True]) +@pytest.mark.parametrize( + "sdk_enabled,kill_api", + [ + (False, False), + (True, False), + pytest.param( + True, + True, + marks=pytest.mark.skipif( + sys.platform != "linux", reason="owned child lookup uses /proc" + ), + id="sdk-survives-api-exit", + ), + ], +) +async def test_sdk_invocation_preserves_native_runtime_health( + tmp_path, api_server, streaming, sdk_enabled, kill_api +): + os.environ["ADAPTER_PYTHON"] = sys.executable + os.environ["FABRIC_TEST_MODEL_KEY"] = MODEL_KEY + port = _free_port() + config = FabricConfig.from_mapping( + { + "metadata": {"name": "native-only-health"}, + "harness": { + "adapter_id": "nvidia.fabric.hermes", + "settings": { + "sdk": {"enabled": sdk_enabled}, + "interfaces": {"api": {"port": port}}, + }, + }, + "models": { + "default": { + "provider": "openai", + "model": "fabric-echo", + "base_url": f"{api_server}/v1", + "api_key_env": "FABRIC_TEST_MODEL_KEY", + } + }, + "tools": {"enabled": []}, + "environment": { + "provider": "local", + "workspace": str(tmp_path), + "artifacts": str(tmp_path / "artifacts"), + }, + } + ) + if streaming: + config.enable_relay(output_dir=str(tmp_path / "relay")) + fabric = Fabric() + environment = await fabric.prepare_environment(config, base_dir=tmp_path) + try: + async with await fabric.start_runtime_in( + config, environment, base_dir=tmp_path, streaming=streaming + ) as runtime: + for _ in range(2): + if streaming: + stream = runtime.invoke_stream(input="Check the SDK path.") + async for _ in stream: + pass + result = await stream.result() + else: + result = await runtime.invoke(input="Check the SDK path.") + if sdk_enabled: + assert result.status == "succeeded", result.to_mapping() + assert result.error is None + else: + assert result.status == "failed" + assert result.error.code == "hermes_sdk_disabled" + assert result.error.retryable is False + for level in ("live", "active", "ready"): + report = await fabric.check( + environment=environment, + request=CheckRequest( + scope="runtime", + runtime_id=runtime.runtime_id, + level=level, + ), + ) + assert report.outcome == "passed", json.dumps(report.to_mapping()) + assert report.runtime["activity"] == ( + "idle" if sdk_enabled else "unknown" + ) + token = ( + tmp_path + / "artifacts/.fabric/hermes/runtimes" + / runtime.runtime_id + / "interfaces/api.token" + ).read_text() + response = await asyncio.to_thread( + requests.post, + f"http://127.0.0.1:{port}/v1/chat/completions", + headers={"Authorization": f"Bearer {token}"}, + json={ + "model": "hermes-agent", + "messages": [{"role": "user", "content": "native still works"}], + }, + timeout=30, + ) + assert response.status_code == 200, response.text + assert response.json()["choices"][0]["message"]["content"] + if kill_api: + host_pid = result.metadata["host_pid"] + child_pids = ( + Path(f"/proc/{host_pid}/task/{host_pid}/children") + .read_text() + .split() + ) + api_children = [] + for pid in child_pids: + arguments = Path(f"/proc/{pid}/cmdline").read_bytes().split(b"\0") + if b"nemo_fabric_adapters.hermes.native" in arguments: + assert any( + runtime.runtime_id.encode() in argument + for argument in arguments + ) + api_children.append(int(pid)) + assert len(api_children) == 1 + os.kill(api_children[0], signal.SIGKILL) + async with asyncio.timeout(5): + while not interfaces.local_interfaces.port_available(port): + await asyncio.sleep(0.01) + for level in ("live", "active", "ready"): + report = await fabric.check( + environment=environment, + request=CheckRequest( + scope="runtime", runtime_id=runtime.runtime_id, level=level + ), + ) + assert report.outcome == "passed", report.to_mapping() + assert report.runtime == {"state": "active", "activity": "idle"} + result = await runtime.invoke(input="SDK after interface failure.") + assert result.status == "succeeded", result.to_mapping() + assert result.metadata["host_pid"] == host_pid + captured = await asyncio.to_thread( + requests.get, f"{api_server}/_requests", timeout=5 + ) + assert any( + "Check the SDK path." in json.dumps(request["messages"]) + and "SDK after interface failure." + in json.dumps(request["messages"]) + for request in captured.json() + ) + finally: + await fabric.release_environment(environment) + + +@pytest.mark.parametrize( + "settings, expected", + [ + ({"terminal": {"port": 19119}}, {"terminal": 19119}), + ( + {"terminal": {"port": 19119}, "dashboard": {"port": 19119}}, + {"dashboard": 19119}, + ), + ( + { + "terminal": {"port": 19119, "enabled": False}, + "dashboard": {"port": 19119}, + }, + {"dashboard": 19119}, + ), + ], +) +def test_tui_clients_share_one_backend(settings, expected): + assert interfaces.interface_ports({"interfaces": settings}) == expected + + +@pytest.mark.parametrize( + "settings, code", + [ + ( + {"terminal": {"port": 19119}, "dashboard": {"port": 19120}}, + "hermes_tui_port_conflict", + ), + ( + {"terminal": {"port": 19119}, "api": {"port": 19119}}, + "hermes_interface_port_conflict", + ), + ], +) +def test_tui_backend_port_conflicts_fail(settings, code): + with pytest.raises(adapter.lifecycle.LifecycleError) as error: + interfaces.interface_ports({"interfaces": settings}) + assert error.value.code == code + + +@pytest.mark.parametrize("dashboard_enabled", [True, False]) +async def test_terminal_backend_authentication_and_cleanup( + tmp_path, web_dist, dashboard_enabled +): + from nemo_fabric_adapters.hermes import terminal + + port = _free_port() + settings = {"sdk": {"enabled": False}, "interfaces": {"terminal": {"port": port}}} + if dashboard_enabled: + settings["interfaces"]["dashboard"] = {"port": port} + else: + os.environ["HERMES_WEB_DIST"] = str(tmp_path / "missing-web-assets") + runtime = adapter.HermesRuntime() + connection = _token_path(tmp_path).with_name("tui.json") + try: + await runtime.start(_payload(tmp_path, settings)) + assert runtime._agent is None + assert len(runtime._interfaces._children) == 1 + assert connection.stat().st_mode & 0o777 == 0o600 + token = connection.with_name("tui.token").read_text() + assert token not in connection.read_text() + assert await interfaces.tui_ready(port, token) + assert not await interfaces.tui_ready(port, "wrong-token") + environment = await asyncio.to_thread( + terminal.attachment_environment, connection + ) + assert environment["HERMES_TUI_GATEWAY_URL"].startswith( + f"ws://127.0.0.1:{port}/api/ws?" + ) + assert (await runtime.check("ready"))["outcome"] == "passed" + assert (await runtime.check("ready"))["activity"] == "unknown" + finally: + await runtime.stop() + assert not connection.exists() + assert not connection.with_name("tui.token").exists() + assert interfaces.local_interfaces.port_available(port) + + +@pytest.mark.parametrize("dashboard_enabled", [False, True]) +async def test_tui_turns_export_atif_while_client_stays_connected( + tmp_path, api_server, web_dist, dashboard_enabled +): + import aiohttp + + port = _free_port() + settings = {"sdk": {"enabled": False}, "interfaces": {"terminal": {"port": port}}} + if dashboard_enabled: + settings["interfaces"]["dashboard"] = {"port": port} + payload = _payload(tmp_path, settings, base_url=f"{api_server}/v1") + payload["runtime_context"]["telemetry"] = { + "relay_enabled": True, + "metadata": {"telemetry_providers": ["relay"]}, + } + relay_dir = tmp_path / "relay" + wrapper = tmp_path / "relay.json" + wrapper.write_text( + json.dumps( + { + "relay": { + "config": { + "version": 4, + "atof": { + "enabled": True, + "sinks": [ + { + "type": "file", + "output_directory": str(relay_dir / "atof"), + "filename": "events.atof.jsonl", + } + ], + }, + "atif": { + "enabled": True, + "output_directory": str(relay_dir / "atif"), + }, + } + } + } + ) + ) + os.environ["FABRIC_RELAY_CONFIG_PATH"] = str(wrapper) + runtime = adapter.HermesRuntime() + try: + await runtime.start(payload) + token = _token_path(tmp_path).with_name("tui.token").read_text() + async with aiohttp.ClientSession() as client: + async with client.ws_connect( + f"http://127.0.0.1:{port}/api/ws", params={"token": token} + ) as ws: + await ws.send_json( + { + "jsonrpc": "2.0", + "id": 1, + "method": "session.create", + "params": {"cwd": str(tmp_path)}, + } + ) + while True: + frame = await ws.receive_json(timeout=30) + if frame.get("id") == 1: + assert "result" in frame, frame + session_id = frame["result"]["session_id"] + break + for turn in range(2): + await ws.send_json( + { + "jsonrpc": "2.0", + "id": turn + 2, + "method": "prompt.submit", + "params": { + "session_id": session_id, + "text": f"native TUI turn {turn}", + }, + } + ) + while True: + frame = await ws.receive_json(timeout=60) + if frame.get("id") == turn + 2: + assert "result" in frame, frame + if ( + frame.get("method") == "event" + and frame["params"]["type"] == "message.complete" + ): + break + for _ in range(50): + trajectories = list((relay_dir / "atif").rglob("*.json")) + if len(trajectories) == turn + 1: + break + await asyncio.sleep(0.1) + assert len(trajectories) == turn + 1 + assert all( + json.loads(path.read_text())["steps"] for path in trajectories + ) + assert not ws.closed + captured = await asyncio.to_thread( + requests.get, f"{api_server}/_requests", timeout=5 + ) + model_turns = [request for request in captured.json() if request.get("tools")] + history = json.dumps(model_turns[-1]["messages"]) + assert "native TUI turn 0" in history + assert "native TUI turn 1" in history + assert runtime._agent is None + finally: + await runtime.stop() + + +async def test_tui_readiness_times_out_after_websocket_upgrade(): + from aiohttp import web + + async def silent_backend(request): + ws = web.WebSocketResponse() + await ws.prepare(request) + async for _ in ws: + pass + return ws + + app = web.Application() + app.router.add_get("/api/ws", silent_backend) + runner = web.AppRunner(app) + await runner.setup() + port = _free_port() + try: + await web.TCPSite(runner, "127.0.0.1", port).start() + started = time.monotonic() + assert not await asyncio.wait_for( + interfaces.tui_ready(port, "test-token"), timeout=4 + ) + assert time.monotonic() - started < 3 + finally: + await runner.cleanup() + + +async def test_four_paths_keep_conversations_separate_and_restart_cleanly( + tmp_path, api_server, web_dist +): + import aiohttp + from nemo_fabric_adapter_contract.models import AgentRunRequest + + api_port, tui_port = _free_port(), _free_port() + payload = _payload( + tmp_path, + { + "interfaces": { + "api": {"port": api_port}, + "dashboard": {"port": tui_port}, + "terminal": {"port": tui_port}, + } + }, + base_url=f"{api_server}/v1", + ) + runtime = adapter.HermesRuntime() + labels = [ + "sdk-concurrent-marker", + "api-concurrent-marker", + "browser-concurrent-marker", + "terminal-concurrent-marker", + ] + + async def tui_turn(client, token, label): + async with client.ws_connect( + f"http://127.0.0.1:{tui_port}/api/ws", params={"token": token} + ) as ws: + await ws.send_json( + { + "jsonrpc": "2.0", + "id": 1, + "method": "session.create", + "params": {"cwd": str(tmp_path)}, + } + ) + while True: + frame = await ws.receive_json(timeout=30) + if frame.get("id") == 1: + session_id = frame["result"]["session_id"] + break + await ws.send_json( + { + "jsonrpc": "2.0", + "id": 2, + "method": "prompt.submit", + "params": {"session_id": session_id, "text": label}, + } + ) + while True: + frame = await ws.receive_json(timeout=60) + if ( + frame.get("method") == "event" + and frame["params"]["type"] == "message.complete" + ): + return session_id + + try: + await runtime.start(payload) + token = _token_path(tmp_path).read_text() + tui_token = _token_path(tmp_path).with_name("tui.token").read_text() + assert len(runtime._interfaces._children) == 2 + async with aiohttp.ClientSession() as client: + + async def api_turn(): + async with client.post( + f"http://127.0.0.1:{api_port}/v1/chat/completions", + headers={"Authorization": f"Bearer {token}"}, + json={ + "model": "hermes-agent", + "messages": [{"role": "user", "content": labels[1]}], + }, + ) as response: + assert response.status == 200 + await response.read() + + results = await asyncio.gather( + runtime.invoke( + AgentRunRequest.from_mapping({"input": labels[0]}), + RuntimeContext.from_mapping(payload["runtime_context"]), + ), + api_turn(), + tui_turn(client, tui_token, labels[2]), + tui_turn(client, tui_token, labels[3]), + ) + assert results[2] != results[3] + async with client.get(f"{api_server}/_requests") as response: + captured = await response.json() + turns = [request for request in captured if request.get("tools")] + assert len(turns) == 4 + for request in turns: + history = json.dumps(request["messages"]) + assert sum(label in history for label in labels) == 1 + await runtime.stop() + assert interfaces.local_interfaces.port_available(api_port) + assert interfaces.local_interfaces.port_available(tui_port) + await runtime.start(payload) + assert (await runtime.check("ready"))["outcome"] == "passed" + finally: + await runtime.stop() diff --git a/tests/adapters/test_hermes_native_config.py b/tests/adapters/test_hermes_native_config.py index c4014aff3..d0c52daf5 100644 --- a/tests/adapters/test_hermes_native_config.py +++ b/tests/adapters/test_hermes_native_config.py @@ -122,9 +122,15 @@ def test_native_tool_defaults_survive_without_explicit_policy(tools, enabled, bl @pytest.mark.parametrize( "platform,relay_enabled,expected", - [("api_server", True, True), ("api_server", False, False), ("cli", True, False)], + [ + ("api_server", True, True), + ("api_server", False, False), + ("tui", True, True), + ("desktop", True, True), + ("cli", True, True), + ], ) -def test_only_relay_enabled_api_turns_finalize( +def test_relay_enabled_native_turns_finalize( monkeypatch, platform, relay_enabled, expected ): if relay_enabled: @@ -135,7 +141,7 @@ def test_only_relay_enabled_api_turns_finalize( monkeypatch.setattr(native.telemetry, "finalize_native_relay_segment", finalize) agent = MagicMock(platform=platform, session_id="native-session") - native.finalize_api_turn(agent) + native.finalize_native_turn(agent) if expected: finalize.assert_called_once_with("native-session") @@ -143,15 +149,15 @@ def test_only_relay_enabled_api_turns_finalize( finalize.assert_not_called() -def test_api_relay_finalization_failure_is_reported_without_raising( +def test_native_relay_finalization_failure_is_reported_without_raising( monkeypatch, caplog ): os.environ["HERMES_NEMO_RELAY_PLUGINS_TOML"] = "/test/plugins.toml" finalize = MagicMock(side_effect=RuntimeError("test export failure")) monkeypatch.setattr(native.telemetry, "finalize_native_relay_segment", finalize) - native.finalize_api_turn( + native.finalize_native_turn( MagicMock(platform="api_server", session_id="native-session") ) - assert "Hermes API turn Relay finalization failed" in caplog.text + assert "Hermes native turn Relay finalization failed" in caplog.text diff --git a/tests/adapters/test_hermes_terminal.py b/tests/adapters/test_hermes_terminal.py new file mode 100644 index 000000000..75f902d42 --- /dev/null +++ b/tests/adapters/test_hermes_terminal.py @@ -0,0 +1,99 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +"""Private attachment and managed browser ownership regressions.""" + +import json +import os +from unittest.mock import AsyncMock, MagicMock + +import pytest +from nemo_fabric_adapters.hermes import terminal + +pytestmark = pytest.mark.skipif( + os.name != "posix", reason="Hermes native interfaces require POSIX" +) + + +@pytest.fixture(name="connection") +def connection_fixture(tmp_path): + connection = tmp_path / "tui.json" + connection.write_text( + json.dumps( + { + "version": 1, + "mode": "headless", + "port": 19119, + "token_file": "tui.token", + "workspace": str(tmp_path), + } + ) + ) + connection.chmod(0o600) + token = tmp_path / "tui.token" + token.write_text("test-private-token") + token.chmod(0o600) + return connection + + +def test_terminal_refuses_public_credentials(connection, monkeypatch): + probe = AsyncMock(return_value=True) + monkeypatch.setattr(terminal, "tui_ready", probe) + connection.with_name("tui.token").chmod(0o644) + with pytest.raises(ValueError, match="owner-only"): + terminal.attachment_environment(connection) + probe.assert_not_called() + + +def test_terminal_refuses_symlink_credentials(connection, tmp_path): + token = connection.with_name("tui.token") + private = tmp_path / "other-token" + token.rename(private) + token.symlink_to(private) + with pytest.raises(OSError): + terminal.attachment_environment(connection) + + +def test_terminal_stale_backend_does_not_spawn(connection, monkeypatch): + monkeypatch.setattr(terminal, "tui_ready", AsyncMock(return_value=False)) + execute = MagicMock() + monkeypatch.setattr(terminal.os, "execvpe", execute) + with pytest.raises(ValueError, match="unavailable"): + terminal.attachment_environment(connection) + execute.assert_not_called() + + +def test_terminal_credentials_stay_in_environment(connection, monkeypatch): + monkeypatch.setattr(terminal, "tui_ready", AsyncMock(return_value=True)) + os.environ["HERMES_TUI_SIDECAR_URL"] = "ws://unrelated-server" + environment = terminal.attachment_environment(connection) + assert "test-private-token" in environment["HERMES_TUI_GATEWAY_URL"] + assert environment["HERMES_TUI_CWD"] == str(connection.parent) + assert "HERMES_TUI_SIDECAR_URL" not in environment + assert "test-private-token" not in connection.read_text() + + +@pytest.mark.usefixtures("requires_hermes_agent") +def test_browser_default_profile_attaches_to_owned_backend(tmp_path, monkeypatch): + from hermes_cli import main_tui_launch, web_server_chat + from nemo_fabric_adapters.hermes import native + + os.environ["HERMES_HOME"] = str(tmp_path) + command = ["/usr/bin/node", "--expose-gc", str(tmp_path / "dist/entry.js")] + monkeypatch.setattr(terminal, "frontend_command", lambda: command) + make_argv = MagicMock(return_value=(command, tmp_path)) + monkeypatch.setattr(main_tui_launch, "_make_tui_argv", make_argv) + monkeypatch.setattr( + web_server_chat, + "_build_gateway_ws_url", + lambda: "ws://127.0.0.1:19119/api/ws?token=test", + ) + original = web_server_chat._resolve_chat_argv + monkeypatch.setattr(web_server_chat, "_resolve_chat_argv", original) + native.configure_browser_attachment() + argv, _, environment = web_server_chat._resolve_chat_argv(profile="default") + assert argv == command + assert environment["HERMES_TUI_GATEWAY_URL"].startswith("ws://127.0.0.1:19119/") + with pytest.raises(ValueError, match="runtime's profile"): + web_server_chat._resolve_chat_argv(profile="other-profile") + make_argv.assert_called_once()