Skip to content

Embed AtomWorks Studio with chat-led RFD4 execution - #58

Draft
dburkhardt wants to merge 36 commits into
NVIDIA-BioNeMo:mainfrom
dburkhardt:feat/RFD4-plugin
Draft

dburkhardt wants to merge 36 commits into
NVIDIA-BioNeMo:mainfrom
dburkhardt:feat/RFD4-plugin

Conversation

@dburkhardt

@dburkhardt dburkhardt commented Oct 20, 2026 •

Copy link
Copy Markdown

Embed the public AtomWorks Studio 0.0.1 editor in BioNeMo Agent Toolkit as RFD4-Proteina Studio (Beta). Users can load PD-L1, annotate structures, save drafts, review generation settings, send a design request to chat, view results and export CIF/ZIP. The bundled frontend avoids HTTP iframes and preserves upstream source, provenance and license notices.

Compute is user-managed. Studio has only Run on: machine-name and job status. Chat asks for a named SSH host, checks RFD4 readiness and submits the saved request. If useful, chat points to Brev CLI for machine creation/start/stop; the plugin has no machine registry UI, provisioning, ownership claims, privileged setup or lifecycle service. No NGS Workbench dependency is required.

Execution and preservation

Generate freezes the annotated CIF, notes/image notes and settings before asking chat to read the request ID. Validation remains visible. Unsupported chat handoff preserves the request and reports an actionable error. The documented rfd4-studio-chat skill guides execution without hidden provisioning instructions.

Validation, upload and launch acknowledgement happen inside the active tool call. Only the model process continues remotely. Durable request IDs, remote locks and receipts reconcile retries without duplicating an ambiguous launch. Local locks do not span network calls. Collection verifies the original host, terminal receipt and result hashes; verified collected outputs remain available offline. SSH and TLS verification remain enabled.

Existing drafts, annotations, inputs, exports and results remain in place. Legacy compute records are unverified suggestions; legacy jobs retain read-only/reconciliation support and are never replayed through the new path. Obsolete manager/worker modules and their obsolete tests are removed. The new source modules remain under 1,000 lines.

Public runtime

Studio frontend: 4b419ebd327405035c8cf6ccde7f333d0878888e; CPU editor: atomworks-studio==0.0.1. Public model: RosettaCommons/RFD4-Proteina at c7c6cacbc617ab074b4e234448bed6581840949f; HF bundle: a06e2509c7033b010fc43614e9fd840303588502. Existing canonical setup/readiness helpers and locked dependencies are reused. No weights, credentials, local state or private evaluation bundles are shipped.

Current validation — October 23

  • 130 Python tests passed, including request integrity, named-host execution/reconciliation, failure-log collection, wrong-host/receipt refusal, offline exports, draft/metadata preservation, strict SSH, editor startup, source and packaged MCP protocols, and distribution checks.
  • Real Linux fixtures on the existing host passed duplicate-launch prevention, terminal failure/log retention, competing output-root lock and ambiguous-receipt refusal. These fixtures used temporary Python processes, not inference. Additional tests verify network calls run outside the local submission lock and failed uploads remain safely pre-launch.
  • 58 frontend tests passed, including failed chat handoff, repeated Generate, stale sessions, edits during review, export fallback, annotation/document navigation and metadata restoration.
  • Aggregate sync passes for 46 skills. The pinned frontend builds successfully. Authored diff whitespace checks pass; generated upstream HTML retains upstream formatting.
  • One real public PD-L1 design completed on the existing Brev A100 through the new source execution path. A fresh client recovered the same job and collected results/logs. Computer use at unchanged localhost:8769 opened its 1,454-atom result and verified the machine/status label. Draft notes were saved, switched and restored without selected compute. Existing compute was left running.
  • One authorized new A100 was created via Brev CLI at the quoted $1.98/hour. Its default driver 565 was incompatible. After explicit user approval, driver 580.196.07 was installed from NVIDIA's signed packages and the same machine rebooted. A leftover 565 library conflict was resolved without force-overwriting files; dpkg audit passed. The locked public runtime and verified HF bundle installed; all nine readiness checks passed. One canonical short-monomer design completed once (low, seed 42, exit 0). A fresh client recovered the same job, collected all logs/results, validated 866 finite-coordinate atoms and produced a CRC-clean ZIP without contacting compute. Computer use then opened the 866-atom result in Studio with the original machine name. Brev confirmed the new machine STOPPED afterward and the original A100 RUNNING. Stopped storage may still incur charges. No replacement machine was created.

Explicitly unpassed / before merge

  • Workspace publication and installed-plugin UAT remain unpassed. The same workspace plugin identity still has the previous installed release. Plugin Creator reports installed, but its metadata/files/update tools are unavailable in this session. Required next step: publish this branch's staged package as an update to the existing identity, then run a real installed-plugin Generate → chat → named-host design → result → CIF/ZIP export pass. Source/localhost tests do not substitute for this.
  • Fresh-machine runtime/design passed through the source execution path after the approved manual driver upgrade. This does not qualify unattended setup or substitute for installed-plugin UAT. The runtime setup uses CLI steps; no lifecycle manager was added back to the plugin.
  • Actual-host TLS root cause is not proven. The detached local worker is removed, but this alone does not establish the cause or resolution of the prior certificate failure. Compare harmless diagnostics and installed submission before/after publication.
  • The full failure/UAT matrix is not yet complete, including failed provider stop and installed-host interruption/export behavior. Earlier release UAT does not qualify this changed path.
  • Repository CI, maintainer review and resolution of any historical DCO failures are required. This PR has not been merged; the broad runtime profile and ChatGPT.com deployment remain unqualified. Inference success is not scientific validation.

…skills

Signed-off-by: Daniel Burkhardt <8322751+dburkhardt@users.noreply.github.com>
Signed-off-by: Daniel Burkhardt <8322751+dburkhardt@users.noreply.github.com>
Signed-off-by: Daniel Burkhardt <8322751+dburkhardt@users.noreply.github.com>
Signed-off-by: Daniel Burkhardt <8322751+dburkhardt@users.noreply.github.com>
Signed-off-by: Daniel Burkhardt <8322751+dburkhardt@users.noreply.github.com>
@dburkhardt

Copy link
Copy Markdown
Author

Public model release verified: anonymous Git access succeeds for https://github.com/RosettaCommons/RFD4-Proteina at c7c6cacbc617ab074b4e234448bed6581840949f; repository visibility is public and its license is BSD-3-Clause. Documentation updated in fcd02e1.

This resolves the public-source availability question, but the locked runtime cannot be changed by substituting the URL. Its old commit is not available from the public repository, and the release changes AtomWorks (3.0.4), TMol (0.1.64), Hugging Face Hub constraints (>=1,<2), and model loading to the Hugging Face bundle. The existing dependency lock/checkpoint contract needs a deliberate public-release migration and qualification. No model source/install script was executed during verification; the runtime pin remains unchanged.

The app's Connect compute request also arrived in the actual conversation. Reading its authoritative operation confirmed the correct session, connect action, no selected machine, no attempts and dispatched=false. Chat handoff is verified without provisioning side effects. Native-host download retesting and GPU/reconnect acceptance remain pending as documented.

@dburkhardt

Copy link
Copy Markdown
Author

Public migration commit 72f15e9e62af8d7b7e65659af47f9c3b051f293a is pushed. The supported local plugin installer reports version 0.3.0-rfd4.17-uat.1; all 88 integration files in its installed cache match the staged source hashes. This verifies installation, not reload of this conversation's existing MCP process.

The standalone preview was updated on the same approved origin, http://127.0.0.1:8769/. Draft/state/catalog file hashes were unchanged across startup. Actual Chrome checks then loaded the 926-atom PD-L1 demo, selected ASN17 (eight atoms), and exported a real 211,902-byte annotated CIF to Downloads. Independent Biotite parsing confirmed 926 atoms and two chains; the demo's two coordinate-less atoms remain explicitly disclosed by Studio. The browser automation download event timed out, but the actual received file was located and independently parsed.

This browser pass does not close native-host export acceptance: the embedded app inspector timed out during that separate attempt. Linux/GPU generation and disconnect/reconnect remain gated on independently verified SSH host trust. PR remains draft.

@dburkhardt

Copy link
Copy Markdown
Author

Demo-focused acceptance attempt: preserve existing functionality and machines; no compute refactor or new ownership/provisioning/recovery requirements were introduced.

The public-release migration remains at 72f15e9; synchronization still matches the 38-skill public distribution. Two environment blockers were reproduced before attempting inference:

  • .17-uat.1 is installed, but process inspection shows this Codex session's app and agent MCP servers still running from .16-uat.2. Opening Studio again did not reload them. The supported MCP CLI has no restart operation. A user-triggered Codex restart has been requested; installed-version acceptance cannot be claimed against the old servers. The MCP app inspector also times out reading the existing panels.
  • The existing RFD4 Brev target remains saved. Normal authenticated brev ls --json failed with unexpected EOF; bounded retries timed out, including a diagnostic HTTP/1.1-only attempt with TLS checks unchanged. The signed-in Chrome console resolves the expected organization but its inventory stays loading. Its AWS disruption banner is visible, but this does not prove our existing GPU is affected by that specific outage.
  • A harmless SSH probe to the exact saved alias, with strict checking and normal user/global trust stores, fails before running a command: No ED25519 host key is known ... Host key verification failed. No key was enrolled, no insecure fallback used, and no provider lifecycle action issued.

No design was submitted and disconnect/reconnect was not counted as passed. Drafts, annotations, results, saved targets and machines were preserved. Resume the same basic workflow after the current MCP servers load and the existing authenticated Brev/host trust connection is usable. No architectural change is proposed from these failures.

Signed-off-by: Daniel Burkhardt <8322751+dburkhardt@users.noreply.github.com>
Signed-off-by: Daniel Burkhardt <8322751+dburkhardt@users.noreply.github.com>
Signed-off-by: Daniel Burkhardt <8322751+dburkhardt@users.noreply.github.com>
Signed-off-by: Daniel Burkhardt <8322751+dburkhardt@users.noreply.github.com>
Signed-off-by: Daniel Burkhardt <8322751+dburkhardt@users.noreply.github.com>
@dburkhardt

dburkhardt commented Oct 22, 2026 •

Copy link
Copy Markdown
Author

RC3 installed-host UAT — October 22, 2026

Published and installed NVIDIA BioNeMo Agent Toolkit rc3-v0.2, version 0.2.0-rc.3.2, through the existing workspace plugin’s supported upload/install workflow. Backend plugin ID and workspace sharing were preserved. Exact source is pushed commit 50807d37b8a468cd908e8dc1a6f08e667c028044, including the final README limitation. All 529 installed file contents match the release ZIP; the installed launcher is executable (0755). The package retains 45 skills, both MCP servers, original prompts and MCP configuration.

A packaging mistake in RC3/RC3.1 dropped executable ZIP attributes and prevented both MCP servers from starting (Permission denied). RC3.2 restores every original file mode, including 18 executable files, without changing code/assets. Corrected ZIP SHA-256: 0915d80cd87b93d17c63261e7d0da6c07540891f72474d55c7bc7eaf5711aa1c. No installed cache was edited. Both actual extracted launchers passed MCP initialization/tool listing (27 app tools, 36 agent tools); the installed RC3.2 tools are active.

Fresh source checks passed: 77 frontend tests, 17 managed-service tests, 11 runtime-recipe tests, and aggregate synchronization for 45 skills. Existing clean-package/restart and real GPU evidence remains separately documented; no new inference was submitted in this pass.

Passed in the updated embedded host:

  • Opened the PD-L1 demo, listed the preserved existing Brev machine and submitted Connect through the UI.
  • Runs listed the historical failed plan and completed job b6ff223037f5472990f9c7fae8d4445f. View result opened the completed design: 1,454 atoms and 33 annotation categories.
  • Save CIF displayed upstream validation’s two missing heavy atoms; Save anyway completed the local-file/chat-link fallback. The installed retrieval tool verified export f8848134524e4ce5b17d2aedd41391e8: 581,240 bytes, SHA-256 728eaf1224e248058c94cf14bc79443f1b0dfc5d8969041e04c5789322ab0b38.
  • Runs > Download all completed the ZIP fallback. Installed retrieval verified export 0aa501223f56476fb242d5218bd2507b: 161,073 bytes, SHA-256 c1d2959af15e207d00b0594634349664c667981fca702cc8c26b562eb8575593. ZIP CRC passed; it includes the saved input and generated CIF.
  • Saved-draft listing displayed the existing UI-created annotated drafts. All 12 pre-pass draft files and the two pre-existing export files remain present and byte-identical. Additional autosaved test drafts and new exports were retained.

Remaining UAT gates: successful Connect/Ready and disconnect/reconnect, Manage in chat, and explicit expired-session restoration. The embedded connection/readiness flow hit a real Brev API 503 Service Unavailable at /api/me; the UI displayed the failure instead of Ready. A fresh installed-tool check at 20:39:38 UTC still returned provider_unavailable, empty GPU checks and unknown power. This is external unavailability, not proof that the machine is stopped or absent. The existing machine was not restarted, stopped, recreated or claimed; SSH checks were not weakened.

During draft-note testing, the embedded computer-use inspector began timing out. Rebinding and a screenshot-only attempt also timed out; further UI tests could not be verified. No window movement/resizing was requested. Expiry was not deliberately induced after inspector loss, preserving the current document. A transient run-history refresh also removed the Download all controls between inspection and click; reopening View result and clicking Download all succeeded. This remains a minor UI usability observation.

Full installed-host UAT: INCOMPLETE, not signed off. Source/backend and localhost success do not substitute for these remaining embedded gates. The output is operationally rendered/exported, not biologically validated. Fresh-machine unattended setup remains unqualified because the provider image required a manual driver upgrade. No merge; CI and maintainer review remain required.

Outage recovery — 21:37–21:39 UTC

The authorized 10-minute watcher detected recovery: direct brev ls succeeded in dburkhardt-hq and reported existing instance 86m880rf3 RUNNING / COMPLETED / READY. The installed RC3.2 check_compute tool verified the same A100 and all nine readiness checks at 21:37:42 UTC. No inference was run.

Installed-tool session disconnect/reconnect also passed: selection revision 1→2 detached only the retained UAT session; revision 2→3 selected the same saved machine; a fresh check at 21:39:05 UTC returned Ready with all nine checks. The obsolete, undispatched connection operation from the outage was cancelled by the normal selection-change guard. No power or provisioning action, runtime installation, ownership marker or SSH security change occurred. These receipts are installed-backend evidence, not a successful UI reconnect test.

The existing embedded panel handle is still listed, but inspection of that exact handle times out after recovery. Remaining installed-UI gates are Manage in chat, explicit expired-session restoration, and UI Connect/Ready plus disconnect/reconnect. Full UAT remains incomplete because inspector access has not recovered. The watcher remains enabled to retry access and continue autonomously; no user window manipulation is requested.

Embedded inspector diagnosis — October 22, 2026

Read-only isolation tests: MCP Apps inventory lists both RFD4 panels. Direct title() and url() on panel 2 return immediately. A minimal documented playwright.evaluate(() => document.title, undefined, {timeoutMs:3000}) returns CDP operation exceeded its deadline before command dispatch (6.5758 seconds wall time). Full getTab/DOM observation timed out and reset the control kernel. Chrome inspection of the existing localhost preview returns in 4.36 seconds. This narrows the failure to the embedded inspection/control route; it does not prove a specific internal root cause or that every timed-out action was never dispatched.

Computer Use inventory separately reports the Mac locked and automatic unlock unsuccessful. Lock timing is not established, so it is a possible contributing condition, not a proven trigger. Desktop diagnostic logs show both current RFD4 guest documents and widget frames committed at 20:32:45 and 20:33:08 UTC, with no recorded RFD4 crash in the examined entries. Offscreen suspension at 20:37:54 belongs to a different sandbox, not either RFD4 panel; it must not be cited as the RFD4 cause.

Source Notes input handler invalidates the plan and schedules a 1.5-second asynchronous draft save. No synchronous loop was found in that path. The temporal association between the Notes fill and inspector loss is not causal evidence. No plugin code/cache or saved document was modified during diagnosis.

Web research found a closely matching, still-open macOS MCP Apps report on the official repository: openai/codex#55016 (Oct 18). It describes working metadata, failed DOM requests with the same error, and snapshot timeouts. This is another user's primary report, not maintainer confirmation of our root cause or an established fix. Official guidance recommends isolating server/component/client and gathering diagnostics: https://developers.openai.com/plugins/deploy/troubleshooting .

No supported agent-accessible guest reset/reload endpoint is advertised by this MCP Apps backend. Native Codex control remains prohibited. Retain the existing watcher and resume remaining installed-host gates only after a bounded DOM read succeeds. Full installed-host UAT remains incomplete.

Final installed embedded-host UAT — October 22, 2026, 22:57–23:04 UTC

This section supersedes the earlier inspector-blocked gates. The old panel handles were authoritatively absent after control-runtime reinitialization. The installed open_rfd4_studio tool reopened the same RC3.2 package through its supported interface; a new panel returned a successful DOM title and full UI snapshot. No installed caches, permissions, window geometry, machines or SSH security settings were changed.

Passed installed embedded UI gates:

  • Existing-machine Connect reached Ready for rfd4-32202dfb684c4d67b72fd6e8e7fa342e, showing NVIDIA A100 80GB PCIe, 81151 MiB and Connection: connected (last checked 18:58:27 Toronto).
  • Disconnect this session reached Not connected; Connect on the same preserved machine reached Ready again (18:59:20 Toronto). No provisioning, power action or new inference was performed.
  • Manage in chat was clicked after reconnect. The embedded UI acknowledged Compute request sent to chat. and Waiting for your compute choice in chat. This proves host acceptance of the UI message request; subsequent conversational handling is outside the button acknowledgment itself.
  • Created an isolated UI note RC3.2 installed embedded recovery UAT — preserve this draft. Backend draft file confirmed the exact note persisted at 22:59:21 UTC. Kept a reversible editor-session backup in /private/tmp before expiring only this test session via the supported editor DELETE endpoint (204). Saved drafts were not deleted.
  • Explicitly selected that saved draft in the embedded UI. Editor log recorded export of the expired session returning 404, followed by creation of a new editor session and successful CIF read. The restored UI showed the original note, PD-L1 binder demo, 926 atoms and both chains. Direct readback of the restored annotated CIF matched the saved CIF line-for-line except _entry.time changing from 18:59:21 to 19:01:33. Thus molecular data and annotations were retained through the actual installed UI restore path.

Non-blocking UI observations: the draft badge remained Unsaved changes after the exact note was verified persisted; the explanatory expired-session message was not observed, although the restoration completed and its content was verified. These visual feedback issues are not claimed fixed. An attempted draft-list refresh while unsaved was rejected by automatic review; after read-only proof of the exact saved file, the refresh was permitted. No confirmation or window manipulation was requested.

Final preservation audit: all 12 pre-pass saved draft files and both pre-pass exports remain byte-identical to the original manifest. Managed compute state changed as expected during session selection/connection tests and is not claimed byte-identical. Original and test drafts, results and existing machines are retained. Screenshot: rc3-final-restored-ui.png.

Combined installed-host coverage is now complete for the requested operational demo: result viewing and CIF/ZIP fallbacks passed earlier in RC3.2; Connect/Ready, disconnect/reconnect, Manage in chat acknowledgment and explicit expired-session draft restoration passed in this final session. This is operational UAT, not biological validation or qualification of unattended fresh provisioning. The two upstream missing-heavy-atom warnings remain. PR head is 50807d3, PR remains OPEN, and GitHub reports no CI checks. No merge performed.

Manage in chat delivery receipt

The queued embedded Manage in chat request arrived in this same conversation after UAT completed. Authoritative get_compute_operation verified operation 28ef1bcb7b5741dfb4d83b388b3d0412 belonged to the isolated, subsequently expired test editor session. It was needs_attention, undispatched, with no attempts and no machine started. This confirms actual chat delivery in addition to the UI acknowledgment. The obsolete test request was cancelled without dispatch; no machine lifecycle action was taken.

@dburkhardt

dburkhardt commented Oct 23, 2026 •

Copy link
Copy Markdown
Author

UI-feedback fixes published as 0.2.0-rc.4 on the unchanged NVIDIA BioNeMo Agent Toolkit rc4-v0.2 workspace plugin (Plugin_8fc62477dcd08191a72f8147702def01). Source commit 0184def62b98bdc0a799d7bb2d1f1288cc574c26 is pushed to feat/RFD4-plugin on the explicit fork URL.

Compute fingerprint/selection changes invalidate reviewed plans but no longer invalidate draft-save acknowledgments or pending autosaves. Draft-content revisions remain independently guarded, so old saves cannot mark newer edits or a different session saved. Explicit expired-session restoration now shows its notice immediately after the replacement document acknowledges opening, before the compute refresh; it no longer tests the intentionally invalidated old-session guard. Other save errors still block replacement, and failed opens never announce success.

Validation: all 81 JavaScript frontend tests pass. The two targeted regressions fail against the prior implementation (with its restore handler exposed to the same test harness) and pass against this change. Rebuilt the pinned embedded Studio HTML. RC4 ZIP retains all 529 paths and original external mode attributes; only the two version manifests and bundled UI HTML differ from RC3.2. SHA256: 85e649579031c952033ab8dea649a1d76d22a0aafe0f1f48ae560fdfa20abb70.

Publication was verified on the existing workspace-plugin page at version 0.2.0-rc.4; identity, 45 skills, both MCP servers, prompts and sharing scope were retained. The desktop cache still contains RC3.2, so RC4 installed embedded-host verification is pending normal desktop update/reload. Prior RC3.2 operational UAT is separate evidence. No installed cache edits, machine lifecycle operations, new GPU designs, state deletion, watcher recreation or merge occurred. Screenshot: rc4-published.png.

Per user naming correction, RC4 supersedes RC3.3 with the same tested UI bytes. Only manifest version/display name changed; package identity, executable modes and sharing scope remain unchanged.

Signed-off-by: Daniel Burkhardt <8322751+dburkhardt@users.noreply.github.com>
@dburkhardt dburkhardt changed the title Embed AtomWorks Studio 0.0.1 and add standalone RFD4 compute management Embed AtomWorks Studio with chat-led RFD4 execution Oct 23, 2026
Signed-off-by: Daniel Burkhardt <8322751+dburkhardt@users.noreply.github.com>
Signed-off-by: Daniel Burkhardt <8322751+dburkhardt@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant