Donkey Plugin

[iliketocode2]

Here, I've explored making the donkey more of a plugin which different components can easily use. It has been implemented for the chart, webcam and codeeditor widgets. I have created "test pages" for the codeeditor and the chart widgets. Claude and I spent a while discussing/organizing an architecture for the donkey plugin itself, so while that is more defined, the simple /tests folder with the aforementioned test pages can be moved around.

This PR also relates to the idea of interactive test pages for components such as BLE and webserial as I have built in some "assertions" that show whether or not the donkey has run as expected. Similar logic could be applied to features that require developers to allow BLE connections for example.

The following spec (thanks Claude!) describes the donkey plugin implementation:

Donkey Plugin: Webcam / OpenCV Adapter

Summary

This PR completes the migration of OpenCV/webcam support into the donkey plugin architecture. OpenCVDonkey and create_opencv_donkey have been removed. OpenCV now runs through WebcamDonkeyAdapter, which is a first-class WidgetDonkeyAdapter subclass alongside the existing ChartDonkeyAdapter and CodeEditorDonkeyAdapter.

A test page for the webcam adapter has been added under tests/webcam/, following the same pattern as the chart and codeeditor test pages.

---

File overview

device.py:

• Owns worker lifecycle (DonkeyConnection), datastore status/result wiring, and widget adapters (ChartDonkeyAdapter, CodeEditorDonkeyAdapter, WebcamDonkeyAdapter).
• Defines donkey interface

donkey_plugin.py:

• DonkeyPluginFlow, make_plugin_runner, and make_assertion_callbacks reduce repetitive main.py code for start/run/error handling.
• For app pages

test_helpers.py:

• pass_html, fail_html, wait_html, and StatusProxy for assertion/status display and channel publishing.
• For manual/interactive testing

tests/

• New test pages for the webcam/OpenCV adapter, chart and codeeditor.
• Uses make_plugin_runner and make_assertion_callbacks for worker lifecycle and assertion display

---

Removed

• OpenCVDonkey — superseded by WebcamDonkeyAdapter.
• create_opencv_donkey — superseded by WebcamDonkeyAdapter.initialize().

---

Architecture Notes

WebcamDonkeyAdapter follows the WidgetDonkeyAdapter contract with one intentional deviation: it overrides run() rather than routing through DonkeyConnection.run_code(). This is because the OpenCV worker calls worker_run_user_code(code, data_url) directly rather than invent_run_code(code, context_json). The deviation is contained inside the adapter and is invisible to callers.

_context() returns {} in WebcamDonkeyAdapter. It satisfies the base class interface but is not used, because run() builds the worker expression itself. This is a known stub; it would only become meaningful if the OpenCV worker were later refactored to accept context via invent_run_code.

---

Donkey Plugin Spec (v1)

Purpose

Define a standard way to attach Python worker logic ("donkey plugins") to widgets so Invent apps can compose features like LEGO bricks.

This spec captures both the target model and the current implementation.

Core Decisions

1. Strict output contract
   Plugin task output is strict and must be an object:

   • success: {"ok": true, "result": <object>}
   • failure: {"ok": false, "error": "<message>"}

2. Worker ownership
   Each plugin instance owns its own donkey worker. Plugins do not share workers by default.

3. Trigger model
   Execution is triggered by explicit button press only in v1.

4. Status visibility
   Every plugin publishes worker status to datastore.

5. Interpreter and packages
   Worker type is py only. Packages are framework-managed, not user-managed.

Runtime Archetype

Runtime implementation lives in invent.tools.device via DonkeyConnection.

Required API:

• initialize()
• execute(code)
• evaluate(expression)
• run_code(code, result_key, context=None)
• kill()
• ready property

The runtime is widget-agnostic and must not include widget-specific logic.

Datastore Contract

Each plugin defines two keys:

• status_key: worker lifecycle and run-time status
• result_key: latest task result payload

Status Values

Use framework constants:

• _DEVICE_DONKEY_CREATING
• _DEVICE_DONKEY_BUSY
• _DEVICE_DONKEY_READY
• _DEVICE_DONKEY_KILLED
• _DEVICE_DONKEY_ERROR: <details>

Result Values

result_key must always receive an object:

• success: {"ok": true, "result": {...}}
• failure: {"ok": false, "error": "..."}

Adapter Archetype

Adapter base implementation lives in invent.tools.device via WidgetDonkeyAdapter.

Subclass responsibilities:

• _context() returns JSON-safe context dict
• _apply_result(payload) validates payload and applies widget updates

Current adapters:

• ChartDonkeyAdapter
• CodeEditorDonkeyAdapter
• WebcamDonkeyAdapter

Task Contract

Runtime task execution inputs:

• code: Python code string to execute
• context: JSON-safe object available to executed code
• result_key: datastore key for strict result object

Executed code rules:

• user code must assign result
• result must be JSON-safe and adapter-compatible

Page Workflow Helpers

To keep user-facing main.py files small, workflow helpers live in invent.tools.donkey_plugin.

Implemented helpers:

• DonkeyPluginFlow
  • shared worker startup and run-state handling
  • standard error extraction for strict result payloads
• make_plugin_runner(...)
  • returns standard ensure_worker and run_code callables
• make_assertion_callbacks(...)
  • returns reusable worker/run assertion callbacks for tests

These helpers are optional but recommended for user-facing pages.

Example: Chart Plugin

Chart adapter context:

{
    "chart_type": chart.chart_type,
    "data": chart.data,
    "options": chart.options,
}

Expected user code:

result = {
    "data": transformed_data,
    "options": transformed_options,
}

Apply rules:

• if result["data"] exists, update chart.data
• if result["options"] exists, update chart.options

Example: CodeEditor Plugin

CodeEditor adapter context:

{
    "editor_code": source_editor.code,
}

Expected user code:

result = {
    "output": "some text derived from editor_code",
}

Apply rules:

• result["output"] is written to output widget .text

Example: Webcam Plugin

The webcam adapter does not use a context dict. run() retrieves the latest photo capture from the widget directly and passes it to the OpenCV worker alongside the user code.

User code has access to:

image_bgr   # captured frame as a BGR numpy array
image_rgb   # same frame in RGB
grey        # greyscale version
cv2         # OpenCV module
np          # NumPy module

Expected user code:

result_image = grey  # or any numpy ndarray

Apply rules:

• result_image (or result) must be a numpy ndarray
• the processed frame is displayed via webcam.show_image(data_url)

Acceptance Criteria

A donkey plugin is compliant if it:

• owns one donkey runtime instance
• runs only on button press
• writes status updates to datastore
• writes strict result object to datastore
• updates widget state through an adapter layer
• keeps widget code free of donkey internals

---

Since this PR is on top of the webcam one, here is a brief summary paragraph of the changes to the webcam:

The webcam widget now supports a photo_output property controlling whether captured photos are downloaded, shown in a preview panel, or both. A preview_layout property allows the live video feed and capture preview to be displayed stacked or side-by-side. Captures are stored internally via a capture history, which exposes captures() and latest_capture() for querying by media type. This is what allows the donkey adapter to retrieve the most recent frame for OpenCV processing. A show_image() method allows any external data URL to be displayed in the preview panel, replacing its current content. The photo_captured event now carries a capture argument containing the photo metadata, including its data URL, timestamp, and ID.

[iliketocode2]

Ok, this implementation feels much cleaner! 🧹

[ntoll]

Improvised API from our call 😉 :

from invent.tools import make_helper

# We're starting the donkey in the background with a certain script (containing
# the function "do_stuff") with a certain interpreter and configuration.
#
# Calling the make_helper function kicks off the startup of the web-worker
# with the given configuration and source code. All further communication with
# the helper is via the given channel. The helper sends two types of message
# distinguished by subject:
#
#  1. "result" - the result of a call (see details below)
#  2. "status" - to indicate the helper is ready, broken, killed, whatever.
#
# Furthermore, the helper listens on the channel for messages with the subject
# "run", which define a function to call and args/kwargs to pass in. See below
# for details.
make_helper(
    src="something.py",
    interpreter="py",
    config={...},
    channel="my_helper"
)


def handle_click(msg):
    invent.publish(
        Message(
            subject = "run",
            function="do_stuff",
            args = [
                1, 2, 3
            ]
            kwargs={
                a=4,
                b=5,
            }
        )
    )
    # At this point the helper swings into action. It will publish the return
    # value to `my_helper` with the subject "result".


def handle_raw_result(msg):
    """
    Message from the helper looks like this:

    msg.function (the name of the function that was called)
    msg.result (the raw returned value from the helper)
    msg.error (if there was a problem, describe it here)
    """
    if msg.function == "do_stuff":
        # Do stuff
        if msg.error:
            # Something went wrong.
            ... # handle error condition.
        else:
            invent.datastore["results"] = msg.result # or perhaps post-process the result derived from msg.

invent.subscribe(handle_click, "make_stuff", "click")
invent.subscribe(handle_raw_result, "my_helper", "result")

app = invent.App(
    ...
    children = [
        Chart(
            ... # we're showing a bar chart
            data=from_datastore("results"),  # The data in the bar chart is always what's at "results" in the datastore.
        ),
        Button(label="Click me", channel="make_stuff")  # Pressing this kicks things off.
    ]
)