diff --git a/CONTEXT.md b/CONTEXT.md new file mode 100644 index 0000000..915941b --- /dev/null +++ b/CONTEXT.md @@ -0,0 +1,71 @@ +# Instrumentserver + +Distributed control of QCoDeS instruments over ZMQ: one server owns the hardware, many clients talk to it through proxies. Part of the Tools For Experiments suite (sibling of labcore). + +## Language + +**Server**: +The single process that owns the QCoDeS Station and its instruments, serving requests over ZMQ. +_Avoid_: instrument server (two words), backend + +**Client**: +A Python object (or process using one) that sends requests to the Server and receives proxies back. + +**Proxy Instrument**: +A client-side object mirroring a server-side instrument, built dynamically from its Blueprint. +_Avoid_: remote instrument, instrument handle + +**Blueprint**: +A serializable description of an instrument, parameter, or method that lets clients reconstruct its interface without importing the driver. +_Avoid_: schema, spec + +**Broadcast**: +A parameter-change event published by the Server on its PUB socket for any subscriber. +_Avoid_: notification, event stream + +**Virtual Instrument**: +An instrument that lives entirely in the Server with no hardware behind it. +_Avoid_: soft instrument, fake instrument (that's a dummy instrument, for testing) + +**Parameter Manager**: +The flagship Virtual Instrument: a hierarchical, persistent, profile-aware store of experiment parameters. +_Avoid_: param store, PM + +**Client Station**: +A client-side grouping of Proxy Instruments giving one client a scoped view of a shared Server. +_Avoid_: sub-server (colloquial; useful as an explanation, not a name) + +**Listener**: +A standalone subscriber to Broadcasts that exports parameter changes to a sink (CSV, InfluxDB). +_Avoid_: monitor, logger + +**Detached GUI**: +The Server's GUI running in a separate process so UI failures cannot take down the Server. + +**Dummy Instrument**: +A hardware-free test instrument shipped in `instrumentserver.testing` for development and verified documentation. + +**Chained Servers**: +A Server acting as a Client of another Server, so instruments can be re-exported downstream. +_Avoid_: server-in-server, daisy-chaining (fine in prose, not as the term) + +## Relationships + +- The **Server** owns instruments; **Clients** reach them only through **Proxy Instruments** built from **Blueprints** +- Every parameter change on the **Server** produces a **Broadcast**; **Listeners** and GUIs consume Broadcasts +- A **Client Station** groups Proxy Instruments for one client; many Client Stations can share one Server +- A **Virtual Instrument** is served like any other instrument; the **Parameter Manager** is one +- **Chained Servers** compose: a downstream Server proxies an upstream Server's instruments + +## Example dialogue + +> **Dev:** "If I set a value on a **Proxy Instrument**, who finds out?" +> **Domain expert:** "The **Server** executes the set under that instrument's lock, then emits a **Broadcast** — every subscribed GUI and **Listener** sees it, no polling needed." +> **Dev:** "And a **Client Station** is a second server?" +> **Domain expert:** "No — it never owns instruments. It's a scoped view: one client's chosen set of **Proxy Instruments** over the same shared **Server**. If you actually need a second server re-exporting instruments, that's **Chained Servers**." + +## Flagged ambiguities + +- "apps" — the five console entry points are launchers, not five separate applications; they collapse into three features (Server, Client Station, Monitoring) plus two convenience launchers (Detached GUI, Parameter Manager GUI). Resolved: docs pages follow features, not entry points. +- "sub-server" — used colloquially for **Client Station**; resolved: explanation, not terminology. +- "virtual instrument" vs "dummy instrument" — distinct: virtual = production feature with no hardware; dummy = testing stand-in for hardware. diff --git a/docs/_static/animations/request_flow.html b/docs/_static/animations/request_flow.html new file mode 100644 index 0000000..3d04a66 --- /dev/null +++ b/docs/_static/animations/request_flow.html @@ -0,0 +1,865 @@ + + +
+ +
+
+ Step 01 + The Server loads its configuration +
+ + + How a configured instrument becomes a shared remote instrument + The Server reads a configuration file, creates the + generator QCoDeS driver, and connects it to an RF source. Your script connects + through a Client, requests a Blueprint, and builds a Proxy Instrument. Later + calls and their results travel directly between that Proxy and the Server, and + Broadcasts update the Server GUI and a Listener. + + + + + + + connects + + + + + Blueprint + + + + + request / result + + + + connection + + + + Broadcast + + + + Broadcast + + + + + + + + + + + + + your script + + + + + + + constructs + + + + + + Client + + + + + generator + Proxy + Instrument + + + + + Server + + + + + + + creates + + + + + + configuration + file + + + + + generator + QCoDeS driver + + + + + RF source + laboratory hardware + + + + + the Server's GUI + updates the display + + + + + a Listener + records the change + + + + +
+ +
    +
  1. +
    +

    Step 01

    +

    The Server loads its configuration

    +

    At startup, the Server reads its configuration file and + creates generator, the QCoDeS driver for the instrument. + (Instruments can also be started from a Client at runtime.)

    +
    +
    2. + +
  2. +
    +

    Step 02

    +

    The driver connects to the hardware

    +

    The generator driver establishes the Server's connection to the + RF source on the bench. No other process connects to the hardware directly.

    +
    +
    3. + +
  3. +
    +

    Step 03

    +

    A Client connects

    +

    Your script creates a Client and establishes a two-way + connection with the Server.

    +
    +
    4. + +
  4. +
    +

    Step 04

    +

    The Client requests a Blueprint

    +

    You ask for generator. The Client asks the Server for a + Blueprint describing that instrument's interface.

    +
    +
    5. + +
  5. +
    +

    Step 05

    +

    The Server returns the Blueprint

    +

    The Server replies to the Client with the Blueprint for + generator.

    +
    +
    6. + +
  6. +
    +

    Step 06

    +

    The Client constructs the Proxy

    +

    The Client uses the Blueprint to construct a local + Proxy Instrument named generator.

    +
    +
    7. + +
  7. +
    +

    Step 07

    +

    You call the Proxy

    +

    You call generator.frequency(5e9). The Proxy sends the request + directly to the Server; the standalone Client that constructed it is not + involved. Nothing is set locally.

    +
    +
    8. + +
  8. +
    +

    Step 08

    +

    The Server runs the call

    +

    The Server runs the set on the real generator, which communicates + with the RF source through the Server's hardware connection.

    +
    +
    9. + +
  9. +
    +

    Step 09

    +

    The Server returns the result

    +

    The Server sends the result back to the Proxy that made the request. The Proxy + call completes.

    +
    +
    10. + +
  10. +
    +

    Step 10

    +

    The Server publishes a Broadcast

    +

    The Server separately publishes the parameter change as a + Broadcast. It publishes the message once, and every subscriber + that is listening can receive it.

    +
    +
    11. + +
  11. +
    +

    Step 11

    +

    Subscribers react

    +

    The Server's GUI updates its display, and the Listener records + the change.

    +
    +
    12. + +
+
+ + + + diff --git a/docs/_static/getting_started/quickstart/generator_widget_dark.png b/docs/_static/getting_started/quickstart/generator_widget_dark.png new file mode 100644 index 0000000..379feab Binary files /dev/null and b/docs/_static/getting_started/quickstart/generator_widget_dark.png differ diff --git a/docs/_static/getting_started/quickstart/generator_widget_light.png b/docs/_static/getting_started/quickstart/generator_widget_light.png new file mode 100644 index 0000000..2153d44 Binary files /dev/null and b/docs/_static/getting_started/quickstart/generator_widget_light.png differ diff --git a/docs/_static/getting_started/quickstart/generator_widget_set_dark.png b/docs/_static/getting_started/quickstart/generator_widget_set_dark.png new file mode 100644 index 0000000..1352508 Binary files /dev/null and b/docs/_static/getting_started/quickstart/generator_widget_set_dark.png differ diff --git a/docs/_static/getting_started/quickstart/generator_widget_set_light.png b/docs/_static/getting_started/quickstart/generator_widget_set_light.png new file mode 100644 index 0000000..b9f79f8 Binary files /dev/null and b/docs/_static/getting_started/quickstart/generator_widget_set_light.png differ diff --git a/docs/_static/getting_started/quickstart/server_bare_dark.png b/docs/_static/getting_started/quickstart/server_bare_dark.png new file mode 100644 index 0000000..36490f2 Binary files /dev/null and b/docs/_static/getting_started/quickstart/server_bare_dark.png differ diff --git a/docs/_static/getting_started/quickstart/server_bare_light.png b/docs/_static/getting_started/quickstart/server_bare_light.png new file mode 100644 index 0000000..b3f306a Binary files /dev/null and b/docs/_static/getting_started/quickstart/server_bare_light.png differ diff --git a/docs/_static/getting_started/quickstart/server_generator_dark.png b/docs/_static/getting_started/quickstart/server_generator_dark.png new file mode 100644 index 0000000..ac59cf5 Binary files /dev/null and b/docs/_static/getting_started/quickstart/server_generator_dark.png differ diff --git a/docs/_static/getting_started/quickstart/server_generator_light.png b/docs/_static/getting_started/quickstart/server_generator_light.png new file mode 100644 index 0000000..2430c23 Binary files /dev/null and b/docs/_static/getting_started/quickstart/server_generator_light.png differ diff --git a/docs/about.md b/docs/about.md deleted file mode 100644 index f45d7b3..0000000 --- a/docs/about.md +++ /dev/null @@ -1,33 +0,0 @@ -# About InstrumentServer - -InstrumentServer is part of the [Tools For Experiments](https://github.com/toolsforexperiments) initiative—a collection of software tools developed by the [Pfaff-lab at the University of Illinois at Urbana-Champaign](https://pfaff.physics.illinois.edu/). - -## Purpose - -InstrumentServer solves the problem of remote instrument access in modern laboratories. Whether you need to control laboratory equipment from a different machine, enable multiple researchers to access the same instruments simultaneously, or build distributed measurement systems, InstrumentServer provides a robust, scalable solution. - -## Design Philosophy - -InstrumentServer is built on practical experience with real laboratory needs: - -- **Simplicity**: Uses well-established ZMQ messaging patterns and QCoDeS integration -- **Reliability**: Per-instrument locking ensures thread-safe concurrent access -- **Performance**: Asynchronous request handling with concurrent instrument control -- **Transparency**: Proxy objects provide native Python interfaces to remote instruments - -## Contributing - -InstrumentServer is open source and welcomes contributions. Visit the [GitHub repository](https://github.com/toolsforexperiments/instrumentserver) to report issues, submit pull requests, or participate in development. - -## Citation - -If you use InstrumentServer in your research, please cite the project: - -``` -@software{instrumentserver, - title={InstrumentServer: Distributed QCoDeS Instrument Control}, - author={Pfaff, Wolfgang}, - url={https://github.com/toolsforexperiments/instrumentserver}, - year={2020} -} -``` \ No newline at end of file diff --git a/docs/api/index.md b/docs/api/index.md index 7bb817e..b30a5a9 100644 --- a/docs/api/index.md +++ b/docs/api/index.md @@ -1,38 +1,27 @@ # API Reference -Complete API documentation for InstrumentServer, automatically generated from source code docstrings. - -## Main Modules +This section is generated straight from the source code docstrings, so it always +reflects what the package actually ships. It's the place to look up exact signatures, +classes, and module contents. For narrative documentation, start with the +[User Guide](../user_guide/index.md) for how to use the package, or the +[Technical Guide](../technical_guide/index.md) for how it works inside. ```{eval-rst} .. autosummary:: :toctree: generated :recursive: - instrumentserver.server - instrumentserver.client + instrumentserver.apps + instrumentserver.base instrumentserver.blueprints + instrumentserver.client + instrumentserver.config + instrumentserver.gui + instrumentserver.helpers + instrumentserver.log instrumentserver.monitoring + instrumentserver.params + instrumentserver.serialize + instrumentserver.server + instrumentserver.testing ``` - -## Quick Navigation - -### Server API -- `instrumentserver.server.core.StationServer` - Main server class -- `instrumentserver.server.application` - Server GUI components - -### Client API -- `instrumentserver.client.proxy.Client` - High-level client API -- `instrumentserver.client.proxy.ProxyParameter` - Remote parameter interface -- `instrumentserver.client.proxy.ProxyInstrumentModule` - Remote instrument interface -- `instrumentserver.client.core.BaseClient` - Low-level ZMQ client - -### Messaging -- `instrumentserver.blueprints.ServerInstruction` - Client requests -- `instrumentserver.blueprints.ServerResponse` - Server responses -- `instrumentserver.blueprints.ParameterBluePrint` - Parameter metadata -- `instrumentserver.blueprints.InstrumentModuleBluePrint` - Instrument metadata - -### Monitoring -- `instrumentserver.monitoring.monitor.ParameterListener` - Real-time parameter updates -- `instrumentserver.monitoring.monitor.ParameterLogger` - Parameter logging \ No newline at end of file diff --git a/docs/conf.py b/docs/conf.py index 463f94b..22d2a3f 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -13,8 +13,8 @@ # https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information project = 'InstrumentServer' -copyright = '2020-2026, Wolfgang Pfaff' -author = 'Wolfgang Pfaff' +copyright = '2020-2026, Tools For Experiments' +author = 'Tools For Experiments' # -- General configuration --------------------------------------------------- # https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration @@ -27,6 +27,7 @@ 'sphinx.ext.viewcode', # Add links to source code 'nbsphinx', # Jupyter notebook support 'sphinx.ext.intersphinx', # Link to other project docs + 'sphinx_design', # Tabs, cards, grids ] # MyST Parser configuration @@ -70,12 +71,12 @@ intersphinx_mapping = { 'python': ('https://docs.python.org/3', None), 'numpy': ('https://numpy.org/doc/stable/', None), - 'qcodes': ('https://qcodes.github.io/Qcodes/', None), - 'zmq': ('https://pyzmq.readthedocs.io/', None), + 'qcodes': ('https://microsoft.github.io/Qcodes/', None), + 'zmq': ('https://pyzmq.readthedocs.io/en/latest/', None), } templates_path = ['_templates'] -exclude_patterns = ['build', 'Thumbs.db', '.DS_Store', '**.ipynb_checkpoints'] +exclude_patterns = ['build', 'agents', 'README.md', 'Thumbs.db', '.DS_Store', '**.ipynb_checkpoints'] # -- Internationalization ---------------------------------------------------- # https://www.sphinx-doc.org/en/master/usage/configuration.html#internationalization diff --git a/docs/examples/index.md b/docs/examples/index.md deleted file mode 100644 index ee83549..0000000 --- a/docs/examples/index.md +++ /dev/null @@ -1,3 +0,0 @@ -# Code Examples - -This section contains practical examples demonstrating InstrumentServer usage patterns. diff --git a/docs/first_steps/index.md b/docs/first_steps/index.md deleted file mode 100644 index 4894947..0000000 --- a/docs/first_steps/index.md +++ /dev/null @@ -1,7 +0,0 @@ -# First Steps - -The following pages will help you get familiar with InstrumentServer. They provide a quick introduction to installation, basic server setup, and your first client connection. - -```{toctree} -overview -``` \ No newline at end of file diff --git a/docs/getting_started/how_it_works.md b/docs/getting_started/how_it_works.md new file mode 100644 index 0000000..ced45b2 --- /dev/null +++ b/docs/getting_started/how_it_works.md @@ -0,0 +1,79 @@ +# How It Works + +Laboratory hardware needs a single authoritative owner. If every script, GUI, and +Listener opened its own connection, they could disagree about the instrument's state or +send it conflicting commands. Instrumentserver keeps the real instrument in one +**Server** and gives each consumer a shared way to reach it. + +## Following the diagram + +The diagram follows one instrument, `generator`, from Server startup through a parameter +call and the resulting **Broadcast**. + +```{raw} html +:file: ../_static/animations/request_flow.html +``` + +### The Server owns the instrument + +A QCoDeS driver is a live software object that maintains a connection to an instrument +and translates parameter operations into commands the hardware understands. +Instrumentserver instantiates that driver once and keeps it alive in the Server, independently +of any individual script or GUI. In the diagram, `generator` is this real driver. + +The Server can create instruments from its configuration when it starts, as shown here, +or a Client can ask it to create one later. The creation path only determines when the +instrument becomes available. Once created, every Client reaches the same driver and the +same hardware connection. Closing one Client does not close the instrument or transfer +ownership to another Client. + +### The Client builds a Proxy Instrument + +The **Client** provides the initial connection to the Server and discovers which +instruments are available. When it asks for `generator`, the Server returns a +**Blueprint** rather than the real driver. The Blueprint describes the driver's public +interface, including its parameters, methods, and submodules. + +The Client uses this description to construct a **Proxy Instrument** inside your script. +The distinction between the Client and the Proxy is useful: the Client provides access +to the Server as a whole, while each Proxy represents one particular instrument. Your +code can work with the Proxy through the familiar QCoDeS interface without creating the +real driver or connecting to the hardware itself. + +The Blueprint is only the information needed to construct that local interface. The +Proxy is not a copy of the instrument or its state. It forwards operations to the one +real driver owned by the Server, which keeps every consumer working with the same +instrument. + +### A call reaches the hardware + +Calling `generator.frequency(5e9)` looks like an ordinary parameter operation in your +script, but the Proxy does not set anything locally. It forwards the operation to the +Server, which runs it on the real `generator` driver. The driver then translates the +parameter operation into the command understood by the RF source. + +From your script's perspective, the call remains a single operation. It completes when +the Server has finished interacting with the instrument and returned the result to the +Proxy. Parameter reads follow the same round trip, so they retrieve the Server's current +instrument state rather than relying on a separate local copy. + +### A Broadcast reaches subscribers + +The Proxy that made a call already receives its result directly, but other consumers may +also need to know that a parameter changed. The Server therefore publishes a +**Broadcast** that every listening subscriber can receive. One message can update many +consumers without each of them querying the instrument again. + +Broadcasts are especially useful for consumers that maintain a view or record of +instrument activity. The Server's GUI uses them to keep its displayed values current, +while a **Listener** can save the same activity elsewhere. These subscribers observe what +happened, but they do not take part in the original call or become owners of the +instrument. + +A Broadcast is an announcement, not another copy of the instrument's state. The Server +and its real driver remain authoritative, while subscribers use Broadcasts to keep their +own displays and records in sync. + +This overview leaves out the machinery beneath these paths. Continue to the +[Technical Guide](../technical_guide/architecture.md) for the request lifecycle, +networking, concurrency, and other implementation details. diff --git a/docs/getting_started/index.md b/docs/getting_started/index.md new file mode 100644 index 0000000..6659977 --- /dev/null +++ b/docs/getting_started/index.md @@ -0,0 +1,30 @@ +# Getting Started + +Start here if you're new to instrumentserver. These pages take you from installation +to a running Server and your first parameter call, then explain how the pieces work +together. + +Read them in order for a complete introduction: + +1. [Installation](installation.md) sets up instrumentserver and checks that its command + line tools are available. +2. [Quickstart](quickstart.md) starts a Server, creates a dummy instrument, and controls + it from a separate Python session. +3. [How it works](how_it_works.md) introduces Servers, Clients, Proxy Instruments, + Blueprints, and Broadcasts. + +:::{admonition} Legacy overview +:class: caution +The older [Instrumentserver Overview](overview.md) is still available because it covers +some Server networking, Parameter Manager, and configuration topics that the new guides +don't cover yet. It has not been re-verified and may be out of date. +::: + +```{toctree} +:maxdepth: 1 + +installation +quickstart +how_it_works +overview +``` diff --git a/docs/getting_started/installation.md b/docs/getting_started/installation.md new file mode 100644 index 0000000..5ae209e --- /dev/null +++ b/docs/getting_started/installation.md @@ -0,0 +1,107 @@ +# Installation + +:::{admonition} Not on PyPI yet +:class: note +instrumentserver is not published on PyPI yet (a release is planned). For now, every +install comes straight from the +[GitHub repository](https://github.com/toolsforexperiments/instrumentserver). +::: + +## Requirements + +You need **Python 3.11 or newer**. That's the same floor as QCoDeS, which instrumentserver is built on. + +The recommended starting point is a local clone of the repository. In a terminal, `cd` +to the directory where you want the clone to live (the command below creates an +`instrumentserver` folder right where you run it), then: + +```bash +git clone https://github.com/toolsforexperiments/instrumentserver.git +``` + +We recommend an editable installation of that clone: updating to the latest version is then +just a `git pull` away, and you can read (or tweak) the code you're actually running. +(If you'd rather skip the clone, the uv tab below has a clone-free alternative.) + +## Installing + +Pick the tab that matches how you manage your Python environments. We recommend uv. + +::::{tab-set} + +:::{tab-item} uv (recommended) +With [uv](https://docs.astral.sh/uv/), instrumentserver becomes a dependency of the +project you run your measurements from. From inside that project, add your local clone +as an editable dependency: + +```bash +uv add --editable path/to/instrumentserver +``` + +This records the dependency in your project's `pyproject.toml` and installs it into the +project environment. After a `git pull` in the clone, your project picks up the new +version automatically. + +If you'd rather not keep a local clone, you can add it as a git dependency instead: + +```bash +uv add git+https://github.com/toolsforexperiments/instrumentserver.git +``` + +Updating then means running `uv lock --upgrade-package instrumentserver`. +::: + +:::{tab-item} conda +We recommend one conda environment per measurement setup, with instrumentserver +installed alongside the rest of your measurement stack. Conda doesn't ship +instrumentserver as a package, so the installation goes through pip, inside the right +environment: + +```bash +conda activate your-measurement-env +pip install -e path/to/instrumentserver +``` + +Double-check which environment is active before installing. A correct installation in the +wrong environment is the classic way to end up with "but I installed it!" confusion. +::: + +:::{tab-item} pip + venv +The standard-library route: create a virtual environment, activate it, and install the +clone in editable mode. + +```bash +python -m venv .venv +source .venv/bin/activate # on Windows: .venv\Scripts\activate +pip install -e path/to/instrumentserver +``` +::: + +:::: + +## Optional: monitoring extra + +If you plan to export instrument parameter changes to InfluxDB (see the +[monitoring guide](../user_guide/monitoring.md)), install the `monitoring` extra, which +adds the `influxdb-client` package: + +```bash +pip install -e "path/to/instrumentserver[monitoring]" +``` + +Or with uv: + +```bash +uv add --editable path/to/instrumentserver --extra monitoring +``` + +## Check that it worked + +The installation puts five command line tools on your path. Ask the main one for help: + +```bash +instrumentserver --help +``` + +If you see the usage message, you're done. Head to the [quickstart](quickstart.md) to +start a server and talk to your first instrument. diff --git a/docs/first_steps/overview.md b/docs/getting_started/overview.md similarity index 97% rename from docs/first_steps/overview.md rename to docs/getting_started/overview.md index 5061a5d..f1df070 100644 --- a/docs/first_steps/overview.md +++ b/docs/getting_started/overview.md @@ -1,5 +1,11 @@ # Instrumentserver Overview +:::{admonition} 🗃️ Legacy page +:class: caution +This page predates the documentation refactor and has not been re-verified; parts of +it may be out of date. It will be replaced by the new Getting Started pages. +::: + The aim of Instrumentserver is to facilitate [QCoDeS](https://qcodes.github.io/Qcodes/) access across a variety of process and devices. We communicate with the server through a TCP/IP connection allowing us to talk to it from any independent process or separate device in the same network. @@ -8,11 +14,6 @@ Instrumentserver also includes a virtual instrument called Parameter Manager, wh single source of truth for various parameters values with a user-friendly graphical interface to facilitate changing parameters. -:::{warning} -This guide is not up to date. Some new core features are not currently documented like configuration files and -new features are in development. If you have questions on how to use these, please contact Marcos at: [marcosf2@illinois.edu](). -::: - ## Installation At the moment Instrumentserver is not on pip or conda so the only way of installing it is to install it from github directly. diff --git a/docs/getting_started/quickstart.md b/docs/getting_started/quickstart.md new file mode 100644 index 0000000..b1e4b89 --- /dev/null +++ b/docs/getting_started/quickstart.md @@ -0,0 +1,187 @@ +# Quickstart + +In the next few minutes you'll start a server, give it an instrument, and talk to that +instrument from a separate Python session. All you need is a working +[installation](installation.md). No hardware required: we'll use a dummy instrument +that ships with the package. + +## Start the server + +In a terminal (with your instrumentserver environment active), run: + +```bash +instrumentserver +``` + +A window opens: this is the Server, the process that will own all your instruments. +It's empty for now, and it's listening for clients. + +```{image} ../_static/getting_started/quickstart/server_bare_light.png +:class: only-light +:alt: The server window right after launch +``` + +```{image} ../_static/getting_started/quickstart/server_bare_dark.png +:class: only-dark +:alt: The server window right after launch +``` + +Leave it running. Everything else in this guide happens from a second terminal, a +notebook, or wherever you like to run Python. + +## Connect a client and create an instrument + +In Python, create a Client. With no arguments it connects to a server on +your own machine, which is exactly where the server is running: + +```python +from instrumentserver.client import Client + +cli = Client() +``` + +The server is still empty, so let's give it an instrument. We'll use the dummy RF +generator that ships with the package: + +```python +generator = cli.find_or_create_instrument( + "generator", + "instrumentserver.testing.dummy_instruments.rf.Generator", +) +``` + +Here's what just happened: the client asked the Server for an instrument named +`"generator"`. There wasn't one, so the Server created it from the import path you +gave, inside the server process. (If you've used QCoDeS, the name is no accident: this +is the same idea as qcodes' +[find_or_create_instrument](https://microsoft.github.io/Qcodes/api/instrument/index.html#qcodes.instrument.find_or_create_instrument), +except the instrument ends up in the Server, not in your own process.) + +What you get back is a Proxy Instrument. When it's created, the client asks the Server +what the instrument looks like (its parameters and methods) and builds a local object +with that same interface. So `generator` has everything the real driver has, and using +it forwards each call to the real instrument in the Server: + +```python +list(generator.parameters) +# ['IDN', 'frequency', 'power', 'rf_on'] +``` + +The new instrument also shows up in the server window the moment it's created: + +```{image} ../_static/getting_started/quickstart/server_generator_light.png +:class: only-light +:alt: The server window showing the generator and its parameters +``` + +```{image} ../_static/getting_started/quickstart/server_generator_dark.png +:class: only-dark +:alt: The server window showing the generator and its parameters +``` + +Because it's find *or* create, the same line is safe to run again: anyone who asks for +`"generator"` later, from this session or any other, gets a Proxy Instrument for the +one that already exists. That's the heart of instrumentserver: one process owns the +instrument, everyone else shares it. + +## Open the instrument's window + +The server window lists every instrument the Server owns. Double-click `generator` +and a window for the instrument opens, showing all of its parameters. + +```{image} ../_static/getting_started/quickstart/generator_widget_light.png +:class: only-light +:alt: The generator's instrument window with parameters at their initial values +``` + +```{image} ../_static/getting_started/quickstart/generator_widget_dark.png +:class: only-dark +:alt: The generator's instrument window with parameters at their initial values +``` + +Keep it open: it's about to prove a point. + +## Get and set a parameter + +Parameters work exactly like they do in QCoDeS: call with no arguments to get, call +with a value to set. + +```python +generator.frequency() +# 10000000000.0 + +generator.frequency(5e9) +generator.frequency() +# 5000000000.0 +``` + +Remember, the proxy forwards everything: both the get and the set were executed by the +real instrument inside the Server. If this were a physical generator, its output would +now actually be at 5 GHz. + +Now look at the generator window you left open: the frequency already shows the new +value. No refresh needed. Whenever anything changes on the Server, it announces the +change as a Broadcast, and every GUI subscribes and updates the moment it hears one. +You'll meet Broadcasts properly in [How it works](how_it_works.md). + +```{image} ../_static/getting_started/quickstart/generator_widget_set_light.png +:class: only-light +:alt: The generator's instrument window showing frequency at 5 GHz after the set +``` + +```{image} ../_static/getting_started/quickstart/generator_widget_set_dark.png +:class: only-dark +:alt: The generator's instrument window showing frequency at 5 GHz after the set +``` + +It works the other way too: edit a parameter in the generator window and your next +`generator.frequency()` in Python returns what you typed. Same instrument, any number +of views. + +## Starting with a config file + +Creating instruments from a client is handy for experimenting, but a real setup +shouldn't depend on someone re-running creation calls after every restart. The same +setup can be declared in a config file. Save this as `serverConfig.yml`: + +```yaml +instruments: + generator: + type: instrumentserver.testing.dummy_instruments.rf.Generator + initialize: True +``` + +Stop the server you started earlier (close its window) and start a new one with the +config: + +```bash +instrumentserver -c serverConfig.yml +``` + +The generator exists the moment the window appears, no client calls needed. Every +client that connects finds it ready to use. + +:::{note} +Instruments aren't limited to the generic parameter list you've seen so far. An +instrument can bring its own custom GUI, a purpose-built widget that the server window +shows in its place, to enable richer workflows than reading and setting parameters one +at a time. You wire one up with a `gui` entry in the config. See +[GUI features](../user_guide/gui_features.md) for how it works. +::: + +This single entry only scratches the surface of what the config file controls; the +[configuration guide](../user_guide/configuration.md) covers all of it. + +## Where to go next + +You've seen the whole loop: a Server owning instruments, clients reaching them through +Proxy Instruments, and every change broadcast to anyone watching. From here: + +- [Basic usage](../user_guide/basic_usage.md): the Python client in depth, the + interface you'll use the most. +- [The server](../user_guide/server.md): launch options, headless operation, and the + Detached GUI. +- [The Parameter Manager](../user_guide/parameter_manager.md): the instrument you just + configured, properly introduced. +- [How it works](how_it_works.md): what Proxy Instruments and Broadcasts actually are, + one level deeper. diff --git a/docs/index.md b/docs/index.md index 67b3774..82182a6 100644 --- a/docs/index.md +++ b/docs/index.md @@ -7,68 +7,59 @@ html_theme.sidebar_secondary.remove: true # InstrumentServer -**Distributed instrument control system for QCoDeS instruments via ZMQ** +Distributed control of QCoDeS instruments over ZMQ: one server owns the hardware, many +clients talk to it through proxies. Part of the +[Tools for Experiments](https://toolsforexperiments.github.io/) suite. -[GitHub Repository](https://github.com/toolsforexperiments/instrumentserver) | [About](about.md) +[GitHub Repository](https://github.com/toolsforexperiments/instrumentserver) | [About Us](https://toolsforexperiments.github.io/about_us/organization.html) -:::{warning} -The documentation site is currently under construction. This site is still very early so not all information here is up to date. Expect more changes incoming 🏗️👷 +:::{note} +We are rewriting this site page by page. Pages marked 🚧 are planned but not yet +written, and show what they will cover. ::: -## Overview +::::{grid} 1 2 2 2 +:gutter: 3 -InstrumentServer is a distributed system for remote access to QCoDeS instruments. It enables multi-client instrument control through a ZMQ-based server-client architecture with real-time parameter broadcasting and concurrent request handling. +:::{grid-item-card} Getting Started +:link: getting_started/index +:link-type: doc -For more information on how the server works please see our [overview page](./first_steps/overview.md) +Installation, your first server and client connection, and a conceptual overview +of how it all works. +::: -### Key Features +:::{grid-item-card} User Guide +:link: user_guide/index +:link-type: doc -**Multi-Client Access** -- Multiple clients can simultaneously control the same server -- Thread-safe per-instrument locking prevents race conditions -- Concurrent access to different instruments +How to use each feature: the Python client, the Server, the Parameter Manager, +Client Stations, monitoring, and configuration. +::: -**Real-Time Monitoring** -- Broadcast parameter changes to all listening clients -- Asynchronous parameter updates via ZMQ PUB socket -- Real-time GUI updates across the network +:::{grid-item-card} Technical Guide +:link: technical_guide/index +:link-type: doc -**QCoDeS Integration** -- Native QCoDeS Station support -- Full instrument metadata and blueprint system -- Seamless proxy objects for remote instruments +How instrumentserver works inside: architecture, Blueprints and proxies, Broadcasts, +and custom widgets. +::: -**Robust Architecture** -- ZMQ ROUTER/DEALER pattern for reliable messaging -- ThreadPoolExecutor for concurrent request handling -- Automatic connection recovery with retry logic +:::{grid-item-card} API Reference +:link: api/index +:link-type: doc + +Reference documentation for all public modules, generated from the source code. +::: -## Documentation +:::: ```{toctree} :maxdepth: 2 -:caption: Contents +:hidden: -first_steps/index +getting_started/index user_guide/index -``` - -## Code Examples - -```{toctree} -:maxdepth: 1 -:caption: Examples - -examples/index -``` - -## API Reference - -The API documentation is automatically generated from the source code. - -```{toctree} -:maxdepth: 1 -:caption: API Reference - +technical_guide/index api/index -``` \ No newline at end of file +``` diff --git a/docs/technical_guide/architecture.md b/docs/technical_guide/architecture.md new file mode 100644 index 0000000..ced3fe7 --- /dev/null +++ b/docs/technical_guide/architecture.md @@ -0,0 +1,13 @@ +# Architecture + +:::{admonition} 🚧 This page is planned, not yet written +:class: warning +It will be replaced with verified content as the documentation refactor progresses. +::: + +The deeper twin of the Getting Started "How It Works" page. This page will cover: + +- The ROUTER/DEALER request path; thread pool; per-instrument locks +- The PUB/SUB broadcast path; ports (request port, port+1) +- A request lifecycle walkthrough (set-parameter end to end) +- Process layout: server / detached GUI / clients / listeners diff --git a/docs/technical_guide/blueprints_and_proxies.md b/docs/technical_guide/blueprints_and_proxies.md new file mode 100644 index 0000000..cefa4d5 --- /dev/null +++ b/docs/technical_guide/blueprints_and_proxies.md @@ -0,0 +1,13 @@ +# Blueprints and Proxies + +:::{admonition} 🚧 This page is planned, not yet written +:class: warning +It will be replaced with verified content as the documentation refactor progresses. +::: + +The round trip as one story. This page will cover: + +- Server side: introspection to Blueprint +- The wire: serialization of blueprints and values +- Client side: Blueprint to dynamic proxy (methods, signatures, submodules) +- Blueprint caching and invalidation diff --git a/docs/technical_guide/broadcasts.md b/docs/technical_guide/broadcasts.md new file mode 100644 index 0000000..4a6c58e --- /dev/null +++ b/docs/technical_guide/broadcasts.md @@ -0,0 +1,12 @@ +# Broadcasts + +:::{admonition} 🚧 This page is planned, not yet written +:class: warning +It will be replaced with verified content as the documentation refactor progresses. +::: + +This page will cover: + +- What triggers a Broadcast; the message format +- SubClient mechanics; how GUIs stay live +- External broadcast forwarding diff --git a/docs/technical_guide/custom_widgets.md b/docs/technical_guide/custom_widgets.md new file mode 100644 index 0000000..287beec --- /dev/null +++ b/docs/technical_guide/custom_widgets.md @@ -0,0 +1,11 @@ +# Custom Widgets + +:::{admonition} 🚧 This page is planned, not yet written +:class: warning +It will be replaced with verified content as the documentation refactor progresses. +::: + +This page will cover: + +- How `gui.type` resolves to a widget class; the widget contract +- Writing and registering your own (worked example) diff --git a/docs/technical_guide/index.md b/docs/technical_guide/index.md new file mode 100644 index 0000000..e4ce2d3 --- /dev/null +++ b/docs/technical_guide/index.md @@ -0,0 +1,17 @@ +# Technical Guide + +:::{admonition} 🚧 This page is planned, not yet written +:class: warning +It will be replaced with verified content as the documentation refactor progresses. +::: + +How instrumentserver works inside, the deeper twin of the User Guide. + +```{toctree} +:maxdepth: 1 + +architecture +blueprints_and_proxies +broadcasts +custom_widgets +``` diff --git a/docs/user_guide/advanced/chaining_servers.md b/docs/user_guide/advanced/chaining_servers.md new file mode 100644 index 0000000..c4e9981 --- /dev/null +++ b/docs/user_guide/advanced/chaining_servers.md @@ -0,0 +1,12 @@ +# Chaining Servers + +:::{admonition} 🚧 This page is planned, not yet written +:class: warning +It will be replaced with verified content as the documentation refactor progresses. +::: + +This page will cover: + +- Concept: a Server as a Client of another Server +- A working setup walkthrough +- Limits and gotchas diff --git a/docs/user_guide/advanced/virtual_instruments.md b/docs/user_guide/advanced/virtual_instruments.md new file mode 100644 index 0000000..9dff313 --- /dev/null +++ b/docs/user_guide/advanced/virtual_instruments.md @@ -0,0 +1,11 @@ +# Virtual Instruments + +:::{admonition} 🚧 This page is planned, not yet written +:class: warning +It will be replaced with verified content as the documentation refactor progresses. +::: + +This page will cover: + +- The concept (vs Dummy Instruments) +- Writing your own Virtual Instrument diff --git a/docs/user_guide/basic_usage.md b/docs/user_guide/basic_usage.md new file mode 100644 index 0000000..14daac1 --- /dev/null +++ b/docs/user_guide/basic_usage.md @@ -0,0 +1,14 @@ +# Basic Usage + +:::{admonition} 🚧 This page is planned, not yet written +:class: warning +It will be replaced with verified content as the documentation refactor progresses. +::: + +The Python client, the most-used interface. This page will cover: + +- Client basics: connect, list, get instrument +- Proxy Instruments: parameters, methods, submodules +- Batch parameter operations: `getParamDict` / `setParameters` / to and from file +- Subscribing to Broadcasts (SubClient) +- Error handling, timeouts, reconnection behavior diff --git a/docs/user_guide/client_station.md b/docs/user_guide/client_station.md new file mode 100644 index 0000000..9da8dec --- /dev/null +++ b/docs/user_guide/client_station.md @@ -0,0 +1,13 @@ +# Client Station + +:::{admonition} 🚧 This page is planned, not yet written +:class: warning +It will be replaced with verified content as the documentation refactor progresses. +::: + +This page will cover: + +- Concept: scoped views of one shared Server +- YAML config; parameter save/load paths +- The Client Station GUI and the `instrumentserver-client-station` launcher +- Auto-reconnect behavior diff --git a/docs/user_guide/configuration.md b/docs/user_guide/configuration.md index 748604f..f17fc50 100644 --- a/docs/user_guide/configuration.md +++ b/docs/user_guide/configuration.md @@ -1,6 +1,10 @@ # Configuration Files -[//]: # (TODO: Make sure all of the comments are correct and up to date.) +:::{admonition} 🗃️ Legacy page +:class: caution +This page predates the documentation refactor and has not been re-verified; parts of +it may be out of date. It will be replaced by the new configuration reference. +::: This page covers the configuration files used by InstrumentServer: the **server configuration** file and the **listener configuration** file. diff --git a/docs/user_guide/gui_features.md b/docs/user_guide/gui_features.md new file mode 100644 index 0000000..918f20b --- /dev/null +++ b/docs/user_guide/gui_features.md @@ -0,0 +1,13 @@ +# GUI Features + +:::{admonition} 🚧 This page is planned, not yet written +:class: warning +It will be replaced with verified content as the documentation refactor progresses. +::: + +This page will cover: + +- Shared concepts: star / trash / hide / filter patterns +- Keyboard shortcuts: defaults, customizing via config +- Detachable tabs +- Custom instrument widgets: the config hook, with a link to the Technical Guide diff --git a/docs/user_guide/index.md b/docs/user_guide/index.md index fde0222..c66f6b7 100644 --- a/docs/user_guide/index.md +++ b/docs/user_guide/index.md @@ -3,6 +3,14 @@ This user guide is organized by different topics, each having their own guides. Use the left menu to navigate through them. ```{toctree} +basic_usage +server +gui_features +parameter_manager +client_station +monitoring configuration instrumentmonitoring +advanced/virtual_instruments +advanced/chaining_servers ``` \ No newline at end of file diff --git a/docs/user_guide/instrumentmonitoring.md b/docs/user_guide/instrumentmonitoring.md index 9a31dfc..8d6ad80 100644 --- a/docs/user_guide/instrumentmonitoring.md +++ b/docs/user_guide/instrumentmonitoring.md @@ -1,5 +1,11 @@ # Instrument Monitoring +:::{admonition} 🗃️ Legacy page +:class: caution +This page predates the documentation refactor and has not been re-verified; parts of +it may be out of date. It will be replaced by the new monitoring guide. +::: + The following is a guide to set up a dashboard for the monitoring of instruments. It contains capabilities for data storage, data visualization, and real-time alerts. More information on the tool is provided in the next section. :::{note} @@ -75,7 +81,7 @@ Install docker and start the docker engine. Follow the [docker section](#docker) On the same PC that Grafana and Influx were started on: - Keep track of the address and port that the instrumentserver is broadcasting to in the previous section. Use this information, the information you used to set up InfluxDB, plus the parameters you want to monitor to fill out the [config file](#config-file-1). + Keep track of the address and port that the instrumentserver is broadcasting to in the previous section. Use this information, the information you used to set up InfluxDB, plus the parameters you want to monitor to fill out the {ref}`config file `. You can then [start the listener](#starting-the-listener). @@ -170,6 +176,7 @@ The following portion assumes the user has: To use the dashboard, we will also need to run an instance of the listener on whichever computer you wish to host the dashboard on. (The computer with the listener and the computer with the instrumentserver must be on the same network). The listener can be used for writing data either in a CSV file or the InfluxDB database. +(listener-config-file)= ### Config File Below is an example listener configuration file that can be used for the dashboard. diff --git a/docs/user_guide/monitoring.md b/docs/user_guide/monitoring.md new file mode 100644 index 0000000..0b51800 --- /dev/null +++ b/docs/user_guide/monitoring.md @@ -0,0 +1,14 @@ +# Monitoring + +:::{admonition} 🚧 This page is planned, not yet written +:class: warning +It will be replaced with verified content as the documentation refactor progresses. +::: + +This page will cover: + +- Concept: Broadcasts → Listener → sink → dashboard +- Polling: making the Server emit without client activity (`pollingRate`) +- The Listener app and listenerConfig; CSV sink; InfluxDB sink +- Writing a custom Listener +- The deployment stack: Docker compose, Grafana + InfluxDB, provisioning, dashboards, alerting diff --git a/docs/user_guide/parameter_manager.md b/docs/user_guide/parameter_manager.md new file mode 100644 index 0000000..4f664ed --- /dev/null +++ b/docs/user_guide/parameter_manager.md @@ -0,0 +1,14 @@ +# Parameter Manager + +:::{admonition} 🚧 This page is planned, not yet written +:class: warning +It will be replaced with verified content as the documentation refactor progresses. +::: + +This page will cover: + +- Concept: the flagship Virtual Instrument; single source of truth +- Hierarchical parameters: add / remove / nesting +- Persistence: JSON files; profiles (refresh / switch) +- The Parameter Manager GUI and the `instrumentserver-param-manager` launcher +- Using it from measurement code diff --git a/docs/user_guide/server.md b/docs/user_guide/server.md new file mode 100644 index 0000000..916c917 --- /dev/null +++ b/docs/user_guide/server.md @@ -0,0 +1,13 @@ +# The Server + +:::{admonition} 🚧 This page is planned, not yet written +:class: warning +It will be replaced with verified content as the documentation refactor progresses. +::: + +This page will cover: + +- Starting the server: GUI, headless, CLI flags, addresses and ports +- The Detached GUI: what it's for (UI faults can't kill the Server) +- Instruments from config vs programmatic creation, and init scripts +- External broadcast: what it enables (live UIs), with a link to the Technical Guide diff --git a/pyproject.toml b/pyproject.toml index 377736a..43bde0c 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -42,6 +42,13 @@ instrumentserver-param-manager = "instrumentserver.apps:parameterManagerScript" [tool.setuptools] package-dir = {"" = "src"} +# Non-Python files inside the package are NOT picked up by packages.find and must be +# declared here, or wheels/sdists ship without them. instrumentserver/__init__.py opens +# schemas/parameters.json at import time, so a wheel without it cannot even be imported +# (editable installs hide this because they read straight from the source tree). +[tool.setuptools.package-data] +instrumentserver = ["schemas/*.json", "deployment/**/*"] + [tool.setuptools.packages.find] where = ["src"] @@ -110,5 +117,6 @@ docs = [ "pydata-sphinx-theme", "myst-parser", "nbsphinx", + "sphinx-design", "linkify-it-py", ] diff --git a/src/instrumentserver/blueprints.py b/src/instrumentserver/blueprints.py index 8b7b3c1..7cbcd54 100644 --- a/src/instrumentserver/blueprints.py +++ b/src/instrumentserver/blueprints.py @@ -796,10 +796,12 @@ def iterable_to_serialized_dict( returns a list with the args as serialized dictionaries. The current rules: - - Any arbitrary object that is being serialized here must have a class attribute listing all the classes attributes that - the constructor needs to create an identical instance of that class - - The serialized dictionary need to have the field: '_class_type', to indicate what it is that needs to be - instantiated. + + - Any arbitrary object that is being serialized here must have a class attribute + listing all the class attributes that the constructor needs to create an + identical instance of that class. + - The serialized dictionary needs to have the field '_class_type', to indicate + what it is that needs to be instantiated. """ converted_iterable: list | dict | None = None if iterable is not None: diff --git a/src/instrumentserver/gui/base_instrument.py b/src/instrumentserver/gui/base_instrument.py index 2e28793..f3e610e 100644 --- a/src/instrumentserver/gui/base_instrument.py +++ b/src/instrumentserver/gui/base_instrument.py @@ -50,11 +50,12 @@ **don't use insertItemTo** This should just be used to insert the correct number of items to the correct place. Things to pay attention when implementing your own: - * If your model is going to display more than one column (this is usually the case) you need to set the correct - number of columns and set the horizontal headers. - * Implement the function insertItemTo: This is the only function that actually adds items to the model. When the - model contains more than one column this function creates QStandardItems and adds them to the correct columns. - **don't forget** to emit the newItem signal if you are going to implement a view that utilizes delegates. + +* If your model is going to display more than one column (this is usually the case) you need to set the correct + number of columns and set the horizontal headers. +* Implement the function insertItemTo: This is the only function that actually adds items to the model. When the + model contains more than one column this function creates QStandardItems and adds them to the correct columns. + **don't forget** to emit the newItem signal if you are going to implement a view that utilizes delegates. InstrumentSortFilterProxyModel ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -299,7 +300,7 @@ def insertItemTo( def addItem(self, fullName: str, **kwargs: Any) -> "ItemBase": """ - Adds an item to the model. The *args and **kwargs are whatever the specific item needs for a new item. + Adds an item to the model. The ``*args`` and ``**kwargs`` are whatever the specific item needs for a new item. :param fullName: The name of the parameter """ diff --git a/src/instrumentserver/gui/instruments.py b/src/instrumentserver/gui/instruments.py index b96805a..d8c0aec 100644 --- a/src/instrumentserver/gui/instruments.py +++ b/src/instrumentserver/gui/instruments.py @@ -313,8 +313,8 @@ def createEditor( # type: ignore[override] class ModelParameters(InstrumentModelBase): - # : Signal(item, object) : Emitted when an item in the model has received a new value, first object is the item's - # name, second object is its new value + #: Signal(item, object) : Emitted when an item in the model has received a new value, first object is the item's + #: name, second object is its new value itemNewValue = QtCore.Signal(object, object) def __init__(self, *args: Any, **kwargs: Any) -> None: diff --git a/src/instrumentserver/log.py b/src/instrumentserver/log.py index 2c0b5ca..9819b13 100644 --- a/src/instrumentserver/log.py +++ b/src/instrumentserver/log.py @@ -30,6 +30,7 @@ class QLogHandler(QtCore.QObject, logging.Handler): logging.DEBUG: QtGui.QColor("gray"), } + #: Signal(str) : Emitted with the html-formatted log record to append to the widget new_html = QtCore.Signal(str) def __init__(self, parent: Optional[QtWidgets.QWidget]) -> None: diff --git a/src/instrumentserver/server/application.py b/src/instrumentserver/server/application.py index 608082b..86aff31 100644 --- a/src/instrumentserver/server/application.py +++ b/src/instrumentserver/server/application.py @@ -177,6 +177,8 @@ class CreateInstrumentDialog(BaseDialog): :param kwargsStr: Optional, String with te args and kwargs separated by commas. """ + #: Signal(str, str, tuple) -- emitted when the dialog is accepted. Arguments are + #: the instrument type path, the instrument name, and the constructor args. createInstrument = QtCore.Signal(str, str, tuple) def __init__( @@ -290,18 +292,18 @@ class PossibleInstrumentsDisplay(QtWidgets.QTreeWidget): the config or are the original args and kwargs passed when the instrument was created. """ - #: Signal(str, str, str) -- emitted when the one of the create buttons of the items gets pressed + #: Signal(str, str, str) -- emitted when the one of the create buttons of the items gets pressed. #: Arguments are in order: - # The name of the instrument in the config, - # the type of the instrument, - # the name in the line edit indicating what the actual name in the station should be. + #: the name of the instrument in the config, + #: the type of the instrument, + #: the name in the line edit indicating what the actual name in the station should be. createButtonPressed = QtCore.Signal(str, str, str) - #: Signal(str, str, str) -- emitted when the create instrument based on this instrument is triggered + #: Signal(str, str, str) -- emitted when the create instrument based on this instrument is triggered. #: Arguments are in order: - # The name of the instrument in the config, - # the type of the instrument, - # the name in the line edit indicating what the actual name in the station should be. + #: the name of the instrument in the config, + #: the type of the instrument, + #: the name in the line edit indicating what the actual name in the station should be. basedInstrumentRequested = QtCore.Signal(str, str, str) cols = ["Instrument Type & Preset", "Instrument Name", "Create Instrument"] @@ -591,6 +593,7 @@ def createNewInstrument( class ServerGui(QtWidgets.QMainWindow): """Main window of the qcodes station server.""" + #: Signal(int) -- declared but currently never emitted or connected. serverPortSet = QtCore.Signal(int) def __init__( diff --git a/src/instrumentserver/testing/create_instrument.py b/src/instrumentserver/testing/create_instrument.py deleted file mode 100644 index bb7b366..0000000 --- a/src/instrumentserver/testing/create_instrument.py +++ /dev/null @@ -1,17 +0,0 @@ -from instrumentserver.client import Client as InstrumentClient - -""" -Script used to create an instrument in the instrument server used for developing the dashboard/logger. -""" - - -# used for testing, the instruments should be already created for the dashboard to work -cli = InstrumentClient() - -if "test" in cli.list_instruments(): - instrument = cli.get_instrument("test") -else: - instrument = cli.find_or_create_instrument( - "test" - "instrumentserver.testing.dummy_instruments.generic.DummyInstrumentRandomNumber", - ) diff --git a/src/instrumentserver/testing/dummy_instruments/generic.py b/src/instrumentserver/testing/dummy_instruments/generic.py index b0a89c3..bcb7696 100644 --- a/src/instrumentserver/testing/dummy_instruments/generic.py +++ b/src/instrumentserver/testing/dummy_instruments/generic.py @@ -91,7 +91,7 @@ def close(self) -> None: super().close() def ask_raw(self, cmd): - """Dummy ask_raw so *IDN? and similar SCPI queries don't explode the GUI.""" + """Dummy ask_raw so ``*IDN?`` and similar SCPI queries don't explode the GUI.""" if cmd.strip().upper().startswith("*IDN"): return f"dummy,{self.name},0,0" return "" diff --git a/src/instrumentserver/testing/dummy_instruments/rf.py b/src/instrumentserver/testing/dummy_instruments/rf.py index 5d4e7ea..b848caf 100644 --- a/src/instrumentserver/testing/dummy_instruments/rf.py +++ b/src/instrumentserver/testing/dummy_instruments/rf.py @@ -109,6 +109,13 @@ def __init__(self, name: str, f0: float = 5e9, df: float = 1e6, **kw: Any) -> No get_cmd=self._get_data, ) + def ask_raw(self, cmd: str) -> str: + """Dummy ``ask_raw`` so ``*IDN?`` and similar SCPI queries don't log a + scary traceback when qcodes asks for the IDN on instrument creation.""" + if cmd.strip().upper().startswith("*IDN"): + return f"dummy,{self.name},0,0" + return "" + def modulate_frequency(self, delta: float = 0, multiply: bool = False) -> None: """Add an offset to the resonance frequency. @@ -199,6 +206,13 @@ def __init__(self, name: str, *arg: Any, **kw: Any) -> None: "rf_on", set_cmd=None, vals=validators.Bool(), initial_value=False ) + def ask_raw(self, cmd: str) -> str: + """Dummy ``ask_raw`` so ``*IDN?`` and similar SCPI queries don't log a + scary traceback when qcodes asks for the IDN on instrument creation.""" + if cmd.strip().upper().startswith("*IDN"): + return f"dummy,{self.name},0,0" + return "" + class FluxControl(Instrument): """A dummy that hooks to :class:`.ResonatorResponse` and modifies its @@ -228,6 +242,13 @@ def __init__( initial_value=0, ) + def ask_raw(self, cmd: str) -> str: + """Dummy ``ask_raw`` so ``*IDN?`` and similar SCPI queries don't log a + scary traceback when qcodes asks for the IDN on instrument creation.""" + if cmd.strip().upper().startswith("*IDN"): + return f"dummy,{self.name},0,0" + return "" + def _set_flux(self, flux: float) -> None: mod = 1.0 / ( 1.0 + self.inductive_participation_ratio() / np.abs(np.cos(np.pi * flux)) diff --git a/test/docs_verification/README.md b/test/docs_verification/README.md new file mode 100644 index 0000000..1d0e616 --- /dev/null +++ b/test/docs_verification/README.md @@ -0,0 +1,33 @@ +# Docs verification scripts + +Scripts that back every behavioral claim on the documentation site +(see `PLAN_docs_refactor.md` at the repo root). Nothing is documented without +being executed: each docs page has one script here that exercises the behavior +the page describes, before a word of prose is written. + +These scripts are temporary. They live here for the duration of the docs +refactor and are deleted in the final cleanup phase, after the test audit in +`TEST_AUDIT.md` has been fully harvested. + +## Conventions + +- **One script per page**, named `verify_.py`, placed in a subdirectory + matching the page's docs area (for example + `getting_started/verify_quickstart.py`). +- **One clearly marked section per page section**, in the same order as the + page, with a comment header naming the section. +- **Runnable standalone**: each script asserts the documented behavior and + exits 0 on success. No pytest required, no arguments required. +- **Use the shared helpers** in `helpers.py` for server startup/shutdown, + client creation, and Broadcast capture. Scripts never hand-roll server + startup; if a script needs something the helpers lack, extend the helpers. +- **Terminology** in comments and assertions follows `CONTEXT.md` at the repo + root. + +## Relationship to the test suite + +These scripts verify behavior for documentation purposes; they are not the +test suite. At every page section's AUDIT step, the behavior verified here is +checked against the pytest suite under `test/`, and gaps are recorded in +`TEST_AUDIT.md`. Helpers that prove broadly useful are candidates to graduate +into pytest fixtures during the audit harvest. diff --git a/test/docs_verification/getting_started/quickstartConfig.yml b/test/docs_verification/getting_started/quickstartConfig.yml new file mode 100644 index 0000000..4472262 --- /dev/null +++ b/test/docs_verification/getting_started/quickstartConfig.yml @@ -0,0 +1,6 @@ +# The exact config file shown in docs/getting_started/quickstart.md +# (section: Starting with a config file). Keep the two in sync. +instruments: + generator: + type: instrumentserver.testing.dummy_instruments.rf.Generator + initialize: True diff --git a/test/docs_verification/getting_started/verify_quickstart.py b/test/docs_verification/getting_started/verify_quickstart.py new file mode 100644 index 0000000..b172168 --- /dev/null +++ b/test/docs_verification/getting_started/verify_quickstart.py @@ -0,0 +1,133 @@ +"""Verification script for docs/getting_started/quickstart.md. + +One section below per page section, in page order (see +test/docs_verification/README.md for conventions). Asserts every behavioral +claim the quickstart makes; exits 0 on success. + +GUI claims (the server window opening, the live parameter update, the +double-click-to-open generic instrument window and editing parameters from +it, the Parameter Manager widget embedded in the server window) cannot be +asserted here; they are verified manually and captured in the page's +screenshots. +""" + +import sys +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).parent.parent)) + +from helpers import capture_broadcasts, client, server, server_process + +GENERATOR_CLASS = "instrumentserver.testing.dummy_instruments.rf.Generator" +CONFIG = str(Path(__file__).parent / "quickstartConfig.yml") + + +# --------------------------------------------------------------------------- +# Section: Start the server (bare, no config file) +# +# Page claim: running plain `instrumentserver` starts a Server with no +# instruments, ready to take client connections. (The page shows the GUI +# variant; the CLI subprocess here is the same launcher, headless.) +# --------------------------------------------------------------------------- +def section_start_bare() -> None: + with server_process(): + with client() as cli: + # NOTE: list_instruments() is annotated Dict[str, str] but the + # server actually answers with a list of names (audit entry). + instruments = cli.list_instruments() + assert instruments == [], ( + f"A bare server should start with no instruments, got {instruments!r}" + ) + print("section_start_bare: OK") + + +# --------------------------------------------------------------------------- +# Section: First client connection; create a Dummy Instrument +# +# Page claims: a Client connects to a running server on the default port; +# find_or_create_instrument creates the dummy RF generator on the Server and +# returns a Proxy Instrument; the generator comes up with its documented +# initial values; calling find_or_create_instrument again finds the existing +# instrument instead of failing. +# --------------------------------------------------------------------------- +def section_first_client() -> None: + with server(): + with client() as cli: + generator = cli.find_or_create_instrument("generator", GENERATOR_CLASS) + + assert "generator" in cli.list_instruments() + + # The Proxy Instrument mirrors the real instrument's interface; + # the page shows this exact parameter list. + assert sorted(generator.parameters) == [ + "IDN", + "frequency", + "power", + "rf_on", + ], sorted(generator.parameters) + + assert generator.frequency() == 10e9 + assert generator.power() == -100 + assert generator.rf_on() is False + + # Second call: finds, does not re-create or fail. + again = cli.find_or_create_instrument("generator", GENERATOR_CLASS) + assert again.frequency() == 10e9 + print("section_first_client: OK") + + +# --------------------------------------------------------------------------- +# Section: Get and set a parameter; the Server broadcasts the change +# +# Page claims: parameters are read by calling them and set by calling them +# with a value; every change on the Server goes out as a Broadcast (which is +# why the GUI follows along live without refreshing). +# --------------------------------------------------------------------------- +def section_get_set_broadcast() -> None: + with server(): + with client() as cli: + generator = cli.find_or_create_instrument("generator", GENERATOR_CLASS) + + with capture_broadcasts() as cap: + generator.frequency(5e9) + messages = cap.wait_for(1) + + assert generator.frequency() == 5e9 + text = repr(messages) + assert "frequency" in text, ( + f"Expected a Broadcast about 'frequency', got: {text}" + ) + print("section_get_set_broadcast: OK") + + +# --------------------------------------------------------------------------- +# Section: Starting with a config file +# +# Page claims: the same setup can be declared in YAML; an instrument marked +# initialize: True exists as soon as the server is up, no client call needed. +# Uses the exact config file the page shows (quickstartConfig.yml). +# --------------------------------------------------------------------------- +def section_config_file() -> None: + with server_process(config=CONFIG): + with client() as cli: + instruments = cli.list_instruments() + assert "generator" in instruments, instruments + + # The declared generator is the same instrument the page created + # programmatically earlier. + generator = cli.get_instrument("generator") + assert generator.frequency() == 10e9 + print("section_config_file: OK") + + +# --------------------------------------------------------------------------- +# Section: Where to go next — links only, nothing to verify. +# --------------------------------------------------------------------------- + + +if __name__ == "__main__": + section_start_bare() + section_first_client() + section_get_set_broadcast() + section_config_file() + print("verify_quickstart: all sections OK") diff --git a/test/docs_verification/helpers.py b/test/docs_verification/helpers.py new file mode 100644 index 0000000..e596080 --- /dev/null +++ b/test/docs_verification/helpers.py @@ -0,0 +1,224 @@ +"""Shared helpers for docs verification scripts. + +The single canonical way verification scripts start servers, get clients, and +capture Broadcasts (see README.md in this directory). Mirrors the pytest +fixtures in test/pytest/conftest.py wherever one exists; helpers that prove +broadly useful are candidates to graduate into fixtures during the audit +harvest. + +Two ways to run a server: + +- ``server(...)``: in-process, wrapping ``startServer()`` on a QThread exactly + like the ``start_server`` pytest fixture. Fast, clean teardown, and the + yielded ``StationServer`` object can be inspected directly. +- ``server_process(...)``: the real ``instrumentserver`` CLI as a subprocess, + for sections whose claims are about launch behavior itself. + +Both default to port 5555 (the package's ``DEFAULT_PORT``, what every docs +example shows) and fail immediately with a clear message if the port is taken. +""" + +import socket +import subprocess +import time +from contextlib import contextmanager +from typing import Any, Iterator, List, Optional, Sequence + +import qcodes as qc + +from instrumentserver import DEFAULT_PORT, QtCore, QtWidgets +from instrumentserver.client.core import BaseClient +from instrumentserver.client.proxy import Client, SubClient +from instrumentserver.server.core import StationServer, startServer + +#: Import path of the standard test instrument, same as the pytest fixtures use. +DUMMY_INSTRUMENT = ( + "instrumentserver.testing.dummy_instruments.generic.DummyInstrumentWithSubmodule" +) + + +#: Keeps the QApplication alive for the whole script. Without a held +#: reference it gets garbage-collected, destroying every QObject with it. +_qapp: Any = None + + +def ensure_qapp() -> Any: + """Make sure a QApplication exists (QThread, used by startServer, needs one).""" + global _qapp + app = QtWidgets.QApplication.instance() + if app is None: + app = QtWidgets.QApplication([]) + _qapp = app + return app + + +def _require_port_free(port: int) -> None: + """Fail loudly if ``port`` is already bound (e.g. a leftover server).""" + with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock: + sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + try: + sock.bind(("127.0.0.1", port)) + except OSError as e: + raise RuntimeError( + f"Port {port} is already in use. Is another instrumentserver " + f"(or a leftover one) running? Stop it or pass a different port." + ) from e + + +@contextmanager +def server( + config: Optional[str] = None, port: int = DEFAULT_PORT, **kwargs: Any +) -> Iterator[StationServer]: + """Run an in-process server, shut down cleanly on exit. + + :param config: optional path to a station config YAML, passed to + ``startServer(stationConfig=...)``. + :param port: request port; Broadcasts go out on ``port + 1``. + :param kwargs: forwarded to ``startServer()``. + """ + _require_port_free(port) + _require_port_free(port + 1) + ensure_qapp() + srv, thread = startServer(port=port, stationConfig=config, **kwargs) + try: + yield srv + finally: + # The zmq loop in StationServer blocks on poll(); ask it to shut + # itself down via the SAFEWORD, then wait for the thread to exit. + # (Same dance as the start_server pytest fixture.) + try: + with BaseClient(port=port) as shutdown_cli: + shutdown_cli.ask(srv.SAFEWORD) + except Exception: + pass + thread.wait(5000) + thread.deleteLater() + # Clean the qcodes instrument registry so a script can start more + # than one server without name collisions. + qc.Instrument.close_all() + + +@contextmanager +def server_process( + config: Optional[str] = None, + port: int = DEFAULT_PORT, + extra_args: Sequence[str] = (), + startup_timeout: float = 15.0, +) -> Iterator["subprocess.Popen[str]"]: + """Run the real ``instrumentserver`` CLI (headless) as a subprocess. + + Waits until the request port accepts connections before yielding. The + server's SAFEWORD is randomized per process, so teardown terminates the + process instead of asking politely. + + :param config: optional config file path, passed as ``-c``. + :param port: request port, passed as ``-p``. + :param extra_args: additional CLI arguments, appended verbatim. + """ + _require_port_free(port) + _require_port_free(port + 1) + cmd = ["instrumentserver", "--gui", "False", "-p", str(port)] + if config is not None: + cmd += ["-c", config] + cmd += list(extra_args) + proc = subprocess.Popen( + cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True + ) + deadline = time.monotonic() + startup_timeout + while True: + if proc.poll() is not None: + out = proc.stdout.read() if proc.stdout else "" + raise RuntimeError( + f"instrumentserver exited during startup " + f"(code {proc.returncode}). Output:\n{out}" + ) + try: + with socket.create_connection(("127.0.0.1", port), timeout=0.2): + break + except OSError: + if time.monotonic() > deadline: + proc.terminate() + raise RuntimeError( + f"instrumentserver did not open port {port} " + f"within {startup_timeout}s" + ) + time.sleep(0.1) + try: + yield proc + finally: + proc.terminate() + try: + proc.wait(5) + except subprocess.TimeoutExpired: + proc.kill() + proc.wait(5) + + +@contextmanager +def client( + host: str = "localhost", port: int = DEFAULT_PORT, **kwargs: Any +) -> Iterator[Client]: + """Yield a connected ``Client``, disconnected on exit.""" + cli = Client(host=host, port=port, **kwargs) + try: + yield cli + finally: + cli.disconnect() + + +class BroadcastCapture: + """Collects everything a ``SubClient`` receives. Use via ``capture_broadcasts``.""" + + def __init__(self) -> None: + self.messages: List[Any] = [] + + def _collect(self, message: Any) -> None: + # Runs in the SubClient's thread (DirectConnection); list.append is + # atomic under the GIL, so no lock is needed. + self.messages.append(message) + + def wait_for(self, n: int = 1, timeout: float = 5.0) -> List[Any]: + """Block until at least ``n`` messages arrived; return them all.""" + deadline = time.monotonic() + timeout + while len(self.messages) < n: + if time.monotonic() > deadline: + raise TimeoutError( + f"Expected {n} Broadcast(s) within {timeout}s, " + f"got {len(self.messages)}: {self.messages!r}" + ) + time.sleep(0.05) + return list(self.messages) + + +@contextmanager +def capture_broadcasts( + instruments: Optional[List[str]] = None, + host: str = "localhost", + port: int = DEFAULT_PORT + 1, +) -> Iterator[BroadcastCapture]: + """Capture Broadcasts through a ``SubClient`` (the intended live-update path). + + Runs the SubClient on its own QThread, like real consumers do, and yields + a ``BroadcastCapture`` whose ``messages`` list fills up live. + + :param instruments: instrument names to subscribe to; None means all. + :param port: the Broadcast port (request port + 1). + """ + ensure_qapp() + capture = BroadcastCapture() + sub = SubClient(instruments=instruments, sub_host=host, sub_port=port) + sub.update.connect(capture._collect, QtCore.Qt.DirectConnection) + thread = QtCore.QThread() + sub.moveToThread(thread) + thread.started.connect(sub.connect) + sub.finished.connect(thread.quit) + thread.start() + # PUB/SUB slow-joiner: give the SUB socket a moment to connect before the + # caller triggers the Broadcasts it wants to observe. + time.sleep(0.3) + try: + yield capture + finally: + sub.stop() + thread.wait(2000) + thread.deleteLater() diff --git a/uv.lock b/uv.lock index 476f11b..fba690a 100644 --- a/uv.lock +++ b/uv.lock @@ -872,6 +872,7 @@ docs = [ { name = "pydata-sphinx-theme" }, { name = "sphinx", version = "9.0.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.12'" }, { name = "sphinx", version = "9.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, + { name = "sphinx-design" }, ] [package.metadata] @@ -904,6 +905,7 @@ docs = [ { name = "nbsphinx" }, { name = "pydata-sphinx-theme" }, { name = "sphinx" }, + { name = "sphinx-design" }, ] [[package]] @@ -2765,6 +2767,19 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/73/f7/b1884cb3188ab181fc81fa00c266699dab600f927a964df02ec3d5d1916a/sphinx-9.1.0-py3-none-any.whl", hash = "sha256:c84fdd4e782504495fe4f2c0b3413d6c2bf388589bb352d439b2a3bb99991978", size = 3921742, upload-time = "2025-12-31T15:09:25.561Z" }, ] +[[package]] +name = "sphinx-design" +version = "0.7.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "sphinx", version = "9.0.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.12'" }, + { name = "sphinx", version = "9.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/13/7b/804f311da4663a4aecc6cf7abd83443f3d4ded970826d0c958edc77d4527/sphinx_design-0.7.0.tar.gz", hash = "sha256:d2a3f5b19c24b916adb52f97c5f00efab4009ca337812001109084a740ec9b7a", size = 2203582, upload-time = "2026-01-19T13:12:53.297Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/30/cf/45dd359f6ca0c3762ce0490f681da242f0530c49c81050c035c016bfdd3a/sphinx_design-0.7.0-py3-none-any.whl", hash = "sha256:f82bf179951d58f55dca78ab3706aeafa496b741a91b1911d371441127d64282", size = 2220350, upload-time = "2026-01-19T13:12:51.077Z" }, +] + [[package]] name = "sphinxcontrib-applehelp" version = "2.0.0"