From 74f08409d1e16c72d2f9c630195d2160b9cf638d Mon Sep 17 00:00:00 2001 From: Yernat Yestekov <2068106+doublewhy@users.noreply.github.com> Date: Fri, 9 Oct 2026 02:48:47 -0700 Subject: [PATCH 1/4] docs(sdl): assess VS Code support for SDL authoring Add a feasibility note for issue 1412 under docs/research. It records what raes.language_service supports, what a VS Code integration needs, what a prototype extension outside this repository demonstrates, and the gaps an editor exposes, including the completion mismatch tracked in issue 1339. It recommends one language server over the existing helpers with thin editor clients, starting with diagnostics, reference completion and go to definition, and gives a verdict on each of the issue's four alternatives. Add an evidence record beside it with the probe inputs and the outputs observed at 35122105, plus the source locations behind the claims that no probe shows. The note's gap table cites that record; the prototype link remains only for claims about the editor client. --- docs/research/sdl-editor-support/evidence.md | 162 ++++++++++++ .../sdl-editor-support/feasibility.md | 235 ++++++++++++++++++ 2 files changed, 397 insertions(+) create mode 100644 docs/research/sdl-editor-support/evidence.md create mode 100644 docs/research/sdl-editor-support/feasibility.md diff --git a/docs/research/sdl-editor-support/evidence.md b/docs/research/sdl-editor-support/evidence.md new file mode 100644 index 0000000000..ca3f672fc1 --- /dev/null +++ b/docs/research/sdl-editor-support/evidence.md @@ -0,0 +1,162 @@ +# SDL editor support evidence + +Observed 2026-10-09 on repository revision +`35122105b0c1bb648754625d8e3920eafa9bc599`. The +[feasibility note](feasibility.md) interprets these observations. The probes +called the language-service helpers in-process, with the working tree at that +revision and the repository Python environment (CPython 3.14.4, free-threaded). +Nothing was mocked, and no scenario was launched. The probes establish helper +and schema behavior on small inputs. They are not a usability, conformance or +security result. + +## Minimal probe inputs + +Specimen N is a valid document with one feature and one node: + +```yaml +name: demo +features: + app: {type: service, source: webapp} +nodes: + web: + type: compute + os: linux + resources: {ram: 2 GiB, cpu: 1} + features: [app] +``` + +Specimen P is a valid document with a participant relationship: + +```yaml +name: audit +entities: + team: {role: red} +agents: + one: {affiliations: [team]} + two: {affiliations: [team]} +propositions: + ok: + description: declared state + subjects: [agents.one] + basis: declared_state + predicate: + kind: boolean + property: ready + semantic_ref: urn:raes:declared-property:ready + operator: equals + expected: true +assertions: + done: {proposition: ok, role: postcondition, polarity: positive} +relationships: + pair: + type: participant + source: agents.one + target: agents.two + participant: {kind: cooperation} +``` + +Specimen M is a valid document in which every value has its long form: + +```yaml +name: demo +features: + app: {type: service, source: {name: webapp, version: '*'}} +nodes: + web: {type: compute, os: linux, resources: {ram: 2 GiB, cpu: 1}, features: {app: client}, roles: {client: {username: www}}} +``` + +Each row below names the edit that it applies to a specimen. Lines and columns +are 1-based, as the helpers report them. + +## Language-service observations + +| ID | Operation | Observed result | Limit | +| --- | --- | --- | --- | +| L1 | `language_completions(P, cursor_path="/relationships/pair/target")`, then `language_diagnostics` of P with each offered `insert_text` as the target | Context `reference:targetable`, with six items: `agents.one`, `agents.two`, `assertions.done`, `entities.team`, `propositions.ok` and `relationships.pair`. Only `two` is valid. `one` fails with "Relationship 'pair' requires distinct participants". The other four fail because the target must "resolve unambiguously to a declared agent". | Finding F5 in the [participant audit](../participant-identity/audit.md) reports the same mismatch, and #1339 tracks it | +| L2 | Compare each list in `SECTION_FIELD_COMPLETIONS` with the property names that the published schema declares for that section | 18 of the 33 lists differ from the schema. Together they omit 69 schema fields. The `features` list also offers `version`, which the schema does not declare. The table below lists the differences. | The comparison merges every union branch. Seven schema sections have no list. | +| L3 | `language_completions` on a document whose only feature is `app: {type: service}`, with `cursor_path="/features/app"` and `prefix="v"`; then `language_diagnostics` with `version: '1.0'` added to that feature | Completion offers `version`. Diagnostics report `sdl.model.invalid`, "Extra inputs are not permitted", at `/features/app/version`. | | +| L4 | `language_completions(text, cursor_path="/nodes/web/features")` on N and on four edits of N | N gives `ok` with `app`. `features:` with no value gives `ok` with `app`. `features: [` gives `invalid` with `sdl.parse`. `fea` in place of the `features` line gives `invalid` with `sdl.parse`. `type: VM` gives `invalid` with `sdl.legacy_node_type_vm`. | A parse failure anywhere in the document removes every item | +| L5 | `language_diagnostics` on four documents | Adding `colour: blue` under `web` in N gives `sdl.model.invalid` at `/nodes/web/colour`, on line 10, columns 13 to 17. The key starts at column 5. `features: [missing]` in N gives `sdl.semantic` at the `semantic_validation` stage, with no path and no range. `name: demo` followed by `name: again` gives `sdl.mapping_key_conflict` at `/name`, on line 2, columns 1 to 5. An unclosed `web: {type: compute` gives `sdl.parse` with an empty path and one point, line 4, column 1. | The point in the last case is the end of the input | +| L6 | `language_format` on a document with three comments and the feature `app: {type: Service, source: webapp}` | Status `formatted`. The output has no comments. `type: Service` becomes `type: service`, and `source: webapp` becomes a mapping of `name: webapp` and `version: '*'`. | | +| L7 | `apply_structured_edit` on N with two added comments, operation `set`, pointer `/nodes/web/os`, value `windows` | Status `edited`. The output has no comments. `source: webapp` expands as in L6, and `features: [app]` becomes the mapping `app: ''`. | | +| L8 | `parse_sdl_file` and `language_diagnostics` on `contracts/fixtures/sdl/variation-points-v1/composition/root.yaml` | `parse_sdl_file` returns an `ExpandedScenario`. On the same text, `language_diagnostics` returns `sdl.parse`, "SDL imports require file-backed parsing via parse_sdl_file()". Turning off `semantic_validation` gives the same result. | | +| L9 | File sizes of the tracked `*.sdl.yaml` files, against `_MAX_INPUT_BYTES` | The limit is 65,536 bytes. None of the 40 files is over it. The largest is `examples/scenarios/hospital-ransomware-surgery-day.sdl.yaml`, at 46,073 bytes. | | +| L10 | `language_completions(N, cursor_path=path)` for `/nodes`, `/nodes/web`, `/nodes/web/resources` and `/nodes/web/os` | Each call returns context `section:nodes` and the same nine fields: `type`, `description`, `os`, `resources`, `features`, `conditions`, `services`, `roles` and `endpoint_persona`. | | +| L11 | Median of 5 warm calls of `language_diagnostics`, and of `language_completions` at `/nodes`, on the L9 file. Median of 3 fresh interpreters that import `raes` and make one diagnostics call. | At load average 7.2: 172 ms and 159 ms warm, and 1,207 ms cold. At 13.1: 174 ms and 163 ms warm, and 1,297 ms cold. At 22.5: 377 ms and 373 ms warm, and 2,939 ms cold. | An 8-core Apple M3 that other work shared. The timings are indicative. | + +L2 counts, for each list, the distinct property names that the schema declares +for one entry of the section. For a section that is one object, such as +`realization`, it counts that object's properties. The count follows `$ref` and +merges `anyOf`, `oneOf` and `allOf` branches. These lists differ: + +| Section | Offered | In schema | Schema fields not offered | +| --- | --- | --- | --- | +| `accounts` | 7 | 14 | `description`, `disabled`, `groups`, `home`, `mail`, `materialization_profile`, `shell` | +| `agents` | 7 | 12 | `allowed_subnets`, `authority_anchors`, `description`, `observation_boundaries`, `operating_scope` | +| `behavior_specifications` | 17 | 18 | `autonomous_execution` | +| `conditions` | 5 | 10 | `environment`, `name`, `retries`, `start_period`, `timeout` | +| `content` | 7 | 13 | `description`, `destination`, `sensitive`, `tags`, `text`, `text_from` | +| `entities` | 4 | 7 | `description`, `events`, `mission` | +| `events` | 2 | 5 | `description`, `name`, `source` | +| `evidence_requirements` | 17 | 24 | `capture_requirement_ref`, `capture_spec_ref`, `description`, `field_selectors`, `notes`, `observation_demand`, `output_contract` | +| `features` | 4 | 7 | `description`, `destination`, `environment`, `name`. The list also offers `version`. | +| `infrastructure` | 4 | 6 | `acls`, `description` | +| `injects` | 3 | 6 | `description`, `environment`, `name` | +| `nodes` | 9 | 16 | `architecture`, `asset_value`, `injects`, `os_distribution`, `os_version`, `runtime`, `source` | +| `objectives` | 7 | 9 | `description`, `name` | +| `realization` | 2 | 3 | `constraints` | +| `relationships` | 11 | 17 | `database_access`, `description`, `forwarding_edge`, `mail_access`, `proxy_upstream`, `service_integration` | +| `scripts` | 4 | 6 | `description`, `name` | +| `stories` | 2 | 4 | `description`, `name` | +| `workflows` | 2 | 5 | `compensation`, `description`, `timeout` | + +These schema sections have no list: `action_contracts`, `execution_policy`, +`forwarding_agents`, `generated_artifacts`, `observation_boundaries`, +`outcome_interpretation_rules` and `persistent_volumes`. + +## Schema observations + +S1 to S3 load each document with `yaml.safe_load` and validate it against +`contracts/schemas/sdl/sdl-authoring-input-v1.json` with the +`Draft202012Validator` of `jsonschema` 4.26.0. That is how an editor association +checks raw source. + +| ID | Operation | Observed result | Limit | +| --- | --- | --- | --- | +| S1 | The 21 `*.sdl.yaml` files under `examples/` and `docs/public/_static/examples/` | `language_diagnostics` accepts all 21. The schema rejects one, `docs/public/_static/examples/first-scenario.sdl.yaml`: its `/nodes/lab-network/type` is `Switch`, and the schema allows only `compute` and `switch`. | | +| S2 | Six documents that `language_diagnostics` accepts: P without `relationships`, four edits of M, and a document with the node `web` and `infrastructure: {web: 2}` | P without `relationships` has no schema error. The other five have one each. The edits of M are `type: Service` in the feature, `source: webapp`, `features: [app]` in place of the node's `features` and `roles`, and `roles: {client: www}`. | | +| S3 | Four documents that `language_diagnostics` rejects | `features: {missing: client}` in M (`sdl.semantic`) passes the schema. So does `name: demo` followed by `name: again` (`sdl.mapping_key_conflict`). `colour: blue` in M's node (`sdl.model.invalid`) and the key `Features` in M (`sdl.noncanonical_field`) fail it. | `yaml.safe_load` keeps only the last duplicate key, so the schema never sees the first | +| S4 | Read the schema's annotations, and request its `$id` with `curl -L` | `x-raes-validates-raw-source` is `false`. `$id` is `https://openrae.github.io/rae/schemas/sdl-authoring-input-v1.json`, and the request returned HTTP 404 on 2026-10-09 at 11:45 UTC. | | +| S5 | Compare the schema at tag `v6.0.1` with this revision. Run `language_diagnostics` on `name: demo` plus `execution_policy: {}` at both. | At `v6.0.1` the schema has 45 top-level properties and 602 `$defs` entries, and no top-level `execution_policy`. At this revision it has 46 and 606, including `execution_policy`. The document is valid at this revision. With the `v6.0.1` package tree on `PYTHONPATH`, it fails with `sdl.model.invalid`, "Extra inputs are not permitted", at `/execution_policy`. | The `v6.0.1` run used this revision's dependencies, not the wheel's | +| Y1 | yaml-language-server 1.24.0 from npm, over stdio, with the association that a VS Code `yamlValidation` entry for `*.sdl.yaml` produces. The schema is the prototype's copy of this revision's file, SHA-256 `162121460f19a555f4e7c469a3e4574fe4ed5ecabe628836bf1880ede5f1aba0`. | 20 of the 21 S1 files get no diagnostics. `first-scenario.sdl.yaml` gets "Value is not accepted" on `type`. Each of the five accepted shorthand and case forms in S2 gets one diagnostic, and the unresolved reference in S3 gets none. On an empty line inside `/nodes/web`, where only `type` is present, completion offers 15 fields. Hover on `type` shows the `NodeType` description. | Red Hat's YAML extension 1.24.0 uses this server version. VS Code itself was not started. | + +## Inspected seams + +These source locations support the claims that a probe does not show. All +package paths are relative to `implementations/python/packages/`. + +| Concern | Source and inspected symbol | +| --- | --- | +| Field lists | `raes/_language_metadata.py`: `SECTION_FIELD_COMPLETIONS`, a hand-kept tuple of field names for each section | +| Completion depth | `raes/language_service.py`: `language_completions`. A path of one section name, and any longer path that does not end in a reference field, both read `SECTION_FIELD_COMPLETIONS[pointer[0]]`. | +| Reference completion | `raes/_language_metadata.py`: `REFERENCE_COMPLETION_TARGETS`; `raes/language_service.py`: `_completion_target_section`, `_reference_completion_items` | +| Completion on text that does not parse | `raes/language_service.py`: `_load_completion_data` returns the parse diagnostic instead of items | +| Semantic diagnostics | `raes/language_service.py`: `language_diagnostics` turns each message in `SDLValidationError.errors` into an `sdl.semantic` record without a path or range | +| Document context | Each helper takes text only. `language_diagnostics` calls `parse_sdl` without a path. | +| Handled exceptions | `raes/language_service.py`: `language_diagnostics` catches `SDLParseError` and `SDLValidationError`. `_load_completion_data`, `_try_declaration_index` and `language_format` catch `SDLParseError`. `_declaration_index_from_data` catches the pydantic `ValidationError`. `apply_structured_edit` catches `SDLParseError` and `ValueError`. `raes/_language_references.py`: `find_references` catches `SDLParseError`. Any other exception reaches the caller. | +| Input limit | `raes/language_service.py`: `_MAX_INPUT_BYTES`, `_size_error` | +| Formatting | `raes/language_service.py`: `language_format`, whose docstring reads "Migrate recognized legacy spellings and return canonical SDL YAML"; `raes/formatting.py`: `format_sdl_source` | +| Module cache | `raes/module_registry/__init__.py`: `_oci_cache_dir` returns `.raes/module-cache` under the base directory | +| Schema for an installed `raes` | `raes_contracts/corpus.py`: `corpus_family_root`. The wheel build bundles the contract corpus at `raes_contracts/_corpus`, as the comment above `[tool.hatch.build.targets.wheel.hooks.custom]` in `implementations/python/pyproject.toml` describes. | +| Python range | `implementations/python/pyproject.toml` at tag `v6.0.1`: `requires-python = ">=3.11,<3.15"` | + +## Reproduce + +The prototype's +[evidence scripts](https://github.com/doublewhy/vscode-raes-sdl/tree/feac96e0d5262d914c11deecb209426851915763/evidence) +automate L1 to L11, S1 to S3 and Y1. Each script takes the path of a RAES +checkout and runs under that checkout's Python environment. The Y1 script also +takes the directory where `yaml-language-server@1.24.0` is installed. At this +revision the scripts printed the results above, and L11 records three runs of +the latency script. S4 and S5 used `curl`, `git show` and `git archive` on tag +`v6.0.1`. diff --git a/docs/research/sdl-editor-support/feasibility.md b/docs/research/sdl-editor-support/feasibility.md new file mode 100644 index 0000000000..222518a9e6 --- /dev/null +++ b/docs/research/sdl-editor-support/feasibility.md @@ -0,0 +1,235 @@ +# SDL editor support feasibility + +Date: 2026-10-09 + +This note answers issue #1412: should RAES support SDL authoring in VS Code, and what scope +would justify the work? It is research evidence, not contract authority. + +The observations use repository revision `35122105b0c1bb648754625d8e3920eafa9bc599`. The +[evidence record](evidence.md) holds the probe inputs and outputs, and IDs such as L4 name its +rows. Claims about the editor client rest on a prototype extension at +[doublewhy/vscode-raes-sdl](https://github.com/doublewhy/vscode-raes-sdl/tree/feac96e0d5262d914c11deecb209426851915763). +The prototype stays outside this repository because the issue asks for an exploration, and an +extension here would be executable adoption. + +## Recommendation + +Build one language server over the existing helpers and keep each editor client thin. A VS Code +extension is worth having as that thin client. It does not need its own SDL logic. + +The smallest useful feature set is: + +1. File association for `*.sdl.yaml`. +2. Diagnostics from `language_diagnostics`, debounced while the author types. +3. Reference completion and go to definition from `language_completions` and + `language_references`. + +Leave out formatting, edit actions, and schema diagnostics until the gaps below are closed. +Formatting and structured edits drop comments. Schema diagnostics disagree with RAES validation. + +Reusable components already exist. The five helpers carry the semantics. The published schema +serves completion and hover. `raes_contracts.corpus` locates the schema that matches an installed +`raes`. From the prototype, the stdio server and its cursor mapper are reusable starting points. + +These helper changes would remove most of the work that each client now repeats: + +1. Accept a line and column, or return a map from positions to paths. +2. Give semantic diagnostics a path and a source range. +3. Derive field completion from the model at every depth, not from hand-kept lists. +4. Offer only references that the field's purpose accepts, as #1339 requires. +5. Accept an optional document path, so local imports can resolve. +6. Return a diagnostic for every input instead of raising. + +### Alternatives + +The issue lists four alternatives: + +1. **YAML tooling with the schema association and explicit CLI validation: rejected as the + main path.** The association gives completion and hover, but its diagnostics disagree with + RAES (S2, S3 and Y1). `raes semantic validate` reports RAES results only when the author runs + it. Both remain useful without an extension. +2. **A highlighting and snippet extension: rejected.** Highlighting adds little over YAML, and + neither feature gives diagnostics or references. +3. **Direct VS Code language-feature providers over the helpers: rejected.** They still need a + Python process to run the helpers, and they serve only VS Code. Other editors and Monaco would + each need their own providers. +4. **One language server with thin editor clients: selected.** One server serves every LSP + client, and the editor-specific code stays small. + +## What the language service supports + +[`raes.language_service`](../../../implementations/python/packages/raes/language_service.py) +has five helpers that return plain dictionaries. The MCP server wraps them as the +`language_service` tool family. This repository ships no language server. + +| Helper | Input | Result | Editor use | +| --- | --- | --- | --- | +| `language_completions` | Text, a JSON-pointer-like `cursor_path`, a prefix | Top-level keys, section fields, or reference candidates | Completion | +| `language_references` | Text, a bare or qualified symbol | Definitions and occurrences with 1-based ranges | Go to definition, find references | +| `language_diagnostics` | Text, a `semantic_validation` flag | Parse, structural, and semantic records | Diagnostics | +| `language_format` | Text | Canonical YAML with migration advisories | Formatting | +| `apply_structured_edit` | Text, an operation, a JSON pointer, a value | Edited YAML with diagnostics | Code actions | + +The helpers reuse the repository parser and semantic validator. Each one refuses input over +64 KiB. The [language-service guide](../../explain/sdl/language-service.md) describes the +payloads. + +## What the editor integration needs + +A VS Code integration needs these parts: + +- A language contribution that maps `*.sdl.yaml` to a language identifier. +- A client that starts the server. With `vscode-languageclient`, the prototype's client is 41 + lines. +- A server that turns cursor positions into helper paths and helper ranges into LSP ranges. It + also debounces diagnostics and copes with documents that do not parse yet. +- A setting that picks the Python interpreter. The `raes` version installed there decides the + semantics. +- A Workspace Trust declaration, because that setting names a program to run. +- A package built with `vsce`. A VS Code package cannot declare Python dependencies, so authors + install `raes` with `pip`. + +## What the prototype demonstrates + +The prototype has a TextMate grammar, a schema association, and a stdio language server. The +server uses only the Python standard library and `raes`. Sixteen tests drive the server over +stdio with the messages that the VS Code client sends. Nine test the cursor mapper, and six +tokenize SDL with VS Code's YAML grammar. All 31 passed locally against revision `35122105`, +and in [CI run 37926735623](https://github.com/doublewhy/vscode-raes-sdl/actions/runs/37926735623) +with `raes` 6.0.1 from PyPI. In that run, `vsce package` built a 571 KB package. Nothing is +published to the Marketplace, and no test starts VS Code itself. + +| Feature | Result | +| --- | --- | +| Diagnostics | Parse and structural errors keep their RAES ranges. Semantic errors have no location, so they appear on line 1 with a note. | +| Completion | Works on documents that parse. Otherwise the server answers from the last version that parsed and marks the items. | +| Go to definition | Works inside one file through `language_references`. | +| Highlighting | Top-level SDL keys and `${variable}` placeholders get their own scopes. A test checks the key list against the schema. | +| Schema association | Completion and hover work in yaml-language-server 1.24.0. Its diagnostics disagree with RAES. | +| Formatting | Not offered, because the helper drops comments. | + +The server forwards reference candidates unfiltered, so the #1339 mismatch stays visible. It +drops field names only after `key:` or inside a flow sequence, where no mapping key can go. That +is a YAML rule, not an SDL rule. When a helper raises, the server reports one diagnostic and +keeps running. Two tests check this with helpers that raise on a marker line. + +## Gaps + +| Area | Observation | Evidence | +| --- | --- | --- | +| Reference completion | For a participant relationship `target`, completion offers six declarations. Validation accepts one, the agent that is not the source. This is the #1339 mismatch. | L1; finding F5 in the [participant identity audit](../participant-identity/audit.md) | +| Field lists | The hand-kept lists miss 69 schema fields across 18 of 33 sections. The `features` list offers `version`, which validation rejects. | L2 and L3; `SECTION_FIELD_COMPLETIONS` in [`_language_metadata.py`](../../../implementations/python/packages/raes/_language_metadata.py) | +| Completion depth | Field completion uses only the section name. `/nodes`, `/nodes/web`, and `/nodes/web/resources` get the same nine fields. | L10; the field branches of `language_completions` in [`language_service.py`](../../../implementations/python/packages/raes/language_service.py) | +| Cursor paths | The helpers take a JSON pointer, so each client derives one from the text. The prototype's mapper has 204 lines and skips multi-line flow collections, block scalars, anchors, tags, and complex keys. | Prototype [`server/sdl_cursor.py`](https://github.com/doublewhy/vscode-raes-sdl/blob/feac96e0d5262d914c11deecb209426851915763/server/sdl_cursor.py) | +| Source ranges | Semantic diagnostics have no path or range. An unknown field is ranged on its value, not its key. A YAML syntax error is a single point. | L5 | +| Incomplete documents | Completion needs a document that parses. An unclosed `[` or a key without its colon gives a parse error and no items. | L4 | +| Imported modules | The helpers take no file path, so a document with `imports` is invalid even without semantic validation. `parse_sdl_file` accepts the same fixture. | L8 | +| Workspace context | The helpers see one document. Nothing indexes the workspace or follows references across files. | [`language_service.py`](../../../implementations/python/packages/raes/language_service.py) | +| Robustness | Each helper turns the parser's and validator's own errors into results. Any other exception reaches the caller, so a server must guard every call. | Handled exceptions under [inspected seams](evidence.md#inspected-seams) | +| Large scenarios | The largest repository example is 46 KB, under the 64 KiB limit. Each call parses the whole text. On that file a warm call took 160 to 175 ms at load average 7 to 13, and about 375 ms at load average 22. A cold start took 1.2 to 2.9 s. The machine was shared, so the timings are indicative. | L9 and L11 | +| Formatting and edits | Both drop comments and expand shorthand. Formatting also lowers enum case and migrates recognized legacy spellings. | L6 and L7; the `language_format` docstring | +| Schema validation | The schema describes the normalized object, so `x-raes-validates-raw-source` is `false`. It flags accepted shorthand and case variants and misses semantic errors. | S2 to S4 and Y1 | +| Schema versions | The schema's `$id` returned HTTP 404. The schema on `dev` declares `execution_policy`, which `raes` 6.0.1 rejects. | S4 and S5 | + +The schema check flags one of the 21 accepted examples (S1): the quickstart's +`first-scenario.sdl.yaml` uses `type: Switch`. Any fix for imports must also handle registry +imports. When `parse_sdl` gets a path, an OCI import fetches the module over the network. It also +writes `.raes/module-cache` next to the document (`raes.module_registry`). + +## Answers to the issue's questions + +### Which features are most useful + +Diagnostics come first. They need no cursor mapping and reuse the full validator. Reference +completion and go to definition come next. They help most on reference-heavy sections, but +completion inherits the #1339 mismatch. Schema hover shows field descriptions. Highlighting adds +little over plain YAML. Snippets and a document outline were not prototyped. Formatting should +wait until comments survive it. + +### Can YAML support and the schema provide enough value + +They are enough for completion and hover, not for diagnostics. Inside a node, schema completion +offered the 15 node fields not yet present (Y1). The helper's list for nodes has 9 fields in all +(L10). Hover showed the schema's descriptions. But schema diagnostics flagged all five accepted +shorthand and case forms that the probe tried. They also missed an unresolved reference. A +dedicated extension is warranted only to add RAES diagnostics and reference features. + +Red Hat's YAML extension serves only the languages in its own document selector ([R2](#references)). +So an extension must choose: + +- Its own language identifier, with SDL highlighting but no schema completion. +- The `yaml` identifier, with schema completion but plain YAML highlighting. + +The prototype supports both. + +### Would an LSP adapter enable reuse + +Yes, for desktop editors. Any LSP client can start the same stdio command. VS Code needs the thin +extension described above. Other editors that speak LSP should need only their own settings for +the command and the file pattern. They were not tested. + +Monaco is different. A browser page cannot start a Python process. A Monaco client would need a +server reached over a WebSocket, for example through +[monaco-languageclient](https://github.com/TypeFox/monaco-languageclient). Running `raes` in +the browser was not evaluated. [monaco-yaml](https://github.com/remcohaszing/monaco-yaml) gives +Monaco the schema features, as the YAML extension does in VS Code. A VS Code extension does not +run inside Monaco. + +### What gaps exist + +See [Gaps](#gaps). The cursor-to-path mapping, source ranges, incomplete documents, imports, +workspace context, and large scenarios each have a row there. So does the #1339 completion +mismatch. + +### How packaging and versions should work + +- **File association.** The repository names 40 SDL files `*.sdl.yaml` (L9). VS Code prefers the + longest matching extension, so `.sdl.yaml` wins over `.yaml` ([R1](#references)). Files such as + the composition fixtures end in `.yaml` and need a `files.associations` entry. +- **Version selection.** The server runs in an interpreter that the author picks, and it reports + the `raes` version. A schema bundled in the extension can drift from that version, as S5 shows. + The `raes` wheel bundles its own copy, reachable through + `raes_contracts.corpus.corpus_family_root`, so a server could serve the matching schema. +- **Local and offline use.** The helpers work on the buffer text in-process. They read no other + document and use no network. Resolving imports would change that, so it should stay opt-in. + Red Hat's YAML extension downloads the SchemaStore catalog by default + (`yaml.schemaStore.enable`, [R3](#references)). +- **Packaging.** `vsce package` works without bundling Python. Authors install `raes` from PyPI. + Version 6.0.1 requires Python 3.11 to 3.14. In Restricted Mode, VS Code ignores a workspace + value of the interpreter setting if the extension lists it in `restrictedConfigurations` + ([R4](#references)). + +## Compatibility notes + +- **SDL semantics stay in RAES.** The prototype server adds no SDL rules. The grammar's key list + mirrors the schema's top-level properties, and a test checks that they match. +- **No launch and no fetch.** The helpers parse and validate text. Nothing runs a scenario or + downloads content. +- **Formatting.** `language_format` drops comments, expands shorthand, normalizes enum case, and + migrates legacy spellings. Format on save would rewrite authored text, so formatting should be + an explicit command at most. +- **Monaco.** Shared services are the helpers, the published schema, and one language server. + Editor adapters are the VS Code client, a Monaco client, and other editors' settings. + +## Method + +The probes call the helpers on small documents and on the repository examples. They compare the +results with the published schema and with yaml-language-server 1.24.0. Version 1.24.0 is the +server that Red Hat's YAML extension 1.24.0 uses. Nothing was mocked. The +[evidence record](evidence.md) gives each probe's input and output, the source locations behind +the other claims, and the scripts that reproduce the probes. The prototype's tests and CI run +cover only the editor client. + +## References + +- R1: VS Code 1.140.0, + [`languagesAssociations.ts`](https://github.com/microsoft/vscode/blob/1.140.0/src/vs/editor/common/services/languagesAssociations.ts), + the longest extension match. +- R2: Red Hat YAML extension 1.24.0, + [`src/extension.ts`](https://github.com/redhat-developer/vscode-yaml/blob/1.24.0/src/extension.ts), + the client document selector and the `yamlValidation` association. +- R3: Red Hat YAML extension 1.24.0, + [`package.json`](https://github.com/redhat-developer/vscode-yaml/blob/1.24.0/package.json), + the `yaml.schemaStore.enable` default. +- R4: VS Code, [Workspace Trust extension guide](https://code.visualstudio.com/api/extension-guides/workspace-trust). +- VS Code, [language server extension guide](https://code.visualstudio.com/api/language-extensions/language-server-extension-guide). From 1ebc0bfa9ef25ca48f9973813c4305cd15ae0090 Mon Sep 17 00:00:00 2001 From: Yernat Yestekov <2068106+doublewhy@users.noreply.github.com> Date: Sat, 10 Oct 2026 09:39:44 -0700 Subject: [PATCH 2/4] docs(sdl): correct import trust and format error claims The note said that parse_sdl with a path fetches an OCI import. It refuses the import before any request unless raes-trust.yaml beside the document lists the registry, and it writes .raes/module-cache only after the registry's signature policy passes. Say so, add an Import trust seam row, and note in the Workspace Trust bullet that the trust file is workspace content. The Robustness row said that each helper turns the parser's and validator's errors into results. language_format lets SDLValidationError reach the caller for a materialized document, because parse_sdl runs the semantic validator for it. Record that as probe L12 and complete the handled-exceptions seam row. Also list the nine schema sections without a completion list, give verdicts on snippets and the document outline, and say that the probe scripts live outside this repository. --- docs/research/sdl-editor-support/evidence.md | 16 +++++++----- .../sdl-editor-support/feasibility.md | 26 +++++++++++++------ 2 files changed, 28 insertions(+), 14 deletions(-) diff --git a/docs/research/sdl-editor-support/evidence.md b/docs/research/sdl-editor-support/evidence.md index ca3f672fc1..e53871a706 100644 --- a/docs/research/sdl-editor-support/evidence.md +++ b/docs/research/sdl-editor-support/evidence.md @@ -1,6 +1,6 @@ # SDL editor support evidence -Observed 2026-10-09 on repository revision +Observed 2026-10-09, and L12 on 2026-10-10, on repository revision `35122105b0c1bb648754625d8e3920eafa9bc599`. The [feasibility note](feasibility.md) interprets these observations. The probes called the language-service helpers in-process, with the working tree at that @@ -73,7 +73,7 @@ are 1-based, as the helpers report them. | ID | Operation | Observed result | Limit | | --- | --- | --- | --- | | L1 | `language_completions(P, cursor_path="/relationships/pair/target")`, then `language_diagnostics` of P with each offered `insert_text` as the target | Context `reference:targetable`, with six items: `agents.one`, `agents.two`, `assertions.done`, `entities.team`, `propositions.ok` and `relationships.pair`. Only `two` is valid. `one` fails with "Relationship 'pair' requires distinct participants". The other four fail because the target must "resolve unambiguously to a declared agent". | Finding F5 in the [participant audit](../participant-identity/audit.md) reports the same mismatch, and #1339 tracks it | -| L2 | Compare each list in `SECTION_FIELD_COMPLETIONS` with the property names that the published schema declares for that section | 18 of the 33 lists differ from the schema. Together they omit 69 schema fields. The `features` list also offers `version`, which the schema does not declare. The table below lists the differences. | The comparison merges every union branch. Seven schema sections have no list. | +| L2 | Compare each list in `SECTION_FIELD_COMPLETIONS` with the property names that the published schema declares for that section | 18 of the 33 lists differ from the schema. Together they omit 69 schema fields. The `features` list also offers `version`, which the schema does not declare. The table below lists the differences. | The comparison merges every union branch. Nine schema sections have no list. | | L3 | `language_completions` on a document whose only feature is `app: {type: service}`, with `cursor_path="/features/app"` and `prefix="v"`; then `language_diagnostics` with `version: '1.0'` added to that feature | Completion offers `version`. Diagnostics report `sdl.model.invalid`, "Extra inputs are not permitted", at `/features/app/version`. | | | L4 | `language_completions(text, cursor_path="/nodes/web/features")` on N and on four edits of N | N gives `ok` with `app`. `features:` with no value gives `ok` with `app`. `features: [` gives `invalid` with `sdl.parse`. `fea` in place of the `features` line gives `invalid` with `sdl.parse`. `type: VM` gives `invalid` with `sdl.legacy_node_type_vm`. | A parse failure anywhere in the document removes every item | | L5 | `language_diagnostics` on four documents | Adding `colour: blue` under `web` in N gives `sdl.model.invalid` at `/nodes/web/colour`, on line 10, columns 13 to 17. The key starts at column 5. `features: [missing]` in N gives `sdl.semantic` at the `semantic_validation` stage, with no path and no range. `name: demo` followed by `name: again` gives `sdl.mapping_key_conflict` at `/name`, on line 2, columns 1 to 5. An unclosed `web: {type: compute` gives `sdl.parse` with an empty path and one point, line 4, column 1. | The point in the last case is the end of the input | @@ -83,6 +83,7 @@ are 1-based, as the helpers report them. | L9 | File sizes of the tracked `*.sdl.yaml` files, against `_MAX_INPUT_BYTES` | The limit is 65,536 bytes. None of the 40 files is over it. The largest is `examples/scenarios/hospital-ransomware-surgery-day.sdl.yaml`, at 46,073 bytes. | | | L10 | `language_completions(N, cursor_path=path)` for `/nodes`, `/nodes/web`, `/nodes/web/resources` and `/nodes/web/os` | Each call returns context `section:nodes` and the same nine fields: `type`, `description`, `os`, `resources`, `features`, `conditions`, `services`, `roles` and `endpoint_persona`. | | | L11 | Median of 5 warm calls of `language_diagnostics`, and of `language_completions` at `/nodes`, on the L9 file. Median of 3 fresh interpreters that import `raes` and make one diagnostics call. | At load average 7.2: 172 ms and 159 ms warm, and 1,207 ms cold. At 13.1: 174 ms and 163 ms warm, and 1,297 ms cold. At 22.5: 377 ms and 373 ms warm, and 2,939 ms cold. | An 8-core Apple M3 that other work shared. The timings are indicative. | +| L12 | The five helpers on two documents: `contracts/fixtures/sdl/materialized-scenario-v1/valid/reference.json` with N's node `web` added, and N. In both, the node's `features` is `[missing]`. Completion uses `cursor_path="/nodes/web/features"`, references use the symbol `web`, and the edit is L7's. | On the fixture, `language_diagnostics` returns `invalid` with `sdl.semantic`. `language_format` raises `SDLValidationError`, "Node 'web' references undefined feature 'missing'". Completion and references return `ok`, and the edit returns `edited_with_diagnostics` with `sdl.semantic`. The fixture's JSON text and its YAML dump behave the same. On N nothing raises, and `language_format` returns `formatted_with_diagnostics` with `sdl.semantic`. | | L2 counts, for each list, the distinct property names that the schema declares for one entry of the section. For a section that is one object, such as @@ -111,8 +112,9 @@ merges `anyOf`, `oneOf` and `allOf` branches. These lists differ: | `workflows` | 2 | 5 | `compensation`, `description`, `timeout` | These schema sections have no list: `action_contracts`, `execution_policy`, -`forwarding_agents`, `generated_artifacts`, `observation_boundaries`, -`outcome_interpretation_rules` and `persistent_volumes`. +`forwarding_agents`, `generated_artifacts`, `imports`, `module`, +`observation_boundaries`, `outcome_interpretation_rules` and +`persistent_volumes`. The schema's other four top-level properties are scalars. ## Schema observations @@ -143,10 +145,11 @@ package paths are relative to `implementations/python/packages/`. | Completion on text that does not parse | `raes/language_service.py`: `_load_completion_data` returns the parse diagnostic instead of items | | Semantic diagnostics | `raes/language_service.py`: `language_diagnostics` turns each message in `SDLValidationError.errors` into an `sdl.semantic` record without a path or range | | Document context | Each helper takes text only. `language_diagnostics` calls `parse_sdl` without a path. | -| Handled exceptions | `raes/language_service.py`: `language_diagnostics` catches `SDLParseError` and `SDLValidationError`. `_load_completion_data`, `_try_declaration_index` and `language_format` catch `SDLParseError`. `_declaration_index_from_data` catches the pydantic `ValidationError`. `apply_structured_edit` catches `SDLParseError` and `ValueError`. `raes/_language_references.py`: `find_references` catches `SDLParseError`. Any other exception reaches the caller. | +| Handled exceptions | `raes/language_service.py`: `language_diagnostics` catches `SDLParseError` and `SDLValidationError`. `_load_completion_data`, `_try_declaration_index` and `language_format` catch `SDLParseError`. `_declaration_index_from_data` catches the pydantic `ValidationError`. `apply_structured_edit` catches `SDLParseError` and `ValueError`, and `_split_pointer_or_empty` catches `ValueError`. `raes/_language_references.py`: `_compose_yaml`, which `find_references` calls, catches `SDLParseError`. `_qualified_symbol_section`, `_is_variation_member_symbol` and `_bare_symbol` catch `TypeError` and `ValueError` from symbol parsing. `format_sdl_source` (`raes/formatting.py`) re-parses its output with `parse_sdl(..., skip_semantic_validation=True)`, and `parse_sdl` (`raes/parser.py`) still runs `SemanticValidator` for a `MaterializedScenario`, so `language_format` lets `SDLValidationError` reach the caller. Any other exception reaches the caller. | | Input limit | `raes/language_service.py`: `_MAX_INPUT_BYTES`, `_size_error` | | Formatting | `raes/language_service.py`: `language_format`, whose docstring reads "Migrate recognized legacy spellings and return canonical SDL YAML"; `raes/formatting.py`: `format_sdl_source` | | Module cache | `raes/module_registry/__init__.py`: `_oci_cache_dir` returns `.raes/module-cache` under the base directory | +| Import trust | `raes/composition/_expand.py`: `expand_sdl_modules` calls `load_lockfile` and `load_trust_policy` on the document's directory. `raes/module_registry/models.py`: `load_trust_policy` returns `TrustPolicy()`, with no registries, when `raes-trust.yaml` is missing; `RegistryTrustPolicy.require_signatures` defaults to `True`. `raes/module_registry/resolution.py`: `_resolve_oci_import` raises "is not allowed by trust policy" for an unlisted registry before any request. For a listed registry it then fetches the manifest, config and bundle, checks signatures if the registry requires them, and only then calls `_extract_bundle_to_cache`. The integration test `test_untrusted_and_unsigned_oci_imports_fail_closed` in `implementations/python/tests/test_sdl_module_registry.py` asserts both refusals. | | Schema for an installed `raes` | `raes_contracts/corpus.py`: `corpus_family_root`. The wheel build bundles the contract corpus at `raes_contracts/_corpus`, as the comment above `[tool.hatch.build.targets.wheel.hooks.custom]` in `implementations/python/pyproject.toml` describes. | | Python range | `implementations/python/pyproject.toml` at tag `v6.0.1`: `requires-python = ">=3.11,<3.15"` | @@ -159,4 +162,5 @@ checkout and runs under that checkout's Python environment. The Y1 script also takes the directory where `yaml-language-server@1.24.0` is installed. At this revision the scripts printed the results above, and L11 records three runs of the latency script. S4 and S5 used `curl`, `git show` and `git archive` on tag -`v6.0.1`. +`v6.0.1`. L12 called the helpers directly, in-process, on the inputs that its +row names. diff --git a/docs/research/sdl-editor-support/feasibility.md b/docs/research/sdl-editor-support/feasibility.md index 222518a9e6..b165c4420f 100644 --- a/docs/research/sdl-editor-support/feasibility.md +++ b/docs/research/sdl-editor-support/feasibility.md @@ -85,7 +85,9 @@ A VS Code integration needs these parts: also debounces diagnostics and copes with documents that do not parse yet. - A setting that picks the Python interpreter. The `raes` version installed there decides the semantics. -- A Workspace Trust declaration, because that setting names a program to run. +- A Workspace Trust declaration, because that setting names a program to run. If the server + resolves imports, the workspace's `raes-trust.yaml` also decides which registries an import may + fetch from. - A package built with `vsce`. A VS Code package cannot declare Python dependencies, so authors install `raes` with `pip`. @@ -125,7 +127,7 @@ keeps running. Two tests check this with helpers that raise on a marker line. | Incomplete documents | Completion needs a document that parses. An unclosed `[` or a key without its colon gives a parse error and no items. | L4 | | Imported modules | The helpers take no file path, so a document with `imports` is invalid even without semantic validation. `parse_sdl_file` accepts the same fixture. | L8 | | Workspace context | The helpers see one document. Nothing indexes the workspace or follows references across files. | [`language_service.py`](../../../implementations/python/packages/raes/language_service.py) | -| Robustness | Each helper turns the parser's and validator's own errors into results. Any other exception reaches the caller, so a server must guard every call. | Handled exceptions under [inspected seams](evidence.md#inspected-seams) | +| Robustness | Every helper turns `SDLParseError` into a result. `language_diagnostics` also turns `SDLValidationError` into one, and `apply_structured_edit` and `language_format` check their output with `language_diagnostics`. But `language_format` lets `SDLValidationError` reach the caller for a document with `materialization_provenance`. For such a document, `parse_sdl` runs the semantic validator even though `format_sdl_source` skips it. Other exceptions reach the caller too, unless a catch listed under inspected seams handles them, so a server must guard every call. | L12; handled exceptions under [inspected seams](evidence.md#inspected-seams) | | Large scenarios | The largest repository example is 46 KB, under the 64 KiB limit. Each call parses the whole text. On that file a warm call took 160 to 175 ms at load average 7 to 13, and about 375 ms at load average 22. A cold start took 1.2 to 2.9 s. The machine was shared, so the timings are indicative. | L9 and L11 | | Formatting and edits | Both drop comments and expand shorthand. Formatting also lowers enum case and migrates recognized legacy spellings. | L6 and L7; the `language_format` docstring | | Schema validation | The schema describes the normalized object, so `x-raes-validates-raw-source` is `false`. It flags accepted shorthand and case variants and misses semantic errors. | S2 to S4 and Y1 | @@ -133,8 +135,12 @@ keeps running. Two tests check this with helpers that raise on a marker line. The schema check flags one of the 21 accepted examples (S1): the quickstart's `first-scenario.sdl.yaml` uses `type: Switch`. Any fix for imports must also handle registry -imports. When `parse_sdl` gets a path, an OCI import fetches the module over the network. It also -writes `.raes/module-cache` next to the document (`raes.module_registry`). +imports. When `parse_sdl` gets a path and the document has `imports`, it reads `raes.lock.json` +and `raes-trust.yaml` from the document's directory. It refuses an OCI import before any request +unless that trust file lists the import's registry. For a listed registry it fetches the module. +Once the registry's signature policy passes, it writes `.raes/module-cache` next to the document. +The trust file is workspace content, so a path-aware server should resolve imports only in a +trusted workspace (Import trust under [inspected seams](evidence.md#inspected-seams)). ## Answers to the issue's questions @@ -143,7 +149,10 @@ writes `.raes/module-cache` next to the document (`raes.module_registry`). Diagnostics come first. They need no cursor mapping and reuse the full validator. Reference completion and go to definition come next. They help most on reference-heavy sections, but completion inherits the #1339 mismatch. Schema hover shows field descriptions. Highlighting adds -little over plain YAML. Snippets and a document outline were not prototyped. Formatting should +little over plain YAML. Snippets and a document outline were not prototyped. Snippets give no +diagnostics or references (alternative 2), so they rank low. None of the five helpers returns the +document's declarations with their ranges, so an outline needs new helper or server work. It +would help in long scenarios, but it ranks after diagnostics and references. Formatting should wait until comments survive it. ### Can YAML support and the schema provide enough value @@ -216,9 +225,10 @@ mismatch. The probes call the helpers on small documents and on the repository examples. They compare the results with the published schema and with yaml-language-server 1.24.0. Version 1.24.0 is the server that Red Hat's YAML extension 1.24.0 uses. Nothing was mocked. The -[evidence record](evidence.md) gives each probe's input and output, the source locations behind -the other claims, and the scripts that reproduce the probes. The prototype's tests and CI run -cover only the editor client. +[evidence record](evidence.md) gives each probe's input and output and the source locations +behind the other claims. It also links to the scripts that reproduce most probes. They live in +the prototype's repository, not in this one. The prototype's tests and CI run cover only the +editor client. ## References From cb4dfe3254d2aec91d50e4121e4406bcd4ebf5f8 Mon Sep 17 00:00:00 2001 From: Brad Edwards Date: Thu, 12 Nov 2026 06:44:07 +0100 Subject: [PATCH 3/4] docs(sdl): re-observe editor support evidence on current dev Re-run the probes at c7d49997, after #1446 closed #1339 and the helpers gained an optional document path. Record path mode (L5, L8, L13), remove the two prerequisites that dev delivers, and update seams and counts. Keep L11 and Y1 as observations of 35122105. --- docs/research/sdl-editor-support/evidence.md | 97 +++++++++++------ .../sdl-editor-support/feasibility.md | 103 ++++++++++-------- 2 files changed, 123 insertions(+), 77 deletions(-) diff --git a/docs/research/sdl-editor-support/evidence.md b/docs/research/sdl-editor-support/evidence.md index e53871a706..569e3effec 100644 --- a/docs/research/sdl-editor-support/evidence.md +++ b/docs/research/sdl-editor-support/evidence.md @@ -1,13 +1,19 @@ # SDL editor support evidence -Observed 2026-10-09, and L12 on 2026-10-10, on repository revision -`35122105b0c1bb648754625d8e3920eafa9bc599`. The +Observed 2026-11-12 on repository revision +`c7d499979eb32e16d9a2c46ca10b5aa1403b4511`, except L11 and Y1. The [feasibility note](feasibility.md) interprets these observations. The probes called the language-service helpers in-process, with the working tree at that -revision and the repository Python environment (CPython 3.14.4, free-threaded). -Nothing was mocked, and no scenario was launched. The probes establish helper -and schema behavior on small inputs. They are not a usability, conformance or -security result. +revision and the repository Python environment (CPython 3.14.7). Nothing was +mocked, and no scenario was launched. The probes establish helper and schema +behavior on small inputs. They are not a usability, conformance or security +result. + +The rows were first observed on 2026-10-09 and 2026-10-10 at revision +`35122105b0c1bb648754625d8e3920eafa9bc599`, with CPython 3.14.4, free-threaded. +L11 and Y1 were not run again, and they remain observations of that revision. +[Changes since the first observation](#changes-since-the-first-observation) +lists the rows whose results changed. ## Minimal probe inputs @@ -72,18 +78,19 @@ are 1-based, as the helpers report them. | ID | Operation | Observed result | Limit | | --- | --- | --- | --- | -| L1 | `language_completions(P, cursor_path="/relationships/pair/target")`, then `language_diagnostics` of P with each offered `insert_text` as the target | Context `reference:targetable`, with six items: `agents.one`, `agents.two`, `assertions.done`, `entities.team`, `propositions.ok` and `relationships.pair`. Only `two` is valid. `one` fails with "Relationship 'pair' requires distinct participants". The other four fail because the target must "resolve unambiguously to a declared agent". | Finding F5 in the [participant audit](../participant-identity/audit.md) reports the same mismatch, and #1339 tracks it | -| L2 | Compare each list in `SECTION_FIELD_COMPLETIONS` with the property names that the published schema declares for that section | 18 of the 33 lists differ from the schema. Together they omit 69 schema fields. The `features` list also offers `version`, which the schema does not declare. The table below lists the differences. | The comparison merges every union branch. Nine schema sections have no list. | +| L1 | `language_completions(P, cursor_path="/relationships/pair/target")`, then `language_diagnostics` of P with the offered `insert_text` as the target, and with each of the other five declarations of P as the target | Context `reference:eligible:participant_endpoint`, with one item: label `two`, detail `agents.two`. P with target `two` is valid. `agents.one` fails with "Relationship 'pair' requires distinct participants". `assertions.done`, `entities.team`, `propositions.ok` and `relationships.pair` fail because the target must "resolve unambiguously to a declared agent". | At `35122105`, completion offered all six declarations. Finding F5 in the [participant audit](../participant-identity/audit.md) reports that mismatch, and #1339 closed it. | +| L2 | Compare each list in `SECTION_FIELD_COMPLETIONS` with the property names that the published schema declares for that section | 18 of the 34 lists differ from the schema. Together they omit 69 schema fields. The `features` list also offers `version`, which the schema does not declare. The table below lists the differences. | The comparison merges every union branch. Eleven schema sections have no list. | | L3 | `language_completions` on a document whose only feature is `app: {type: service}`, with `cursor_path="/features/app"` and `prefix="v"`; then `language_diagnostics` with `version: '1.0'` added to that feature | Completion offers `version`. Diagnostics report `sdl.model.invalid`, "Extra inputs are not permitted", at `/features/app/version`. | | | L4 | `language_completions(text, cursor_path="/nodes/web/features")` on N and on four edits of N | N gives `ok` with `app`. `features:` with no value gives `ok` with `app`. `features: [` gives `invalid` with `sdl.parse`. `fea` in place of the `features` line gives `invalid` with `sdl.parse`. `type: VM` gives `invalid` with `sdl.legacy_node_type_vm`. | A parse failure anywhere in the document removes every item | -| L5 | `language_diagnostics` on four documents | Adding `colour: blue` under `web` in N gives `sdl.model.invalid` at `/nodes/web/colour`, on line 10, columns 13 to 17. The key starts at column 5. `features: [missing]` in N gives `sdl.semantic` at the `semantic_validation` stage, with no path and no range. `name: demo` followed by `name: again` gives `sdl.mapping_key_conflict` at `/name`, on line 2, columns 1 to 5. An unclosed `web: {type: compute` gives `sdl.parse` with an empty path and one point, line 4, column 1. | The point in the last case is the end of the input | +| L5 | `language_diagnostics` on four documents, and on the second document again with a `path` to a file that holds it | Adding `colour: blue` under `web` in N gives `sdl.model.invalid` at `/nodes/web/colour`, on line 10, columns 13 to 17. The key starts at column 5. `features: [missing]` in N gives `sdl.semantic` at the `semantic_validation` stage, with no path and no range. With the `path`, the same diagnostic has path `/nodes/web` and a range on the key `web`, line 5, columns 3 to 6. `name: demo` followed by `name: again` gives `sdl.mapping_key_conflict` at `/name`, on line 2, columns 1 to 5. An unclosed `web: {type: compute` gives `sdl.parse` with an empty path and one point, line 4, column 1. | The point in the last case is the end of the input. The ranged semantic diagnostic marks the owner declaration, not the reference `missing`. | | L6 | `language_format` on a document with three comments and the feature `app: {type: Service, source: webapp}` | Status `formatted`. The output has no comments. `type: Service` becomes `type: service`, and `source: webapp` becomes a mapping of `name: webapp` and `version: '*'`. | | | L7 | `apply_structured_edit` on N with two added comments, operation `set`, pointer `/nodes/web/os`, value `windows` | Status `edited`. The output has no comments. `source: webapp` expands as in L6, and `features: [app]` becomes the mapping `app: ''`. | | -| L8 | `parse_sdl_file` and `language_diagnostics` on `contracts/fixtures/sdl/variation-points-v1/composition/root.yaml` | `parse_sdl_file` returns an `ExpandedScenario`. On the same text, `language_diagnostics` returns `sdl.parse`, "SDL imports require file-backed parsing via parse_sdl_file()". Turning off `semantic_validation` gives the same result. | | -| L9 | File sizes of the tracked `*.sdl.yaml` files, against `_MAX_INPUT_BYTES` | The limit is 65,536 bytes. None of the 40 files is over it. The largest is `examples/scenarios/hospital-ransomware-surgery-day.sdl.yaml`, at 46,073 bytes. | | +| L8 | `parse_sdl_file` and `language_diagnostics` on `contracts/fixtures/sdl/variation-points-v1/composition/root.yaml`; then `language_diagnostics` and `language_format` with `path` set to that file | `parse_sdl_file` returns an `ExpandedScenario`. On the same text without a `path`, `language_diagnostics` returns `sdl.parse`, "SDL imports require file-backed parsing via parse_sdl_file()". Turning off `semantic_validation` gives the same result. With the `path`, `language_diagnostics` returns `valid` and `language_format` returns `formatted`. | The fixture has only local imports | +| L9 | File sizes of the tracked `*.sdl.yaml` files, against `_MAX_INPUT_BYTES` | The limit is 65,536 bytes. None of the 60 files is over it. The largest is `examples/scenarios/hospital-ransomware-surgery-day.sdl.yaml`, at 46,073 bytes. | | | L10 | `language_completions(N, cursor_path=path)` for `/nodes`, `/nodes/web`, `/nodes/web/resources` and `/nodes/web/os` | Each call returns context `section:nodes` and the same nine fields: `type`, `description`, `os`, `resources`, `features`, `conditions`, `services`, `roles` and `endpoint_persona`. | | -| L11 | Median of 5 warm calls of `language_diagnostics`, and of `language_completions` at `/nodes`, on the L9 file. Median of 3 fresh interpreters that import `raes` and make one diagnostics call. | At load average 7.2: 172 ms and 159 ms warm, and 1,207 ms cold. At 13.1: 174 ms and 163 ms warm, and 1,297 ms cold. At 22.5: 377 ms and 373 ms warm, and 2,939 ms cold. | An 8-core Apple M3 that other work shared. The timings are indicative. | +| L11 | Median of 5 warm calls of `language_diagnostics`, and of `language_completions` at `/nodes`, on the L9 file. Median of 3 fresh interpreters that import `raes` and make one diagnostics call. | At load average 7.2: 172 ms and 159 ms warm, and 1,207 ms cold. At 13.1: 174 ms and 163 ms warm, and 1,297 ms cold. At 22.5: 377 ms and 373 ms warm, and 2,939 ms cold. | Observed at `35122105` only. An 8-core Apple M3 that other work shared. The timings are indicative. | | L12 | The five helpers on two documents: `contracts/fixtures/sdl/materialized-scenario-v1/valid/reference.json` with N's node `web` added, and N. In both, the node's `features` is `[missing]`. Completion uses `cursor_path="/nodes/web/features"`, references use the symbol `web`, and the edit is L7's. | On the fixture, `language_diagnostics` returns `invalid` with `sdl.semantic`. `language_format` raises `SDLValidationError`, "Node 'web' references undefined feature 'missing'". Completion and references return `ok`, and the edit returns `edited_with_diagnostics` with `sdl.semantic`. The fixture's JSON text and its YAML dump behave the same. On N nothing raises, and `language_format` returns `formatted_with_diagnostics` with `sdl.semantic`. | | +| L13 | The four path-aware helpers on `root.sdl.yaml` in an empty directory, with `path` set to that file. The document imports `oci:127.0.0.1:9/acme/shared`. A `raes-trust.yaml` beside it lists that registry with `require_signatures: false`. A wrapper around `raes.module_registry.urlopen` recorded each request. | `language_diagnostics` and `language_format` return `sdl.import.source-refused`, "Registry import 'oci:127.0.0.1:9/acme/shared' is not resolved in editor path mode; validate with parse_sdl_file()", at `/imports/0`. Completion at the top level and references for `shared` return `ok`. The wrapper recorded no request, and no `.raes` directory appeared. | The diagnostic's range is line 1, column 1, not the import entry | L2 counts, for each list, the distinct property names that the schema declares for one entry of the section. For a section that is one object, such as @@ -93,8 +100,8 @@ merges `anyOf`, `oneOf` and `allOf` branches. These lists differ: | Section | Offered | In schema | Schema fields not offered | | --- | --- | --- | --- | | `accounts` | 7 | 14 | `description`, `disabled`, `groups`, `home`, `mail`, `materialization_profile`, `shell` | -| `agents` | 7 | 12 | `allowed_subnets`, `authority_anchors`, `description`, `observation_boundaries`, `operating_scope` | -| `behavior_specifications` | 17 | 18 | `autonomous_execution` | +| `agents` | 8 | 13 | `allowed_subnets`, `authority_anchors`, `description`, `observation_boundaries`, `operating_scope` | +| `behavior_specifications` | 19 | 20 | `autonomous_execution` | | `conditions` | 5 | 10 | `environment`, `name`, `retries`, `start_period`, `timeout` | | `content` | 7 | 13 | `description`, `destination`, `sensitive`, `tags`, `text`, `text_from` | | `entities` | 4 | 7 | `description`, `events`, `mission` | @@ -113,8 +120,9 @@ merges `anyOf`, `oneOf` and `allOf` branches. These lists differ: These schema sections have no list: `action_contracts`, `execution_policy`, `forwarding_agents`, `generated_artifacts`, `imports`, `module`, -`observation_boundaries`, `outcome_interpretation_rules` and -`persistent_volumes`. The schema's other four top-level properties are scalars. +`observation_boundaries`, `outcome_interpretation_rules`, `persistent_volumes`, +`safe_state` and `trial_parallelism`. The schema's other four top-level +properties are scalars. ## Schema observations @@ -125,12 +133,12 @@ checks raw source. | ID | Operation | Observed result | Limit | | --- | --- | --- | --- | -| S1 | The 21 `*.sdl.yaml` files under `examples/` and `docs/public/_static/examples/` | `language_diagnostics` accepts all 21. The schema rejects one, `docs/public/_static/examples/first-scenario.sdl.yaml`: its `/nodes/lab-network/type` is `Switch`, and the schema allows only `compute` and `switch`. | | +| S1 | The 27 `*.sdl.yaml` files under `examples/` and `docs/public/_static/examples/`. Diagnostics use each file's `path`. | `language_diagnostics` accepts all 27. The schema rejects one, `docs/public/_static/examples/first-scenario.sdl.yaml`: its `/nodes/lab-network/type` is `Switch`, and the schema allows only `compute` and `switch`. | | | S2 | Six documents that `language_diagnostics` accepts: P without `relationships`, four edits of M, and a document with the node `web` and `infrastructure: {web: 2}` | P without `relationships` has no schema error. The other five have one each. The edits of M are `type: Service` in the feature, `source: webapp`, `features: [app]` in place of the node's `features` and `roles`, and `roles: {client: www}`. | | | S3 | Four documents that `language_diagnostics` rejects | `features: {missing: client}` in M (`sdl.semantic`) passes the schema. So does `name: demo` followed by `name: again` (`sdl.mapping_key_conflict`). `colour: blue` in M's node (`sdl.model.invalid`) and the key `Features` in M (`sdl.noncanonical_field`) fail it. | `yaml.safe_load` keeps only the last duplicate key, so the schema never sees the first | -| S4 | Read the schema's annotations, and request its `$id` with `curl -L` | `x-raes-validates-raw-source` is `false`. `$id` is `https://openrae.github.io/rae/schemas/sdl-authoring-input-v1.json`, and the request returned HTTP 404 on 2026-10-09 at 11:45 UTC. | | -| S5 | Compare the schema at tag `v6.0.1` with this revision. Run `language_diagnostics` on `name: demo` plus `execution_policy: {}` at both. | At `v6.0.1` the schema has 45 top-level properties and 602 `$defs` entries, and no top-level `execution_policy`. At this revision it has 46 and 606, including `execution_policy`. The document is valid at this revision. With the `v6.0.1` package tree on `PYTHONPATH`, it fails with `sdl.model.invalid`, "Extra inputs are not permitted", at `/execution_policy`. | The `v6.0.1` run used this revision's dependencies, not the wheel's | -| Y1 | yaml-language-server 1.24.0 from npm, over stdio, with the association that a VS Code `yamlValidation` entry for `*.sdl.yaml` produces. The schema is the prototype's copy of this revision's file, SHA-256 `162121460f19a555f4e7c469a3e4574fe4ed5ecabe628836bf1880ede5f1aba0`. | 20 of the 21 S1 files get no diagnostics. `first-scenario.sdl.yaml` gets "Value is not accepted" on `type`. Each of the five accepted shorthand and case forms in S2 gets one diagnostic, and the unresolved reference in S3 gets none. On an empty line inside `/nodes/web`, where only `type` is present, completion offers 15 fields. Hover on `type` shows the `NodeType` description. | Red Hat's YAML extension 1.24.0 uses this server version. VS Code itself was not started. | +| S4 | Read the schema's annotations, and request its `$id` with `curl -L` | `x-raes-validates-raw-source` is `false`. `$id` is `https://openrae.github.io/rae/schemas/sdl-authoring-input-v1.json`. The request returned HTTP 404 on 2026-10-09 at 11:45 UTC and on 2026-11-12 at 05:38 UTC. | | +| S5 | Compare the schema at tag `v6.0.1` with this revision. Run `language_diagnostics` on `name: demo` plus `execution_policy: {}` at both. | At `v6.0.1` the schema has 45 top-level properties and 602 `$defs` entries, and no top-level `execution_policy`. At this revision it has 49 and 656, including `execution_policy`, `safe_state` and `trial_parallelism`. The document is valid at this revision. With the `v6.0.1` package tree on `PYTHONPATH`, it fails with `sdl.model.invalid`, "Extra inputs are not permitted", at `/execution_policy`. | `v6.0.1` is still the latest release tag. Its run is from 2026-10-09 and used the dependencies of `35122105`, not the wheel's. | +| Y1 | yaml-language-server 1.24.0 from npm, over stdio, with the association that a VS Code `yamlValidation` entry for `*.sdl.yaml` produces. The schema is the prototype's copy of the file at `35122105`, SHA-256 `162121460f19a555f4e7c469a3e4574fe4ed5ecabe628836bf1880ede5f1aba0`. | 20 of the 21 S1 files at `35122105` get no diagnostics. `first-scenario.sdl.yaml` gets "Value is not accepted" on `type`. Each of the five accepted shorthand and case forms in S2 gets one diagnostic, and the unresolved reference in S3 gets none. On an empty line inside `/nodes/web`, where only `type` is present, completion offers 15 fields. Hover on `type` shows the `NodeType` description. | Observed at `35122105` only. The schema at this revision has SHA-256 `9acc517a1ed8dbdb466b3658aede90456cf5c5972e86d061d414944328deb2ce`, and S1 to S3 repeat the same schema results with it. Red Hat's YAML extension 1.24.0 uses this server version. VS Code itself was not started. | ## Inspected seams @@ -141,15 +149,15 @@ package paths are relative to `implementations/python/packages/`. | --- | --- | | Field lists | `raes/_language_metadata.py`: `SECTION_FIELD_COMPLETIONS`, a hand-kept tuple of field names for each section | | Completion depth | `raes/language_service.py`: `language_completions`. A path of one section name, and any longer path that does not end in a reference field, both read `SECTION_FIELD_COMPLETIONS[pointer[0]]`. | -| Reference completion | `raes/_language_metadata.py`: `REFERENCE_COMPLETION_TARGETS`; `raes/language_service.py`: `_completion_target_section`, `_reference_completion_items` | -| Completion on text that does not parse | `raes/language_service.py`: `_load_completion_data` returns the parse diagnostic instead of items | -| Semantic diagnostics | `raes/language_service.py`: `language_diagnostics` turns each message in `SDLValidationError.errors` into an `sdl.semantic` record without a path or range | -| Document context | Each helper takes text only. `language_diagnostics` calls `parse_sdl` without a path. | -| Handled exceptions | `raes/language_service.py`: `language_diagnostics` catches `SDLParseError` and `SDLValidationError`. `_load_completion_data`, `_try_declaration_index` and `language_format` catch `SDLParseError`. `_declaration_index_from_data` catches the pydantic `ValidationError`. `apply_structured_edit` catches `SDLParseError` and `ValueError`, and `_split_pointer_or_empty` catches `ValueError`. `raes/_language_references.py`: `_compose_yaml`, which `find_references` calls, catches `SDLParseError`. `_qualified_symbol_section`, `_is_variation_member_symbol` and `_bare_symbol` catch `TypeError` and `ValueError` from symbol parsing. `format_sdl_source` (`raes/formatting.py`) re-parses its output with `parse_sdl(..., skip_semantic_validation=True)`, and `parse_sdl` (`raes/parser.py`) still runs `SemanticValidator` for a `MaterializedScenario`, so `language_format` lets `SDLValidationError` reach the caller. Any other exception reaches the caller. | +| Reference completion | `raes/_language_metadata.py`: `REFERENCE_COMPLETION_TARGETS`; `raes/language_service.py`: `_completion_target_section`, `_reference_completion_items`, `_purpose_completion_items`; `raes/_reference_targetability.py`: `purposes_for_domain`; `raes/_language_relationship_candidates.py`: `filter_relationship_candidates` | +| Completion on text that does not parse | `raes/_language_context.py`: `load_completion_data` returns the parse diagnostic instead of items | +| Semantic diagnostics | `raes/language_service.py`: `language_diagnostics` uses the entry of `SDLValidationError.diagnostics` for a message when one exists. Otherwise it makes an `sdl.semantic` record without a path or range. `raes/_declaration_sources.py`: `semantic_source_diagnostics` makes an entry only for a finding whose owner declaration has a recorded source, and it ranges the entry on that declaration's key. | +| Document context | Each helper takes the text and an optional `path`. With a `path`, `_local_traversal` starts `CompositionTraversal.local_root` (`raes/_composition_budget.py`), and `raes/module_registry/resolution.py` refuses a registry import in that mode before any request. The helpers read no other open document. | +| Handled exceptions | `raes/language_service.py`: `language_diagnostics` catches `SDLParseError` and `SDLValidationError`. `language_format` catches `SDLParseError`. `apply_structured_edit` catches `SDLParseError` and `InvalidInputError`, and `_split_pointer_or_empty` catches `InvalidInputError`. `raes/_language_context.py`: `load_completion_data` and `reference_context` catch `SDLParseError`, `_composed_data` catches `SDLError` and the pydantic `ValidationError`, and `_validated` catches `ValidationError`. `raes/_language_relationship_candidates.py`: `_semantic_errors` catches `SDLValidationError`, and `_candidate_scenario` catches `ValidationError`. `raes/_language_structural_variation.py`: `_child_candidates` and `_record_candidates` catch `ValueError`. `raes/_language_references.py`: `_compose_yaml`, which `find_references` calls, catches `SDLParseError`. `_qualified_symbol_section`, `_is_variation_member_symbol` and `_bare_symbol` catch `TypeError` and `ValueError` from symbol parsing. `format_sdl_source` (`raes/formatting.py`) re-parses its output with `parse_sdl(..., skip_semantic_validation=True)`, and `parse_sdl` (`raes/parser.py`) still runs `SemanticValidator` for a `MaterializedScenario`, so `language_format` lets `SDLValidationError` reach the caller. Any other exception reaches the caller. | | Input limit | `raes/language_service.py`: `_MAX_INPUT_BYTES`, `_size_error` | | Formatting | `raes/language_service.py`: `language_format`, whose docstring reads "Migrate recognized legacy spellings and return canonical SDL YAML"; `raes/formatting.py`: `format_sdl_source` | | Module cache | `raes/module_registry/__init__.py`: `_oci_cache_dir` returns `.raes/module-cache` under the base directory | -| Import trust | `raes/composition/_expand.py`: `expand_sdl_modules` calls `load_lockfile` and `load_trust_policy` on the document's directory. `raes/module_registry/models.py`: `load_trust_policy` returns `TrustPolicy()`, with no registries, when `raes-trust.yaml` is missing; `RegistryTrustPolicy.require_signatures` defaults to `True`. `raes/module_registry/resolution.py`: `_resolve_oci_import` raises "is not allowed by trust policy" for an unlisted registry before any request. For a listed registry it then fetches the manifest, config and bundle, checks signatures if the registry requires them, and only then calls `_extract_bundle_to_cache`. The integration test `test_untrusted_and_unsigned_oci_imports_fail_closed` in `implementations/python/tests/test_sdl_module_registry.py` asserts both refusals. | +| Import trust | These steps apply to `parse_sdl` and `parse_sdl_file`, not to the helpers' path mode. `raes/composition/_expand.py`: `expand_sdl_modules` calls `load_lockfile` and `load_trust_policy` on the document's directory. `raes/module_registry/models.py`: `load_trust_policy` returns `TrustPolicy()`, with no registries, when `raes-trust.yaml` is missing; `RegistryTrustPolicy.require_signatures` defaults to `True`. `raes/module_registry/resolution.py`: `_resolve_oci_import` raises "is not allowed by trust policy" for an unlisted registry before any request. For a listed registry it then fetches the manifest, config and bundle, checks signatures if the registry requires them, and only then calls `_extract_bundle_to_cache`. The integration test `test_untrusted_and_unsigned_oci_imports_fail_closed` in `implementations/python/tests/test_sdl_module_registry.py` asserts both refusals. | | Schema for an installed `raes` | `raes_contracts/corpus.py`: `corpus_family_root`. The wheel build bundles the contract corpus at `raes_contracts/_corpus`, as the comment above `[tool.hatch.build.targets.wheel.hooks.custom]` in `implementations/python/pyproject.toml` describes. | | Python range | `implementations/python/pyproject.toml` at tag `v6.0.1`: `requires-python = ">=3.11,<3.15"` | @@ -157,10 +165,33 @@ package paths are relative to `implementations/python/packages/`. The prototype's [evidence scripts](https://github.com/doublewhy/vscode-raes-sdl/tree/feac96e0d5262d914c11deecb209426851915763/evidence) -automate L1 to L11, S1 to S3 and Y1. Each script takes the path of a RAES -checkout and runs under that checkout's Python environment. The Y1 script also -takes the directory where `yaml-language-server@1.24.0` is installed. At this -revision the scripts printed the results above, and L11 records three runs of -the latency script. S4 and S5 used `curl`, `git show` and `git archive` on tag -`v6.0.1`. L12 called the helpers directly, in-process, on the inputs that its -row names. +automate L1 to L11, S1 to S3 and Y1 as they ran at `35122105`. Each script +takes the path of a RAES checkout and runs under that checkout's Python +environment. The Y1 script also takes the directory where +`yaml-language-server@1.24.0` is installed. At `35122105` the scripts printed +the first results, and L11 records three runs of the latency script. S4 and S5 +used `curl`, `git show` and `git archive` on tag `v6.0.1`. + +The 2026-11-12 run at this revision called the helpers directly, in-process, on +the inputs that each row names. L1, L5, L8, S1 and L13 add calls that the +prototype's scripts do not make. L2 used the comparison that the paragraph +under the table describes. It also gave the first L2 result when it read the +schema and lists at `35122105`. + +## Changes since the first observation + +These rows changed between `35122105` and this revision: + +| ID | At `35122105` | At this revision | +| --- | --- | --- | +| L1 | Six reference items for the participant target, five of them invalid | One item, `two`, which is valid | +| L2 | 33 lists, 9 schema sections without a list | 34 lists, 11 schema sections without a list. The 18 differing lists and 69 missing fields are the same. | +| L5 | The helpers took no `path` | With a `path`, an owned semantic diagnostic has a path and a range on its owner's key | +| L8 | No `path` parameter, so the fixture could not be checked | With a `path`, the fixture is valid | +| L9 | 40 files | 60 files, with the same largest file | +| L13 | No `path` parameter | Path mode refuses registry imports without a request | +| S1 | 21 example files | 27 example files, with the same single schema rejection | +| S5 | 46 top-level properties and 606 `$defs` entries | 49 and 656 | + +L3, L4, L6, L7, L10, L12, S2, S3 and S4 gave the same results at both +revisions. diff --git a/docs/research/sdl-editor-support/feasibility.md b/docs/research/sdl-editor-support/feasibility.md index b165c4420f..a0aa60862e 100644 --- a/docs/research/sdl-editor-support/feasibility.md +++ b/docs/research/sdl-editor-support/feasibility.md @@ -1,13 +1,14 @@ # SDL editor support feasibility -Date: 2026-10-09 +Date: 2026-10-09, revised 2026-11-12 This note answers issue #1412: should RAES support SDL authoring in VS Code, and what scope would justify the work? It is research evidence, not contract authority. -The observations use repository revision `35122105b0c1bb648754625d8e3920eafa9bc599`. The -[evidence record](evidence.md) holds the probe inputs and outputs, and IDs such as L4 name its -rows. Claims about the editor client rest on a prototype extension at +The observations use repository revision `c7d499979eb32e16d9a2c46ca10b5aa1403b4511`. They were +first made at `35122105b0c1bb648754625d8e3920eafa9bc599`, and the [evidence record](evidence.md) +lists the results that changed. It holds the probe inputs and outputs, and IDs such as L4 name +its rows. Claims about the editor client rest on a prototype extension at [doublewhy/vscode-raes-sdl](https://github.com/doublewhy/vscode-raes-sdl/tree/feac96e0d5262d914c11deecb209426851915763). The prototype stays outside this repository because the issue asks for an exploration, and an extension here would be executable adoption. @@ -34,11 +35,14 @@ serves completion and hover. `raes_contracts.corpus` locates the schema that mat These helper changes would remove most of the work that each client now repeats: 1. Accept a line and column, or return a map from positions to paths. -2. Give semantic diagnostics a path and a source range. +2. Give each semantic diagnostic the path and range of the value at fault, also when the call + has no document path. 3. Derive field completion from the model at every depth, not from hand-kept lists. -4. Offer only references that the field's purpose accepts, as #1339 requires. -5. Accept an optional document path, so local imports can resolve. -6. Return a diagnostic for every input instead of raising. +4. Return a diagnostic for every input instead of raising. + +Two earlier prerequisites are already in the helpers. Reference completion for a participant +endpoint offers only the values that validation accepts (#1339, L1). Each helper accepts an +optional document path, so local imports resolve (L8). ### Alternatives @@ -71,7 +75,9 @@ has five helpers that return plain dictionaries. The MCP server wraps them as th | `apply_structured_edit` | Text, an operation, a JSON pointer, a value | Edited YAML with diagnostics | Code actions | The helpers reuse the repository parser and semantic validator. Each one refuses input over -64 KiB. The [language-service guide](../../explain/sdl/language-service.md) describes the +64 KiB. Each one also accepts an optional `path`, the document's file path. With it, local +imports resolve as in `parse_sdl_file`, and a registry import is refused before any request +(L8 and L13). The [language-service guide](../../explain/sdl/language-service.md) describes the payloads. ## What the editor integration needs @@ -99,7 +105,8 @@ stdio with the messages that the VS Code client sends. Nine test the cursor mapp tokenize SDL with VS Code's YAML grammar. All 31 passed locally against revision `35122105`, and in [CI run 37926735623](https://github.com/doublewhy/vscode-raes-sdl/actions/runs/37926735623) with `raes` 6.0.1 from PyPI. In that run, `vsce package` built a 571 KB package. Nothing is -published to the Marketplace, and no test starts VS Code itself. +published to the Marketplace, and no test starts VS Code itself. The prototype predates the +helpers' `path` parameter and does not pass one. | Feature | Result | | --- | --- | @@ -110,7 +117,8 @@ published to the Marketplace, and no test starts VS Code itself. | Schema association | Completion and hover work in yaml-language-server 1.24.0. Its diagnostics disagree with RAES. | | Formatting | Not offered, because the helper drops comments. | -The server forwards reference candidates unfiltered, so the #1339 mismatch stays visible. It +The server forwards the helper's reference candidates unchanged. At `35122105` this showed the +#1339 mismatch. At this revision the helper offers only the valid endpoint (L1). The server drops field names only after `key:` or inside a flow sequence, where no mapping key can go. That is a YAML rule, not an SDL rule. When a helper raises, the server reports one diagnostic and keeps running. Two tests check this with helpers that raise on a marker line. @@ -119,41 +127,43 @@ keeps running. Two tests check this with helpers that raise on a marker line. | Area | Observation | Evidence | | --- | --- | --- | -| Reference completion | For a participant relationship `target`, completion offers six declarations. Validation accepts one, the agent that is not the source. This is the #1339 mismatch. | L1; finding F5 in the [participant identity audit](../participant-identity/audit.md) | -| Field lists | The hand-kept lists miss 69 schema fields across 18 of 33 sections. The `features` list offers `version`, which validation rejects. | L2 and L3; `SECTION_FIELD_COMPLETIONS` in [`_language_metadata.py`](../../../implementations/python/packages/raes/_language_metadata.py) | +| Reference completion | Closed. For a participant relationship `target`, completion offers one item, the agent that is not the source, and validation accepts it. At `35122105` completion offered six declarations, which was the #1339 mismatch. | L1; finding F5 in the [participant identity audit](../participant-identity/audit.md) | +| Field lists | The hand-kept lists miss 69 schema fields across 18 of 34 sections. The `features` list offers `version`, which validation rejects. | L2 and L3; `SECTION_FIELD_COMPLETIONS` in [`_language_metadata.py`](../../../implementations/python/packages/raes/_language_metadata.py) | | Completion depth | Field completion uses only the section name. `/nodes`, `/nodes/web`, and `/nodes/web/resources` get the same nine fields. | L10; the field branches of `language_completions` in [`language_service.py`](../../../implementations/python/packages/raes/language_service.py) | | Cursor paths | The helpers take a JSON pointer, so each client derives one from the text. The prototype's mapper has 204 lines and skips multi-line flow collections, block scalars, anchors, tags, and complex keys. | Prototype [`server/sdl_cursor.py`](https://github.com/doublewhy/vscode-raes-sdl/blob/feac96e0d5262d914c11deecb209426851915763/server/sdl_cursor.py) | -| Source ranges | Semantic diagnostics have no path or range. An unknown field is ranged on its value, not its key. A YAML syntax error is a single point. | L5 | +| Source ranges | Without a document path, semantic diagnostics have no path or range. With one, an owned semantic error is ranged on its owner's key, not on the reference at fault. An unknown field is ranged on its value, not its key. A YAML syntax error is a single point. A refused registry import is ranged at line 1, column 1. | L5 and L13 | | Incomplete documents | Completion needs a document that parses. An unclosed `[` or a key without its colon gives a parse error and no items. | L4 | -| Imported modules | The helpers take no file path, so a document with `imports` is invalid even without semantic validation. `parse_sdl_file` accepts the same fixture. | L8 | -| Workspace context | The helpers see one document. Nothing indexes the workspace or follows references across files. | [`language_service.py`](../../../implementations/python/packages/raes/language_service.py) | +| Imported modules | Without a document path, a document with `imports` is invalid even without semantic validation. With one, local imports resolve. A registry import is refused, with advice to run `parse_sdl_file`, so the helpers cannot check a document that imports a registry module. | L8 and L13 | +| Workspace context | The helpers see one document, plus the local files that its imports name when the call has a path. Nothing indexes the workspace or finds references to the document from other files. | [`language_service.py`](../../../implementations/python/packages/raes/language_service.py) | | Robustness | Every helper turns `SDLParseError` into a result. `language_diagnostics` also turns `SDLValidationError` into one, and `apply_structured_edit` and `language_format` check their output with `language_diagnostics`. But `language_format` lets `SDLValidationError` reach the caller for a document with `materialization_provenance`. For such a document, `parse_sdl` runs the semantic validator even though `format_sdl_source` skips it. Other exceptions reach the caller too, unless a catch listed under inspected seams handles them, so a server must guard every call. | L12; handled exceptions under [inspected seams](evidence.md#inspected-seams) | -| Large scenarios | The largest repository example is 46 KB, under the 64 KiB limit. Each call parses the whole text. On that file a warm call took 160 to 175 ms at load average 7 to 13, and about 375 ms at load average 22. A cold start took 1.2 to 2.9 s. The machine was shared, so the timings are indicative. | L9 and L11 | +| Large scenarios | The largest repository example is 46 KB, under the 64 KiB limit. Each call parses the whole text. At `35122105`, on that file a warm call took 160 to 175 ms at load average 7 to 13, and about 375 ms at load average 22. A cold start took 1.2 to 2.9 s. The machine was shared, so the timings are indicative. | L9 and L11 | | Formatting and edits | Both drop comments and expand shorthand. Formatting also lowers enum case and migrates recognized legacy spellings. | L6 and L7; the `language_format` docstring | | Schema validation | The schema describes the normalized object, so `x-raes-validates-raw-source` is `false`. It flags accepted shorthand and case variants and misses semantic errors. | S2 to S4 and Y1 | | Schema versions | The schema's `$id` returned HTTP 404. The schema on `dev` declares `execution_policy`, which `raes` 6.0.1 rejects. | S4 and S5 | -The schema check flags one of the 21 accepted examples (S1): the quickstart's -`first-scenario.sdl.yaml` uses `type: Switch`. Any fix for imports must also handle registry -imports. When `parse_sdl` gets a path and the document has `imports`, it reads `raes.lock.json` -and `raes-trust.yaml` from the document's directory. It refuses an OCI import before any request -unless that trust file lists the import's registry. For a listed registry it fetches the module. -Once the registry's signature policy passes, it writes `.raes/module-cache` next to the document. -The trust file is workspace content, so a path-aware server should resolve imports only in a -trusted workspace (Import trust under [inspected seams](evidence.md#inspected-seams)). +The schema check flags one of the 27 accepted examples (S1): the quickstart's +`first-scenario.sdl.yaml` uses `type: Switch`. The helpers' path mode refuses registry imports +and makes no request, even when `raes-trust.yaml` lists the registry (L13). To check such a +document, a server would call `parse_sdl` with a path. When `parse_sdl` gets a path and the +document has `imports`, it reads `raes.lock.json` and `raes-trust.yaml` from the document's +directory. It refuses an OCI import before any request unless that trust file lists the import's +registry. For a listed registry it fetches the module. Once the registry's signature policy +passes, it writes `.raes/module-cache` next to the document. The trust file is workspace +content, so a server should resolve registry imports only in a trusted workspace (Import trust +under [inspected seams](evidence.md#inspected-seams)). ## Answers to the issue's questions ### Which features are most useful Diagnostics come first. They need no cursor mapping and reuse the full validator. Reference -completion and go to definition come next. They help most on reference-heavy sections, but -completion inherits the #1339 mismatch. Schema hover shows field descriptions. Highlighting adds -little over plain YAML. Snippets and a document outline were not prototyped. Snippets give no -diagnostics or references (alternative 2), so they rank low. None of the five helpers returns the -document's declarations with their ranges, so an outline needs new helper or server work. It -would help in long scenarios, but it ranks after diagnostics and references. Formatting should -wait until comments survive it. +completion and go to definition come next. They help most on reference-heavy sections. Since +#1339 closed, participant-endpoint completion agrees with validation (L1). Schema hover shows +field descriptions. Highlighting adds little over plain YAML. Snippets and a document outline +were not prototyped. Snippets give no diagnostics or references (alternative 2), so they rank +low. None of the five helpers returns the document's declarations with their ranges, so an +outline needs new helper or server work. It would help in long scenarios, but it ranks after +diagnostics and references. Formatting should wait until comments survive it. ### Can YAML support and the schema provide enough value @@ -187,20 +197,24 @@ run inside Monaco. ### What gaps exist See [Gaps](#gaps). The cursor-to-path mapping, source ranges, incomplete documents, imports, -workspace context, and large scenarios each have a row there. So does the #1339 completion -mismatch. +workspace context, and large scenarios each have a row there. The reference completion row +records that the #1339 mismatch is closed. ### How packaging and versions should work -- **File association.** The repository names 40 SDL files `*.sdl.yaml` (L9). VS Code prefers the +- **File association.** The repository names 60 SDL files `*.sdl.yaml` (L9). VS Code prefers the longest matching extension, so `.sdl.yaml` wins over `.yaml` ([R1](#references)). Files such as the composition fixtures end in `.yaml` and need a `files.associations` entry. - **Version selection.** The server runs in an interpreter that the author picks, and it reports the `raes` version. A schema bundled in the extension can drift from that version, as S5 shows. The `raes` wheel bundles its own copy, reachable through `raes_contracts.corpus.corpus_family_root`, so a server could serve the matching schema. -- **Local and offline use.** The helpers work on the buffer text in-process. They read no other - document and use no network. Resolving imports would change that, so it should stay opt-in. + `raes` 6.0.1, the latest release, predates the helpers' `path` parameter and the #1339 fix. + A server must therefore check which helper features the installed `raes` has. +- **Local and offline use.** The helpers work on the buffer text in-process. Without a path, + they read no other document. With a path, they also read the local files that the imports + name. In both modes they use no network (L13). Resolving registry imports would use the + network, so it should stay opt-in. Red Hat's YAML extension downloads the SchemaStore catalog by default (`yaml.schemaStore.enable`, [R3](#references)). - **Packaging.** `vsce package` works without bundling Python. Authors install `raes` from PyPI. @@ -213,7 +227,7 @@ mismatch. - **SDL semantics stay in RAES.** The prototype server adds no SDL rules. The grammar's key list mirrors the schema's top-level properties, and a test checks that they match. - **No launch and no fetch.** The helpers parse and validate text. Nothing runs a scenario or - downloads content. + downloads content. Path mode refuses registry imports before any request (L13). - **Formatting.** `language_format` drops comments, expands shorthand, normalizes enum case, and migrates legacy spellings. Format on save would rewrite authored text, so formatting should be an explicit command at most. @@ -222,13 +236,14 @@ mismatch. ## Method -The probes call the helpers on small documents and on the repository examples. They compare the -results with the published schema and with yaml-language-server 1.24.0. Version 1.24.0 is the -server that Red Hat's YAML extension 1.24.0 uses. Nothing was mocked. The +The probes call the helpers on small documents and on the repository examples. They first ran at +`35122105`. They ran again at `c7d49997`, after #1446 and the helpers' path mode merged. They +compare the results with the published schema and with yaml-language-server 1.24.0. Version +1.24.0 is the server that Red Hat's YAML extension 1.24.0 uses. Nothing was mocked. The [evidence record](evidence.md) gives each probe's input and output and the source locations -behind the other claims. It also links to the scripts that reproduce most probes. They live in -the prototype's repository, not in this one. The prototype's tests and CI run cover only the -editor client. +behind the other claims. It also links to the scripts that reproduced most probes at `35122105`. +They live in the prototype's repository, not in this one. The prototype's tests and CI run cover +only the editor client. ## References From 84eec730154caeb8f25b47799047b3ff65d08d91 Mon Sep 17 00:00:00 2001 From: Brad Edwards Date: Thu, 12 Nov 2026 06:52:06 +0100 Subject: [PATCH 4/4] docs(sdl): meet the STE lint limits in the feasibility note Divide one long instruction and six long paragraphs, as the shared STE stage of the docs lane on dev requires. The content does not change. --- .../sdl-editor-support/feasibility.md | 45 ++++++++++++------- 1 file changed, 29 insertions(+), 16 deletions(-) diff --git a/docs/research/sdl-editor-support/feasibility.md b/docs/research/sdl-editor-support/feasibility.md index a0aa60862e..2ac38167cb 100644 --- a/docs/research/sdl-editor-support/feasibility.md +++ b/docs/research/sdl-editor-support/feasibility.md @@ -35,8 +35,8 @@ serves completion and hover. `raes_contracts.corpus` locates the schema that mat These helper changes would remove most of the work that each client now repeats: 1. Accept a line and column, or return a map from positions to paths. -2. Give each semantic diagnostic the path and range of the value at fault, also when the call - has no document path. +2. Give each semantic diagnostic the path and range of the value at fault. Do this also when + the call has no document path. 3. Derive field completion from the model at every depth, not from hand-kept lists. 4. Return a diagnostic for every input instead of raising. @@ -102,7 +102,9 @@ A VS Code integration needs these parts: The prototype has a TextMate grammar, a schema association, and a stdio language server. The server uses only the Python standard library and `raes`. Sixteen tests drive the server over stdio with the messages that the VS Code client sends. Nine test the cursor mapper, and six -tokenize SDL with VS Code's YAML grammar. All 31 passed locally against revision `35122105`, +tokenize SDL with VS Code's YAML grammar. + +All 31 passed locally against revision `35122105`, and in [CI run 37926735623](https://github.com/doublewhy/vscode-raes-sdl/actions/runs/37926735623) with `raes` 6.0.1 from PyPI. In that run, `vsce package` built a 571 KB package. Nothing is published to the Marketplace, and no test starts VS Code itself. The prototype predates the @@ -118,8 +120,9 @@ helpers' `path` parameter and does not pass one. | Formatting | Not offered, because the helper drops comments. | The server forwards the helper's reference candidates unchanged. At `35122105` this showed the -#1339 mismatch. At this revision the helper offers only the valid endpoint (L1). The server -drops field names only after `key:` or inside a flow sequence, where no mapping key can go. That +#1339 mismatch. At this revision the helper offers only the valid endpoint (L1). + +The server drops field names only after `key:` or inside a flow sequence, where no mapping key can go. That is a YAML rule, not an SDL rule. When a helper raises, the server reports one diagnostic and keeps running. Two tests check this with helpers that raise on a marker line. @@ -142,9 +145,13 @@ keeps running. Two tests check this with helpers that raise on a marker line. | Schema versions | The schema's `$id` returned HTTP 404. The schema on `dev` declares `execution_policy`, which `raes` 6.0.1 rejects. | S4 and S5 | The schema check flags one of the 27 accepted examples (S1): the quickstart's -`first-scenario.sdl.yaml` uses `type: Switch`. The helpers' path mode refuses registry imports +`first-scenario.sdl.yaml` uses `type: Switch`. + +The helpers' path mode refuses registry imports and makes no request, even when `raes-trust.yaml` lists the registry (L13). To check such a -document, a server would call `parse_sdl` with a path. When `parse_sdl` gets a path and the +document, a server would call `parse_sdl` with a path. + +When `parse_sdl` gets a path and the document has `imports`, it reads `raes.lock.json` and `raes-trust.yaml` from the document's directory. It refuses an OCI import before any request unless that trust file lists the import's registry. For a listed registry it fetches the module. Once the registry's signature policy @@ -158,18 +165,23 @@ under [inspected seams](evidence.md#inspected-seams)). Diagnostics come first. They need no cursor mapping and reuse the full validator. Reference completion and go to definition come next. They help most on reference-heavy sections. Since -#1339 closed, participant-endpoint completion agrees with validation (L1). Schema hover shows -field descriptions. Highlighting adds little over plain YAML. Snippets and a document outline -were not prototyped. Snippets give no diagnostics or references (alternative 2), so they rank -low. None of the five helpers returns the document's declarations with their ranges, so an -outline needs new helper or server work. It would help in long scenarios, but it ranks after -diagnostics and references. Formatting should wait until comments survive it. +#1339 closed, participant-endpoint completion agrees with validation (L1). + +Schema hover shows field descriptions. Highlighting adds little over plain YAML. Snippets and a +document outline were not prototyped. Snippets give no diagnostics or references (alternative +2), so they rank low. None of the five helpers returns the document's declarations with their +ranges, so an outline needs new helper or server work. + +An outline would help in long scenarios, but it ranks after diagnostics and references. +Formatting should wait until comments survive it. ### Can YAML support and the schema provide enough value They are enough for completion and hover, not for diagnostics. Inside a node, schema completion offered the 15 node fields not yet present (Y1). The helper's list for nodes has 9 fields in all -(L10). Hover showed the schema's descriptions. But schema diagnostics flagged all five accepted +(L10). Hover showed the schema's descriptions. + +But schema diagnostics flagged all five accepted shorthand and case forms that the probe tried. They also missed an unresolved reference. A dedicated extension is warranted only to add RAES diagnostics and reference features. @@ -239,8 +251,9 @@ records that the #1339 mismatch is closed. The probes call the helpers on small documents and on the repository examples. They first ran at `35122105`. They ran again at `c7d49997`, after #1446 and the helpers' path mode merged. They compare the results with the published schema and with yaml-language-server 1.24.0. Version -1.24.0 is the server that Red Hat's YAML extension 1.24.0 uses. Nothing was mocked. The -[evidence record](evidence.md) gives each probe's input and output and the source locations +1.24.0 is the server that Red Hat's YAML extension 1.24.0 uses. Nothing was mocked. + +The [evidence record](evidence.md) gives each probe's input and output and the source locations behind the other claims. It also links to the scripts that reproduced most probes at `35122105`. They live in the prototype's repository, not in this one. The prototype's tests and CI run cover only the editor client.