You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit c159f08
Browse filesBrowse the repository at this point in the historyBrowse files
authored
feat(workflows): install custom step types from local dirs and archives (#4769)
* feat(workflows): install custom step types from local dirs and archives (#4695)
`specify workflow step add` gains `--dev <directory>` and `--from <archive-url>` alongside the existing catalog source. All three converge on a new `step/installer.py` domain module that owns package validation (shape, symlink/special-file rejection, 512-file/50 MiB limits), same-filesystem staging with revalidation, atomic commit, `--force` replacement, and source-kind-only registry provenance. Direct URLs require a default-deny trust prompt before any request.
Docs document the local-authoring flow and the deferred bundle-local limitation.
Assisted-by: opencode (model: deepseek-v4.1-flash, autonomous)
* fix(workflows): harden local step installation
Assisted-by: OpenCode (model: gpt-5.6-terra, autonomous)
Assisted-by: OpenCode (model: deepseek-v4.1-flash, autonomous)
* fix(workflows): address step install review findings
Assisted-by: OpenCode (model: deepseek-v4.1-flash, autonomous)
* fix(workflows): address remaining step install findings
Allow archive byte-format detection without transport hints, escape user-controlled installer errors, reject case-folded orphan directories, and exercise install locking on Windows.
Assisted-by: OpenCode (model: github-copilot/gpt-6-luna, autonomous)
* fix(workflows): bound step package traversal depth and entries
Replace recursive package validation and staging walks with iterative
traversal so deeply nested --dev trees fail with StepInstallError instead
of RecursionError. Enforce a 32-level directory depth limit and count
directories toward the 512-entry package budget in both validation and
copy, covering trees that change after validation.
Assisted-by: GitHub Copilot CLI (model: unknown, Auto mode, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* test(workflows): compare resolved paths in force-reinstall failure mocks
The installer operates on resolved step paths, so on macOS (where the temp
directory lives under the /var -> /private/var symlink) the rmtree and
os.replace mocks never matched the unresolved target and the expected
StepInstallError was not raised. Compare resolved paths instead.
Assisted-by: GitHub Copilot CLI (model: unknown, Auto mode, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Update documentation on installed-package policy limits
Clarify the impact of excluded entries on budget limits.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Refactor step ID collision handling in installer.py
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix(workflows): serialize step remove with installs via shared project lock
`workflow step remove` mutated the step directory and registry without the
install lock, so a concurrent install could resurrect a removed entry or
leave a newly installed package unregistered through a stale registry
snapshot.
- Extract the duplicated workflow/step lock logic into
`shared_infra._exclusive_project_lock`; both install transactions now use it.
- Run step removal under the step lock and resolve the registry and step
directory only after the lock is acquired.
- Report only lock-acquisition failures as lock errors; exceptions raised
inside the critical section now propagate unchanged.
- Add helper tests and deterministic remove-vs-install race tests (both
orderings), plus lock-failure and rollback coverage.
Assisted-by: GitHub Copilot CLI (model: Claude Opus 5.5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(workflows): bound step package traversal by the entry ceiling
The 512-entry ceiling did not bound validation work: each directory
listing was fully materialized and sorted before any entry was counted,
and excluded subtrees (`.git`, `__pycache__`) were walked in full without
consuming the budget, so a `--dev` checkout with a large `.git` cost
unbounded time and memory.
- Stream `os.scandir` through a shared helper that drops excluded names
unread and raises the entry-limit error once the remaining budget is
exceeded, before sorting. Validation and staging copy both use it.
- Prune excluded directories instead of descending into them, matching
`bundles/packager.py`. Their contents were already never staged, so a
symlink or deep nesting inside `.git` no longer fails validation.
- Update the workflow reference to describe the exclusion behavior.
- Replace the symlink-inside-`.git` rejection test with pruning coverage
and add read-count regressions for validation, a cross-directory
budget, and the staging copy.
Assisted-by: OpenCode (model: Claude Opus 5.5, autonomous)
* fix(workflows): escape step IDs in step remove and add messages
Step IDs may legally contain `[`/`]`, so interpolating them unescaped into
Rich markup rendered an ID such as `[red]step` as `step`. `step remove`
printed the ID, step directory, and exception text raw on every path, and
the discovery-only warning in `step add` did the same.
- Escape the step ID, directory path, and exception text in all
`step remove` output, matching `step info`.
- Escape the step ID in the `step add` discovery-only catalog warning.
- Add CLI regressions for the not-installed, lock-failure, orphan-removal,
directory-delete-failure, and discovery-only paths.
Assisted-by: OpenCode (model: Claude Opus 5.5, autonomous)
* fix(workflows): call the step package budget an entry limit
The package budget counts files and directories together, but the
validation, staging, and catalog preflight errors reported it as a
"file limit". Report it as an "N-entry limit (files and directories
combined)" and update the matching test assertions.
Assisted-by: OpenCode (model: Claude Opus 5.5, autonomous)
* fix(workflows): report step lock failures once
`step remove` shares the step lock with `step add`, but the shared helper
labelled every acquisition failure as "Failed to lock step installation".
`step remove` then added its own prefix, printing "Failed to lock step
removal '<id>': Failed to lock step installation: <error>".
- Make the helper's message operation-neutral ("Failed to acquire the
step lock"), since install and remove both use it.
- In `step remove`, report the underlying acquisition error after the
removal prefix instead of the helper's message.
- Assert a single lock message on the remove path, and add a `step add
--dev` lock-failure regression. Both fail against the previous wording.
Assisted-by: OpenCode (model: Claude Opus 5.5, autonomous)
* Update context string in project lock
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix(bundles): restore failed step refreshes under the step lock
When a bundle step refresh removes a step and the reinstall fails, the
rollback reloaded `step-registry.json`, put the old entry back and saved,
all without the step lock that `step add` and `step remove` hold. A
concurrent step operation could commit between that reload and save, or
save a stale snapshot over the restored entry, and lose a registry entry.
- Run the rollback's directory and registry restore inside
`_step_install_transaction`, reloading the registry after the lock is
taken.
- Skip the restore if the step was registered again after the removal,
so a newer package and entry are not overwritten by the stale backup.
- If the lock can't be acquired, leave the project untouched and add a
note to the reinstall error instead of restoring unlocked.
- Add regressions for a concurrent writer holding the lock during the
rollback, a concurrent reinstall of the same step, and lock failure;
all three fail against the previous rollback. Also cover `step remove`
with a symlinked lock file, which fails against the old `step install`
lock context.
Assisted-by: OpenCode (model: Claude Opus 5.5, autonomous)
* revert(bundles): keep the step refresh rollback out of this PR
Reverts the `bundles/primitives.py` part of c7f8a51 and its three
rollback tests. The step refresh rollback is pre-existing code that this
PR did not otherwise touch; locking it pulled the bundle refresh flow into
review scope, and further findings there (snapshot and removal before the
lock, restore failures masking the reinstall error) are also pre-existing
on `main`. It belongs in a separate change for bundle refresh atomicity.
The `step remove` symlinked-lock test from c7f8a51 is kept: it covers the
step lock and removal locking that this PR adds.
Assisted-by: OpenCode (model: Claude Opus 5.5, autonomous)
---------
Co-authored-by: Markus <markus@example.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copy file name to clipboardExpand all lines: docs/reference/bundles.md
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -81,6 +81,8 @@ The source may also be a bundle directory or `.zip` artifact. Refresh uses the s
81
81
82
82
A local bundle source supplies the manifest, not its component payloads. Components resolved through catalogs still require network access to refresh, even when already installed. Add `--offline` only when the components being installed or refreshed ship with Spec Kit; otherwise the command reports which component needs network access. Re-run without `--offline` to fetch that component through its catalog.
83
83
84
+
> **Step payloads resolve through the step catalog only.** A bundle's `provides.steps` entries still resolve exclusively through the active step catalogs. Bundle-local `steps/<id>/` payloads and relative `provides.steps[].source` overrides are **not** resolved in this release, so a step declared that way cannot be installed offline. To ship a step with a bundle today, publish it to a step catalog the bundle's users can reach.
Copy file name to clipboardExpand all lines: docs/reference/workflows.md
+148Lines changed: 148 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -582,6 +582,154 @@ specify workflow run speckit -i spec="Build a kanban board with drag-and-drop ta
582
582
583
583
> **Security note:** a `shell` step runs a local command with **your** privileges. There is no capability sandbox — `requires` is an advisory pre-condition block (spec-kit version, integrations), not a runtime gate, so it does **not** restrict what a step can do. In particular there is no `requires.permissions` capability gate: it is rejected by validation precisely because it would imply a sandbox that does not exist. Review any catalog or downloaded workflow before running it, and use a `gate` step to require explicit approval before sensitive or destructive shell commands.
584
584
585
+
### Custom step packages
586
+
587
+
Custom step types are installed with `specify workflow step`. A step is a
588
+
directory package containing metadata and executable Python:
589
+
590
+
```text
591
+
my-step/
592
+
├── step.yml # required, at the package root
593
+
├── __init__.py # required, at the package root
594
+
└── helpers.py # optional nested modules and data files
595
+
```
596
+
597
+
`step.yml`declares the step's identity. `step.type_key` must exactly match the
598
+
`<step_id>`passed on the command line — the ID is never inferred from package
599
+
content:
600
+
601
+
```yaml
602
+
step:
603
+
type_key: my-step
604
+
name: My Step
605
+
version: 0.1.0
606
+
author: you
607
+
description: What this step does
608
+
```
609
+
610
+
`__init__.py` must define a `StepBase` subclass whose `type_key` matches:
611
+
612
+
```python
613
+
from specify_cli.workflows.base import StepBase, StepResult
0 commit comments