Skip to content

Commit b5c0120

Browse files
mnriemCopilot
andcommitted
fix: address catalog idempotency review
Merge current main and escape user-controlled catalog values before Rich rendering. Add regressions for markup-bearing no-op and conflict output across integration, workflow, and step catalogs. Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2 parents 7466e2a + 690e61d commit b5c0120

20 files changed

Lines changed: 1010 additions & 68 deletions

File tree

‎docs/reference/bundles.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -175,6 +175,10 @@ specify bundle catalog add <url>
175175

176176
Registers a project-scoped catalog source and persists it.
177177

178+
Adding a source is idempotent (identity is the source **id or url**): re-running `catalog add` with the same id/url and identical `--policy`/`--priority` is a successful no-op (exit code 0), so it is safe to include in a re-runnable workflow. Re-adding a matching id/url with *different* settings is rejected as a conflict rather than silently overwriting the existing source — remove it first to change it.
179+
180+
Surrounding whitespace in source ids and URLs is ignored when matching identities and comparing settings. No-ops and conflicts leave the existing configuration unchanged; they do not rewrite stored values to normalize them.
181+
178182
### Remove a Catalog Source
179183

180184
```bash

‎docs/reference/extensions.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -156,6 +156,12 @@ specify extension catalog add <url>
156156
157157
Adds a catalog to the project's `.specify/extension-catalogs.yml`.
158158
159+
Adding a catalog is idempotent (identity is the catalog **name**): re-running `catalog add` with the same name and identical settings is a successful no-op (exit code 0), so it is safe to include in a re-runnable workflow. Re-adding the same name with *different* settings is rejected as a conflict rather than silently overwriting the existing entry — remove it first to change it.
160+
161+
Surrounding whitespace in catalog names and URLs is ignored when comparing entries and stripped from newly added entries. A no-op leaves the existing configuration unchanged.
162+
163+
Stored priorities may use numeric strings, but YAML booleans (`true`/`false`) are invalid and are never equivalent to integer priorities (`1`/`0`). Re-adding a matching catalog with an invalid stored priority reports a conflict.
164+
159165
### Remove a Catalog
160166
161167
```bash

‎docs/reference/integrations.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -233,6 +233,8 @@ specify integration catalog add <url>
233233

234234
Adds a custom catalog URL to the project's `.specify/integration-catalogs.yml`. The URL must use HTTPS (except `http://localhost`, `http://127.0.0.1`, or `http://[::1]` for local testing).
235235

236+
Adding a catalog is idempotent (identity is the catalog **URL**): re-running `catalog add` with the same URL and the same (or no) `--name` is a successful no-op (exit code 0), so it is safe to include in a re-runnable workflow. Re-adding the same URL with a *different* `--name` is rejected as a conflict rather than silently overwriting the existing entry — remove it first to change it.
237+
236238
### Remove a Catalog
237239

238240
```bash

‎docs/reference/presets.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -123,6 +123,12 @@ specify preset catalog add <url>
123123

124124
Adds a catalog to the project's `.specify/preset-catalogs.yml`.
125125

126+
Adding a catalog is idempotent (identity is the catalog **name**): re-running `catalog add` with the same name and identical settings is a successful no-op (exit code 0), so it is safe to include in a re-runnable workflow. Re-adding the same name with *different* settings is rejected as a conflict rather than silently overwriting the existing entry — remove it first to change it.
127+
128+
Surrounding whitespace in catalog URLs is ignored when comparing entries and stripped from newly added entries. A no-op leaves the existing configuration unchanged.
129+
130+
Stored priorities may use numeric strings, but YAML booleans (`true`/`false`) are invalid and are never equivalent to integer priorities (`1`/`0`). Re-adding a matching catalog with an invalid stored priority reports a conflict.
131+
126132
### Remove a Catalog
127133

128134
```bash

‎docs/reference/workflows.md‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -405,6 +405,10 @@ specify workflow catalog add <url>
405405

406406
Adds a custom catalog URL to the project's `.specify/workflow-catalogs.yml`.
407407

408+
Adding a catalog is idempotent (identity is the catalog **URL**): re-running `catalog add` with the same URL and the same (or no) `--name` is a successful no-op (exit code 0), so it is safe to include in a re-runnable workflow. Re-adding the same URL with a *different* `--name` is rejected as a conflict rather than silently overwriting the existing entry — remove it first to change it.
409+
410+
Surrounding whitespace in catalog URLs is ignored when comparing entries and stripped from newly added entries. A no-op leaves the existing configuration unchanged.
411+
408412
### Remove a Catalog
409413

410414
```bash
@@ -422,6 +426,20 @@ Catalogs are resolved in this order (first match wins):
422426
3. **User config** — `~/.specify/workflow-catalogs.yml`
423427
4. **Built-in defaults** — official catalog + community catalog
424428

429+
### Step Catalogs
430+
431+
Custom step types have a separate catalog stack:
432+
433+
```bash
434+
specify workflow step catalog list
435+
specify workflow step catalog add <url> [--name <name>]
436+
specify workflow step catalog remove <index>
437+
```
438+
439+
`step catalog add` writes to `.specify/step-catalogs.yml`. Like workflow catalogs, step catalogs use the **URL** as their identity, ignoring surrounding whitespace. Adding the same URL with the same (or no) `--name` is a successful no-op (exit code 0) that leaves the configuration unchanged. A different `--name` for that URL is a conflict (exit code 1); remove the existing entry first to change it. New entries store the URL without surrounding whitespace.
440+
441+
`step catalog list` shows the active sources, and `step catalog remove` removes a project entry by its index. Step catalog resolution uses `SPECKIT_STEP_CATALOG_URL`, then the project config, then `~/.specify/step-catalogs.yml`, then built-in defaults.
442+
425443
## Workflow Definition
426444

427445
Workflows are defined in YAML files. Here is the built-in **Full SDD Cycle** workflow that ships with Spec Kit:

‎src/specify_cli/bundler/commands_impl/catalog_config.py‎

Lines changed: 33 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -139,7 +139,7 @@ def add_source(
139139
policy: str,
140140
priority: int,
141141
source_id: str | None = None,
142-
) -> CatalogSource:
142+
) -> tuple[CatalogSource, str]:
143143
url = url.strip()
144144
if not url:
145145
raise BundlerError("A catalog url is required.")
@@ -186,21 +186,44 @@ def add_source(
186186
resolved_id = (source_id or _derive_id(url)).strip()
187187

188188
catalogs = _read(project_root)
189-
for existing in catalogs:
190-
if existing.get("id") == resolved_id or existing.get("url") == url:
191-
raise BundlerError(
192-
f"Catalog source '{resolved_id}' (or url) already exists in this project."
193-
)
194-
195-
entry = {
189+
desired = {
196190
"id": resolved_id,
197191
"url": url,
198192
"priority": int(priority),
199193
"install_policy": install_policy.value,
200194
}
201-
catalogs.append(entry)
195+
for existing in catalogs:
196+
if (
197+
str(existing.get("id", "")).strip() == resolved_id
198+
or str(existing.get("url", "")).strip() == url
199+
):
200+
# Idempotent add (#4505): identity is the source id or url. A rerun
201+
# requesting the same settings is a successful no-op; differing
202+
# settings are a conflict rather than a silent overwrite.
203+
#
204+
# Parse the matching entry through CatalogSource.from_dict first:
205+
# _read() only checks that entries are mappings, so a hand-edited
206+
# entry may carry a non-integer priority. Normalizing here surfaces
207+
# that as a clean BundlerError (matching catalog parsing) instead of
208+
# leaking int()'s ValueError/OverflowError past the CLI's
209+
# `except BundlerError`, and lets supported representations (e.g. a
210+
# string priority) compare equal to the requested defaults.
211+
existing_source = CatalogSource.from_dict(dict(existing), Scope.PROJECT)
212+
if (
213+
existing_source.id == resolved_id
214+
and existing_source.url == url
215+
and existing_source.priority == desired["priority"]
216+
and existing_source.install_policy.value == desired["install_policy"]
217+
):
218+
return existing_source, "unchanged"
219+
raise BundlerError(
220+
f"Catalog source '{resolved_id}' (or url) already exists in this "
221+
"project with different settings. Remove it first to change it."
222+
)
223+
224+
catalogs.append(desired)
202225
_write(project_root, catalogs)
203-
return CatalogSource.from_dict(entry, Scope.PROJECT)
226+
return CatalogSource.from_dict(desired, Scope.PROJECT), "added"
204227

205228

206229
def remove_source(project_root: Path, id_or_url: str) -> str:

‎src/specify_cli/commands/bundle/__init__.py‎

Lines changed: 11 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -672,15 +672,21 @@ def catalog_add(
672672
project_root = require_project_root()
673673
from ...bundler.commands_impl.catalog_config import add_source
674674

675-
source = add_source(project_root, url, policy=policy, priority=priority, source_id=source_id)
675+
source, status = add_source(project_root, url, policy=policy, priority=priority, source_id=source_id)
676676
except BundlerError as exc:
677677
_fail(str(exc))
678678
return
679679

680-
console.print(
681-
f"[green]✓[/green] Added catalog '{_escape_markup(str(source.id))}' "
682-
f"(priority {source.priority}, {source.install_policy.value})."
683-
)
680+
if status == "unchanged":
681+
console.print(
682+
f"[green]✓[/green] Catalog '{_escape_markup(str(source.id))}' is already "
683+
f"configured (priority {source.priority}, {source.install_policy.value})."
684+
)
685+
else:
686+
console.print(
687+
f"[green]✓[/green] Added catalog '{_escape_markup(str(source.id))}' "
688+
f"(priority {source.priority}, {source.install_policy.value})."
689+
)
684690

685691

686692
@bundle_catalog_app.command("remove")

‎src/specify_cli/extensions/_commands.py‎

Lines changed: 55 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -327,6 +327,36 @@ def install_extension_from_url(
327327
pass
328328

329329

330+
def _normalize_catalog_priority(value: object) -> int | None:
331+
"""Normalize a stored catalog priority the way the catalog reader does.
332+
333+
The reader (``specify_cli/catalogs.py``) accepts integer-string priorities
334+
like ``"10"`` but rejects bools. Mirror that here so an equivalent rerun
335+
whose persisted priority is a supported string representation is still a
336+
no-op rather than a false conflict (#4505). A value that cannot be
337+
normalized returns ``None`` so it cannot compare equal to an integer.
338+
"""
339+
if isinstance(value, bool):
340+
return None
341+
try:
342+
return int(value)
343+
except (TypeError, ValueError, OverflowError):
344+
return None
345+
346+
347+
def _normalize_catalog_install_allowed(value: object) -> bool:
348+
"""Normalize a stored ``install_allowed`` the way the catalog reader does.
349+
350+
The reader treats the strings ``"true"``/``"yes"``/``"1"`` (case- and
351+
whitespace-insensitive) as truthy; everything else falls back to ``bool``.
352+
Comparing raw values instead would report ``install_allowed: "false"`` as a
353+
conflict because ``bool("false")`` is ``True``.
354+
"""
355+
if isinstance(value, str):
356+
return value.strip().lower() in ("true", "yes", "1")
357+
return bool(value)
358+
359+
330360
def _load_catalog_command_config(project_root: Path, config_path: Path) -> dict:
331361
"""Load extension catalog CLI config with user-facing shape errors."""
332362
try:
@@ -615,6 +645,8 @@ def catalog_add(
615645

616646
project_root = _require_specify_project()
617647
specify_dir = project_root / ".specify"
648+
url = url.strip()
649+
name = name.strip()
618650

619651
# Validate URL
620652
tmp_catalog = ExtensionCatalog(project_root)
@@ -640,10 +672,30 @@ def catalog_add(
640672
safe_name = _escape_markup(name)
641673
safe_url = _escape_markup(url)
642674

643-
# Check for duplicate name
675+
# Idempotent add (#4505): a rerun that requests an identical entry is a
676+
# successful no-op so the same `catalog add` can live in a re-runnable
677+
# workflow without failing. A same-name entry whose settings differ is
678+
# still a conflict — we refuse to silently change priority/install
679+
# permissions and ask the user to remove it first.
644680
for existing in catalogs:
645-
if isinstance(existing, dict) and existing.get("name") == name:
646-
console.print(f"[yellow]Warning:[/yellow] A catalog named '{safe_name}' already exists.")
681+
if isinstance(existing, dict) and str(existing.get("name", "")).strip() == name:
682+
if (
683+
str(existing.get("url", "")).strip() == url
684+
and _normalize_catalog_priority(existing.get("priority")) == priority
685+
and _normalize_catalog_install_allowed(
686+
existing.get("install_allowed", False)
687+
) == install_allowed
688+
and str(existing.get("description", "")) == description
689+
):
690+
console.print(
691+
f"[green]✓[/green] Catalog '[bold]{safe_name}[/bold]' is already "
692+
"configured with these settings; nothing to do."
693+
)
694+
return
695+
console.print(
696+
f"[red]Error:[/red] A catalog named '{safe_name}' already exists with "
697+
"different settings."
698+
)
647699
console.print("Use 'specify extension catalog remove' first, or choose a different name.")
648700
raise typer.Exit(1)
649701

‎src/specify_cli/integrations/_query_commands.py‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -543,14 +543,20 @@ def integration_catalog_add(
543543
normalized_url = url.strip()
544544

545545
try:
546-
catalog.add_catalog(normalized_url, name)
546+
status = catalog.add_catalog(normalized_url, name)
547547
except IntegrationCatalogError as exc:
548548
# Covers both URL validation (base class) and config-file validation
549549
# (IntegrationValidationError subclass).
550-
console.print(f"[red]Error:[/red] {exc}")
550+
console.print(f"[red]Error:[/red] {_rich_escape(str(exc))}")
551551
raise typer.Exit(1)
552552

553-
console.print(f"[green]✓[/green] Catalog source added: {normalized_url}")
553+
safe_url = _rich_escape(normalized_url)
554+
if status == "unchanged":
555+
console.print(
556+
f"[green]✓[/green] Catalog source already configured: {safe_url}"
557+
)
558+
else:
559+
console.print(f"[green]✓[/green] Catalog source added: {safe_url}")
554560

555561

556562
@integration_catalog_app.command("remove")

‎src/specify_cli/integrations/catalog.py‎

Lines changed: 20 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -390,14 +390,20 @@ def get_project_catalog_configs(self) -> Optional[List[Dict[str, Any]]]:
390390
for e in entries
391391
]
392392

393-
def add_catalog(self, url: str, name: Optional[str] = None) -> None:
393+
def add_catalog(self, url: str, name: Optional[str] = None) -> str:
394394
"""Add a catalog source to the project-level config file.
395395
396396
The URL is normalized (whitespace stripped) and validated before being
397-
written. Duplicate URLs are rejected, including near-duplicates that
398-
differ only by surrounding whitespace. Priority is derived as
399-
``max(existing) + 1`` so the new entry sorts last in the resolution
400-
order unless the user edits the file manually.
397+
written. Identity for an integration catalog is the (normalized) URL.
398+
Adding a URL that is already configured is idempotent (#4505): a rerun
399+
that requests the same name (or no explicit name) is a successful
400+
no-op, while a rerun that requests a *different* name is rejected as a
401+
conflict rather than silently overwriting the stored entry. Priority is
402+
derived as ``max(existing) + 1`` so a newly added entry sorts last in
403+
the resolution order unless the user edits the file manually.
404+
405+
Returns ``"added"`` when a new entry is written, or ``"unchanged"``
406+
when an equivalent entry already existed.
401407
"""
402408
url = url.strip()
403409
if not url:
@@ -432,6 +438,7 @@ def add_catalog(self, url: str, name: Optional[str] = None) -> None:
432438
# Validate each existing entry before mutating anything. Fail fast so
433439
# we don't silently preserve a corrupt sibling entry or derive a new
434440
# priority from a bogus value.
441+
requested_name = str(name).strip() if name is not None else ""
435442
existing_priorities: List[int] = []
436443
valid_catalog_count = 0
437444
for idx, cat in enumerate(catalogs):
@@ -452,8 +459,14 @@ def add_catalog(self, url: str, name: Optional[str] = None) -> None:
452459
f"Invalid catalog entry at index {idx} in {config_path}: {exc}"
453460
) from exc
454461
if existing_url == url:
462+
# Idempotent add (#4505): same URL already configured.
463+
existing_name = str(cat.get("name", "")).strip()
464+
if not requested_name or requested_name == existing_name:
465+
return "unchanged"
455466
raise IntegrationValidationError(
456-
f"Catalog URL already configured: {url}"
467+
f"Catalog URL already configured with a different name "
468+
f"('{existing_name}'): {url}. Remove it first or pass "
469+
f"--name '{existing_name}'."
457470
)
458471
valid_catalog_count += 1
459472
if "priority" in cat:
@@ -502,6 +515,7 @@ def add_catalog(self, url: str, name: Optional[str] = None) -> None:
502515
sort_keys=False,
503516
allow_unicode=True,
504517
)
518+
return "added"
505519

506520
def remove_catalog(self, index: int) -> str:
507521
"""Remove a catalog source by 0-based index.

0 commit comments

Comments
 (0)