[Extension]: Evaluator Contract #4414

Extension ID

evaluator

Extension Name

Evaluator Contract

Version

1.0.0

Description

Provider-neutral evaluator contract for evidence, provenance, uncertainty, and recovery across Spec-Driven Development phases.

Author

ElectroHire

Repository URL

https://github.com/electrohire/spec-kit-evaluator

Download URL

https://github.com/electrohire/spec-kit-evaluator/archive/refs/tags/v1.0.0.zip

License

MIT

Homepage (optional)

https://github.com/electrohire/spec-kit-evaluator

Documentation URL (optional)

https://github.com/electrohire/spec-kit-evaluator/blob/main/README.md

Changelog URL (optional)

https://github.com/electrohire/spec-kit-evaluator/blob/main/CHANGELOG.md

Required Spec Kit Version

=1.0.0

Required Tools (optional)

None required

Number of Commands

4

Number of Hooks (optional)

4

Tags

evaluator, evidence, provenance, quality, governance

Key Features

• Versioned JSON Schema for evaluator results with explicit outcomes, finding kinds, evidence kinds, and uncertainty
• Deterministic strict, majority, and optimistic composition with contradiction preservation
• Advisory model-tier routing for each Spec-Driven Development phase
• Terminal, Markdown, JSON, CI-annotation, and gate report formats
• Python, Bash, and PowerShell parity scripts
• Four optional lifecycle hooks: after_specify, after_plan, after_tasks, and after_implement

Testing Checklist

• Extension installs successfully via download URL
• All commands execute without errors
• Documentation is complete and accurate
• No security vulnerabilities identified
• Tested on at least one real project

Submission Requirements

• Valid extension.yml manifest included
• README.md with installation and usage instructions
• LICENSE file included
• GitHub release created with version tag
• All command files exist and are properly formatted
• Extension ID follows naming conventions (lowercase-with-hyphens)

Testing Details

Validated results:

• Extension test suite: 70 passed, 4 skipped
• Spec Kit extension framework suite: 528 passed
• Broader Spec Kit suite: 4,296 passed, 114 skipped
• Sample-project lifecycle: initialization → evaluator installation → verification of all four installed commands → clean removal
• Public v1.0.0 release archive and required repository files verified

Example Usage

specify extension add evaluator --from https://github.com/electrohire/spec-kit-evaluator/archive/refs/tags/v1.0.0.zip


Then run an evaluator after planning:


/speckit.evaluator.run phase=after_plan artifacts=spec.md,plan.md

Proposed Catalog Entry

{
  "evaluator": {
    "name": "Evaluator Contract",
    "id": "evaluator",
    "description": "Provider-neutral evaluator contract for evidence, provenance, uncertainty, and recovery across Spec-Driven Development phases.",
    "author": "ElectroHire",
    "version": "1.0.0",
    "download_url": "https://github.com/electrohire/spec-kit-evaluator/archive/refs/tags/v1.0.0.zip",
    "repository": "https://github.com/electrohire/spec-kit-evaluator",
    "homepage": "https://github.com/electrohire/spec-kit-evaluator",
    "documentation": "https://github.com/electrohire/spec-kit-evaluator/blob/main/README.md",
    "changelog": "https://github.com/electrohire/spec-kit-evaluator/blob/main/CHANGELOG.md",
    "license": "MIT",
    "category": "process",
    "effect": "read-write",
    "requires": {
      "speckit_version": ">=1.0.0"
    },
    "provides": {
      "commands": 4,
      "hooks": 4
    },
    "tags": [
      "evaluator",
      "evidence",
      "provenance",
      "quality",
      "governance"
    ],
    "verified": false,
    "downloads": 0,
    "stars": 0,
    "created_at": "2026-09-03T00:00:00Z",
    "updated_at": "2026-09-03T00:00:00Z"
  }
}

Additional Context

This is a corrected resubmission of #4403 following its automated validation feedback. It also follows the maintainer request on #4390 to host the evaluator as a standalone extension. The original feature discussion is #4290.

AI-assistance disclosure: This resubmission was prepared with ChatGPT assistance to align it with the current Spec Kit Extension Submission template and automated validator. The extension implementation was generated with GitHub Copilot (DeepSeek-V4-Pro), then reviewed and tested by me. I understand the implementation and will maintain it.