Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 75 additions & 0 deletions .github/ISSUE_TEMPLATE/bug.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
name: Bug report
description: Report reproducible incorrect behavior in the kernel or a repository integration.
title: "[Bug]: "
labels:
- bug
body:
- type: markdown
attributes:
value: |
Thanks for helping improve evalopt. Do not include credentials, private data, or undisclosed security issues. Report vulnerabilities through the private security advisory link below.
- type: input
id: version
attributes:
label: evalopt version or commit
description: For example, `0.1.0` or a full commit SHA.
placeholder: 0.1.0
validations:
required: true
- type: dropdown
id: python
attributes:
label: Python version
options:
- "3.10"
- "3.11"
- "3.12"
- "3.13"
- "3.14"
- Not applicable
validations:
required: true
- type: input
id: platform
attributes:
label: Operating system and architecture
placeholder: Ubuntu 24.04, x86_64
validations:
required: true
- type: textarea
id: reproduce
attributes:
label: Minimal reproduction
description: Include the smallest policy, input, and command or code that reproduces the problem.
render: python
validations:
required: true
- type: textarea
id: observed
attributes:
label: Observed behavior
description: Include the decision status, reason codes, traceback, or sanitized output.
validations:
required: true
- type: textarea
id: expected
attributes:
label: Expected behavior
validations:
required: true
- type: textarea
id: context
attributes:
label: Additional context
description: Explain whether this affects the stable kernel, a deprecated compatibility surface, or a host integration.
- type: checkboxes
id: checks
attributes:
label: Checklist
options:
- label: I searched existing issues and discussions.
required: true
- label: This report contains no secrets or private data.
required: true
- label: This is not an undisclosed security vulnerability.
required: true
11 changes: 11 additions & 0 deletions .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
blank_issues_enabled: false
contact_links:
- name: Questions and integration help
url: https://github.com/ajaysurya1221/evalopt-graph/discussions
about: Ask usage questions and share integration ideas in Discussions.
- name: Security vulnerability
url: https://github.com/ajaysurya1221/evalopt-graph/security/advisories/new
about: Report vulnerabilities privately. Do not disclose them in a public issue.
- name: Documentation
url: https://github.com/ajaysurya1221/evalopt-graph#readme
about: Read the quickstart, boundaries, evidence status, and project documentation.
59 changes: 59 additions & 0 deletions .github/ISSUE_TEMPLATE/feature.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
name: Design proposal
description: Propose a focused kernel, adapter, documentation, or conformance improvement.
title: "[Proposal]: "
labels:
- enhancement
body:
- type: markdown
attributes:
value: |
evalopt deliberately keeps orchestration outside its stable kernel. Explain the trust-boundary impact of the proposal and the smallest public surface that solves the problem.
- type: textarea
id: problem
attributes:
label: Problem
description: What concrete user or integration problem needs to be solved?
validations:
required: true
- type: dropdown
id: boundary
attributes:
label: Primary area
options:
- Stable governance kernel
- Host or verifier adapter
- Deprecated compatibility surface
- Documentation or examples
- Conformance or evaluation
validations:
required: true
- type: textarea
id: behavior
attributes:
label: Proposed behavior
description: Describe inputs, outputs, invariants, and failure behavior without assuming a specific agent runtime.
validations:
required: true
- type: textarea
id: alternatives
attributes:
label: Alternatives considered
description: Could the host own this behavior without changing the kernel?
validations:
required: true
- type: textarea
id: compatibility
attributes:
label: Compatibility and evidence impact
description: Note effects on the ten-symbol API, serialized records, replay, reason codes, or evidence claims.
validations:
required: true
- type: checkboxes
id: checks
attributes:
label: Checklist
options:
- label: I searched existing issues and discussions.
required: true
- label: I have not presented authored or generated cases as benchmark or SOTA evidence.
required: true
23 changes: 23 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
version: 2
updates:
- package-ecosystem: "pip"
directory: "/"
schedule:
interval: "weekly"
day: "monday"
time: "04:00"
timezone: "Etc/UTC"
open-pull-requests-limit: 5
commit-message:
prefix: "deps"

- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
day: "monday"
time: "04:30"
timezone: "Etc/UTC"
open-pull-requests-limit: 5
commit-message:
prefix: "ci"
29 changes: 29 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
## Summary

<!-- What changes, and what concrete problem does it solve? -->

## Boundary and compatibility

<!-- Does this belong to the stable kernel, a host adapter, or a deprecated compatibility surface? -->

- Stable ten-symbol API impact: <!-- none, additive, or breaking -->
- Serialized record / replay impact: <!-- none or explain -->
- Trust-boundary impact: <!-- none or explain -->

## Evidence and testing

<!-- List the checks run and the evidence level supported. Authored cases are conformance evidence, not benchmark evidence. -->

```text
commands and results
```

## Checklist

- [ ] The change is focused and includes regression tests where behavior changed.
- [ ] Kernel and host responsibilities remain explicit; no second acceptance-authority path was added.
- [ ] Stable API, reason-code, serialization, and migration effects are documented.
- [ ] Evidence claims match the demonstrated evidence level and do not imply benchmark or SOTA proof.
- [ ] No tests or security checks were weakened to make the change pass.
- [ ] No secrets, private data, generated run artifacts, caches, or local environment files are included.
- [ ] User-facing changes are reflected in the README, changelog, or release notes as appropriate.
195 changes: 195 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,195 @@
name: CI

on:
push:
branches:
- main
- "release/**"
pull_request:
branches:
- main
workflow_call:
workflow_dispatch:

permissions:
contents: read

concurrency:
group: ci-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

env:
GITLEAKS_LINUX_X64_SHA256: "551f6fc83ea457d62a0d98237cbad105af8d557003051f41f3e7ca7b3f2470eb"
GITLEAKS_VERSION: "8.30.1"
UV_VERSION: "0.11.29"

jobs:
linux-tests:
name: Ubuntu / Python ${{ matrix.python-version }}
runs-on: ubuntu-latest
timeout-minutes: 20
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
steps:
- name: Check out source
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: ${{ matrix.python-version }}
- name: Install uv
run: python -m pip install --disable-pip-version-check "uv==${UV_VERSION}"
- name: Sync locked development environment
run: uv sync --frozen --extra dev
- name: Run full test suite
run: uv run --frozen --extra dev python -m pytest -p no:cacheprovider

macos-tests:
name: macOS / Python 3.14
runs-on: macos-latest
timeout-minutes: 25
steps:
- name: Check out source
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.14"
- name: Install uv
run: python -m pip install --disable-pip-version-check "uv==${UV_VERSION}"
- name: Sync locked development environment
run: uv sync --frozen --extra dev
- name: Run full test suite
run: uv run --frozen --extra dev python -m pytest -p no:cacheprovider

quality-build:
name: Quality, demos, smoke, and artifacts
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Check out source
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.14"
- name: Install uv
run: python -m pip install --disable-pip-version-check "uv==${UV_VERSION}"
- name: Verify lock and sync environment
run: |
uv lock --check
uv sync --frozen --extra dev
- name: Verify action pins and test floor
run: |
uv run python scripts/verify_workflow_pins.py
uv run --frozen --extra dev python scripts/verify_test_count.py --minimum 408
- name: Compile and lint
run: |
uv run --frozen --extra dev python -m compileall -q src tests scripts
uv run --frozen --extra dev ruff check src tests scripts
uv run --frozen --extra dev ruff format --check src tests scripts
- name: Run deterministic demos
run: |
uv run --frozen --extra dev python scripts/edit_demo.py
uv run --frozen --extra dev python scripts/epistemic_demo.py
uv run --frozen --extra dev python scripts/unattended_demo.py
uv run --frozen --extra dev python scripts/provenance_benchmark.py --out "$RUNNER_TEMP/conformance.json"
- name: Run offline smoke test
run: bash scripts/smoke_test.sh
- name: Build distributions once
run: uv build --out-dir dist
- name: Validate distributions
run: |
uvx --from twine==6.2.0 twine check dist/*
python scripts/verify_artifacts.py dist --version 0.1.0
- name: Upload CI distributions
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: ci-distributions
path: dist/
if-no-files-found: error
include-hidden-files: false
retention-days: 7

windows-wheel:
name: Windows / clean wheel / Python 3.14
needs: quality-build
runs-on: windows-latest
timeout-minutes: 15
steps:
- name: Check out verification script
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.14"
- name: Download wheel
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: ci-distributions
path: dist
- name: Install wheel without dependencies
shell: pwsh
run: |
$wheel = (Get-ChildItem -Path dist -Filter *.whl -File -Recurse -ErrorAction Stop)
if ($wheel.Count -ne 1) { throw "Expected exactly one wheel, found $($wheel.Count)" }
python -m pip install --disable-pip-version-check --no-deps $wheel[0].FullName
- name: Verify exact API, serialization, and replay
run: python scripts/verify_installed_release.py --version 0.1.0

gitleaks:
name: Gitleaks / full history
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Check out full history
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
fetch-depth: 0
persist-credentials: false
- name: Install checksum-pinned Gitleaks
run: |
archive="$RUNNER_TEMP/gitleaks.tar.gz"
bin_dir="$RUNNER_TEMP/gitleaks-bin"
curl --fail --silent --show-error --location \
"https://github.com/gitleaks/gitleaks/releases/download/v${GITLEAKS_VERSION}/gitleaks_${GITLEAKS_VERSION}_linux_x64.tar.gz" \
--output "$archive"
echo "${GITLEAKS_LINUX_X64_SHA256} ${archive}" | sha256sum --check --strict -
mkdir -p "$bin_dir"
tar -xzf "$archive" -C "$bin_dir" gitleaks
echo "$bin_dir" >> "$GITHUB_PATH"
- name: Scan every reachable commit for secrets
run: gitleaks git --redact --config .gitleaks.toml --log-opts="--all" .

required:
name: CI / required
if: always()
needs:
- linux-tests
- macos-tests
- quality-build
- windows-wheel
- gitleaks
runs-on: ubuntu-latest
steps:
- name: Require every CI lane
env:
LINUX: ${{ needs.linux-tests.result }}
MACOS: ${{ needs.macos-tests.result }}
QUALITY: ${{ needs.quality-build.result }}
WINDOWS: ${{ needs.windows-wheel.result }}
GITLEAKS: ${{ needs.gitleaks.result }}
run: |
for result in "$LINUX" "$MACOS" "$QUALITY" "$WINDOWS" "$GITLEAKS"; do
test "$result" = "success"
done
Loading