Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,10 @@ env:
CI: "1"

jobs:
windows:
name: Windows compatibility
uses: ./.github/workflows/windows.yml

lint:
name: Lint (ruff)
runs-on: ubuntu-latest
Expand Down Expand Up @@ -160,7 +164,7 @@ jobs:
build:
name: Build sdist + wheel
runs-on: ubuntu-latest
needs: [lint, test, owned-contracts, tasksmith-runtimes]
needs: [lint, test, owned-contracts, tasksmith-runtimes, windows]
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7

Expand Down
8 changes: 7 additions & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,12 @@ concurrency:
cancel-in-progress: false

jobs:
windows:
name: Windows compatibility
uses: ./.github/workflows/windows.yml
with:
ref: ${{ github.event.release.tag_name || inputs.tag }}

# Sanity gate — run the full test suite on the release tag before we publish.
test:
name: Tests on release tag (py${{ matrix.python-version }})
Expand Down Expand Up @@ -75,7 +81,7 @@ jobs:
build:
name: Build sdist + wheel
runs-on: ubuntu-latest
needs: [test, tasksmith-runtimes]
needs: [test, tasksmith-runtimes, windows]
outputs:
version: ${{ steps.read-version.outputs.version }}
steps:
Expand Down
44 changes: 44 additions & 0 deletions .github/workflows/windows.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name: Windows compatibility

on:
workflow_call:
inputs:
ref:
description: Commit or release tag to validate
required: false
type: string

permissions:
contents: read

jobs:
wheel:
name: Windows wheel (py${{ matrix.python-version }})
runs-on: windows-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.12", "3.13", "3.14"]
env:
PYTHON_DOTENV_DISABLED: "1"
NO_COLOR: "1"
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
ref: ${{ inputs.ref }}
- uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
with:
python-version: ${{ matrix.python-version }}
enable-cache: false
- run: uv build
- name: Install base wheel in isolation
run: |
uv venv "$env:RUNNER_TEMP/cli-check"
$wheel = (Get-ChildItem dist/*.whl).FullName
uv pip install --python "$env:RUNNER_TEMP/cli-check/Scripts/python.exe" $wheel
- name: Check console entrypoint, discovery and native task validation
run: '& "$env:RUNNER_TEMP/cli-check/Scripts/python.exe" tests/check_cli_install.py'
- name: Check real Windows process locks
run: |
uv pip install --python "$env:RUNNER_TEMP/cli-check/Scripts/python.exe" pytest
& "$env:RUNNER_TEMP/cli-check/Scripts/python.exe" -m pytest -q tests/test_locking.py tests/test_posix_only_imports.py
1 change: 1 addition & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ Closes #N

- **lint** — `uv run ruff check .` + `uv run ruff format --check .`
- **test** — `uv run pytest -q` against Python 3.12, 3.13, 3.14 (matrix)
- **Windows** — base-only wheel CLI smoke checks and real process-lock contention on Python 3.12, 3.13, 3.14; the same gate runs before publication. Full controller portability remains separate work.
- **build** — `uv build` produces sdist + wheel, smoke-installs the wheel, checks `repo2rlenv --version`

A green CI is the floor for merge — green plus at least one approving review is the ceiling.
Expand Down
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,10 @@ and repair emitted tasks; exporting a task alone does not establish its quality.
Requires **Python 3.12+** and Git. This example generates PR-diff tasks without
building a container:

Windows CI covers CLI startup, recipe discovery, native task emission and static
validation. Use Linux, macOS or WSL for Tasksmith, research-recipe generation and
the quality controller; their full native Windows execution is not yet supported.

```bash
pip install repo2rlenv

Expand Down
6 changes: 6 additions & 0 deletions docs/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,12 @@ hf auth login

## Install

Requires Python 3.12+. Windows CI checks the installed CLI, recipe discovery,
native task emission and static validation. Run Tasksmith, research-recipe
generation and the quality controller on Linux, macOS or WSL: those controllers
still depend on POSIX artifact permissions and process cleanup. Remote sandboxes
run Linux; choosing a cloud provider does not remove these host requirements.

```bash
pip install repo2rlenv # from PyPI
# or:
Expand Down
38 changes: 38 additions & 0 deletions src/repo2rlenv/locking.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
"""Process locks for controller receipts and shared worker checkouts."""

from __future__ import annotations

import errno
import sys
import time
from typing import IO


def lock_file(handle: IO, *, blocking: bool = False) -> None:
"""Exclusively lock an open file until it closes, without deleting its path.

POSIX retains flock semantics, including interoperability with older
controllers. Windows locks byte zero; the region may extend beyond EOF,
so an empty lock file needs no write. All callers must keep the same path
and close their handle on success, failure or cancellation.
"""
if sys.platform != "win32":
import fcntl

fcntl.flock(handle, fcntl.LOCK_EX | (0 if blocking else fcntl.LOCK_NB))
return

import msvcrt

handle.seek(0)
while True:
try:
msvcrt.locking(handle.fileno(), msvcrt.LK_NBLCK, 1)
return
except OSError as error:
if error.errno not in {errno.EACCES, errno.EAGAIN, errno.EDEADLK}:
raise
if not blocking:
raise BlockingIOError(errno.EAGAIN, "Another process owns this lock") from error
# LK_LOCK gives up after ten retries; match flock's indefinite wait.
time.sleep(0.1)
4 changes: 2 additions & 2 deletions src/repo2rlenv/pipelines/recipes/history/worker.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
from __future__ import annotations

import argparse
import fcntl
import hashlib
import io
import json
Expand All @@ -16,6 +15,7 @@

from repo2rlenv.execution.lifecycle import save_record
from repo2rlenv.execution.python_repository import bootstrap_snapshot, test_image
from repo2rlenv.locking import lock_file
from repo2rlenv.pipelines.recipes.history.selection import check_entities, within
from repo2rlenv.pipelines.recipes.history.test_suite import stage_tests
from repo2rlenv.quality.python_evidence import test_excerpts
Expand Down Expand Up @@ -131,7 +131,7 @@ def prepare(config: dict, destination: Path) -> dict:
root = Path("/work/history") / hashlib.sha256(repo.url.encode()).hexdigest()[:16]
root.parent.mkdir(parents=True, exist_ok=True)
with root.with_suffix(".lock").open("a") as lock:
fcntl.flock(lock, fcntl.LOCK_EX)
lock_file(lock, blocking=True)
if not root.exists():
temporary = root.with_name(root.name + "-" + uuid.uuid4().hex)
try:
Expand Down
4 changes: 2 additions & 2 deletions src/repo2rlenv/quality/loop/runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@

from __future__ import annotations

import fcntl
import hashlib
import json
import re
Expand All @@ -15,6 +14,7 @@
from repo2rlenv.campaigns.budget import BudgetExceeded, BudgetLedger
from repo2rlenv.campaigns.events import EventJournal, ProgressEvent
from repo2rlenv.execution.lifecycle import save_record
from repo2rlenv.locking import lock_file
from repo2rlenv.quality.loop.artifacts import (
EXPECTED_PASSES_CONTRACT,
apply_repair,
Expand Down Expand Up @@ -641,7 +641,7 @@ def run(
self.directory.mkdir(parents=True, exist_ok=True)
with (self.directory / ".lock").open("a") as lock:
try:
fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
lock_file(lock)
except BlockingIOError as exc:
raise RuntimeError("Another controller owns this quality run") from exc
try:
Expand Down
4 changes: 2 additions & 2 deletions src/repo2rlenv/tasksmith/batch.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
from __future__ import annotations

import argparse
import fcntl
import hashlib
import json
import os
Expand All @@ -23,6 +22,7 @@
from repo2rlenv.campaigns.budget import BudgetLedger
from repo2rlenv.execution.artifacts import check_runtime_wheel
from repo2rlenv.execution.lifecycle import save_record
from repo2rlenv.locking import lock_file
from repo2rlenv.quality.loop.artifacts import digest
from repo2rlenv.quality.loop.client import RunBudget
from repo2rlenv.quality.loop.models import LoopResult, ProbeManifest
Expand Down Expand Up @@ -370,7 +370,7 @@ def run_batch(
directory, campaign, wheel = directory.resolve(), campaign.resolve(), wheel.resolve()
directory.mkdir(parents=True, exist_ok=True)
with (directory / ".lock").open("a") as lock:
fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
lock_file(lock)
prior = [verified_result(path) for path in plan.prior_verified]
if len({item["url"] for item in prior}) != len(prior):
raise ValueError("Prior verified results contain duplicate PRs")
Expand Down
4 changes: 2 additions & 2 deletions src/repo2rlenv/tasksmith/runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
from __future__ import annotations

import asyncio
import fcntl
import hashlib
import json
import shlex
Expand All @@ -27,6 +26,7 @@
save_record,
stop_worker,
)
from repo2rlenv.locking import lock_file
from repo2rlenv.quality.loop.artifacts import task_identity
from repo2rlenv.quality.loop.client import RunBudget
from repo2rlenv.quality.loop.models import ProbeManifest
Expand Down Expand Up @@ -765,7 +765,7 @@ def run(
raise ValueError("stop-after must be within the frozen panel size")
self.directory.mkdir(parents=True, exist_ok=True)
with (self.directory / ".lock").open("a") as lock:
fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
lock_file(lock)
return self._run(panel, limit, generation_run, reuse_evidence, source_records)

def _run(self, panel, limit, generation_run, reuse_evidence=False, source_records=None):
Expand Down
75 changes: 75 additions & 0 deletions tests/check_cli_install.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
"""Smoke-test an installed CLI from outside the checkout, without cloud calls."""

from __future__ import annotations

import json
import os
import subprocess
import sys
import sysconfig
from importlib.metadata import version
from pathlib import Path
from tempfile import TemporaryDirectory

from repo2rlenv.emitter.harbor import HarborTask, write_harbor_task
from repo2rlenv.ui import console


def check() -> dict:
executable = Path(sysconfig.get_path("scripts")) / (
"repo2rlenv.exe" if sys.platform == "win32" else "repo2rlenv"
)
with TemporaryDirectory(prefix="repo2rlenv CLI ") as directory:
root = Path(directory)

def cli(*args: str, expected: int = 0) -> str:
result = subprocess.run(
[str(executable), *args],
cwd=root,
env={**os.environ, "PYTHON_DOTENV_DISABLED": "1", "PYTHONUTF8": "1"},
capture_output=True,
encoding="utf-8",
timeout=60,
check=False,
)
assert result.returncode == expected, result.stderr + result.stdout
return result.stdout

assert version("repo2rlenv") in cli("--version")
assert "generate" in cli("--help")
for command in (
"generate",
"validate",
"push",
"pull",
"tasksmith",
"quality",
"pipelines",
):
assert "usage:" in cli(command, "--help")
listing = json.loads(cli("pipelines", "list", "--json"))
recipes = [item for item in listing["recipes"] if item["implemented"]]
for recipe in recipes:
detail = json.loads(
cli("pipelines", "describe", recipe["pipeline"], "--recipe", recipe["id"], "--json")
)
assert detail["id"] == recipe["id"]
for command in ("tasksmith", "quality"):
error = json.loads(cli(command, "show", "missing.json", "--json", expected=2))
assert set(error) == {"error", "message"}
task = HarborTask(
name="unicode-task",
org="smoke",
description="Handle café input",
instruction="# Task\n\nPreserve café and 日本語 in the output.\n",
oracle_diff="--- a/x.py\n+++ b/x.py\n@@ -1 +1 @@\n-1\n+2\n",
repo2env={"pipeline": "pr_diff", "pipeline_version": "0.1.0", "repo": "smoke/demo"},
)
output = write_harbor_task(task, root / "task output")
assert (output / "instruction.md").read_text(encoding="utf-8") == task.instruction
cli("validate", str(root / "task output"), "--deep")
return {"status": "passed", "version": version("repo2rlenv"), "recipes": len(recipes)}


if __name__ == "__main__":
console.json(check())
Loading