Skip to content

Select native pytest keyword and marker scopes throughout generation and repair #46

Closed

Description

@Jacob-Met

Scope — estate-a3425ebf9874 / github_estate

Add native pytest keyword/marker selection to TestPilot's generation run: run --pytest-k EXPR --pytest-m EXPR and keyword-only TestPilot(..., pytest_k=None, pytest_m=None). The same literal native expressions apply to the existing baseline, generated verification and every repair. Both filters intersect through pytest itself. Selected authored passes alone never qualify a generated patch; generated-case admission remains unchanged.

The optional pytest_selection result context records literal keyword/marker values and appears in JSON, Markdown and HTML with an explicit statement that unselected tests were not verified. Planner/editor/repair prompts receive that context. Default behavior and absent-field compatibility remain unchanged. Saved-test recheck keeps its current full configured-suite behavior; it does not silently inherit run filters.

Actual original evidence

Original main 191cca4e416286a9e1fa4b5d5daf5fc30b936db6, complete tree 2602440d626872f0293755723dc6ca610b1e360b, was captured with all 91 relevant source/test/eval/config Git blobs verified. Native ThinkPad Python 3.14.4, pytest 9.1.1 and coverage 7.16.2 executed six original controls: ordinary generation succeeds, deliberate unselected authored failure causes default failure, both proposed CLI flags refuse with exit 2 and no report, while the existing low-level pytest runner successfully uses each native filter and admits an actual generated pass. Authored input bytes remain unchanged.

The full unchanged baseline records 389 passed, 18 subtests passed and 1 failure: an existing one-second deadline test did not reach its helper-start marker. Its one isolated unchanged control subsequently passed in 3.06 seconds. Both original outcomes remain retained; neither the timeout source nor test is in this scope. Witness receipt SHA256 7bd23268f54dd287ccca324e472f47931685b2251a225a01b56196dfc2398c69; full original log 6fd70efcfbc73ddcd7692b7fed19ce11d943f135cefcbcdc12fd8f837b2d95bd; isolated receipt 4f325ec70c839adcac3a6e2835f77f9fbb83f6ae425a340b36731201ed0d28a5.

Fresh main cec50d8df4499ba509615e179d82ef3861379bd2 adds PR42's raw Git-diff preference isolation. Its complete incoming delta was read and will be preserved, along with PR40's no-coverage behavior.

Narrow source fence and receiving

  • Additive two run options and constructor forwarding in testpilot/__main__.py.
  • Selection configuration/pass-through, result/context and Markdown reporting in testpilot/loop.py.
  • Visible conditional selection context in testpilot/html_report.py.
  • New focused real-pytest tests and docs/PYTEST_SELECTION.md, additive README usage, unique receiving evidence.
  • Preserve sandbox/process custody, generated collector, diff/target selection, recheck, model/provider transport, dependencies, workflows and existing saved-report readers.

Active #35/#43 cancellation, #39 finder, #26/#33 comparison, #44 two-revision regression and #45 diff admission retain their scopes. Shared CLI changes are additive and will compose with current owners. No competing timeout repair will be attempted.

Native owned receiving root: /srv/hamon-estate/receivers/estate-a3425ebf9874-testpilot-pytest-selection; bounded native capture 9d1fcc586481e23416ae6e276c21bb99edafad07 is explicitly not upstream ancestry. Root has independently sealed 13 black-box consumer groups before candidate implementation and separately reproduced the existing default pass/missing-keyword refusal. Actual candidate execution, independent receiving, preserved true-parent source publication and normal hosted CI precede integration.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions