Repository navigation
Expand file tree
/
Copy pathaction.yml
More file actions
88 lines (88 loc) · 5.13 KB
/
Copy pathaction.yml
File metadata and controls
88 lines (88 loc) · 5.13 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
name: Evals
description: Acquire Vally and run native lint or evaluations with structured GitHub reporting.
author: GitHub
inputs:
command:
description: Default all validates, lints, runs the selected evals, and updates one PR comment. Use lint, run, or report for separate jobs. Plan is unsupported.
default: all
github-token:
description: Automatic job token for Copilot and PR commenting in all mode, or artifact discovery and commenting in report mode. Discovery requires actions read. The all job token has combined permissions; use separate commands for isolation.
default: ${{ github.token }}
working-directory:
description: Project directory relative to checkout. Required for all/lint/run; omitted for report.
suite:
description: Native named suite for run. Cannot combine with eval-paths or tags. Ignored for lint.
eval-paths:
description: Newline-separated literal eval file paths for run. No shell glob expansion. Ignored for lint.
tags:
description: Newline-separated native tag selectors for run. Ignored for lint.
vally-version:
description: Default latest resolves the npm latest stable release once per job. Use an exact stable version to pin Vally. Native command and evidence contracts are verified at runtime.
default: latest
model:
description: Explicit native model override; omitted preserves configuration.
runs:
description: Positive integer native repetition override.
workers:
description: Positive integer native concurrency override.
timeout:
description: Positive integer native timeout override in seconds.
max-retries:
description: Nonnegative integer native retry override.
credential-env:
description: Advanced run-only credential environment names for BYOK or supported secrets. Use copilot-token for the workflow token. Never reuse report-token.
copilot-token:
description: Explicit Copilot token for run, normally github.token with copilot-requests write. Mapped to COPILOT_GITHUB_TOKEN; cannot combine with that credential-env name or report-token.
report-token:
description: Optional comment token override for all/report; defaults to github-token. Never pass it to explicit run jobs.
report-id:
description: Shared identity across lint, run, and report; defaults to evals. Report discovers matching current-run/attempt artifacts. Use a distinct literal identity for each independent report and a non-matrix report job.
default: evals
source-artifact-id:
description: Optional primary artifact ID override for report. Pair with source-artifact-digest to bypass discovery and download exactly this artifact.
source-artifact-digest:
description: SHA-256 upload digest paired with source-artifact-id. Omit both for automatic discovery; report command only.
lint-artifact-id:
description: Optional lint artifact override, paired with lint-artifact-digest. Otherwise discovered alongside the run. Supports the shared report ID or legacy prefix plus -lint; report command only.
lint-artifact-digest:
description: SHA-256 digest for the optional companion lint artifact; report command only.
baseline-report-id:
description: Optional baseline report identity for report mode. Discovers the matching current-run/attempt run artifact unless baseline-artifact-id is set.
baseline-artifact-id:
description: Optional baseline run artifact ID override for report mode, paired with baseline-artifact-digest.
baseline-artifact-digest:
description: SHA-256 digest for the optional baseline run artifact; report command only.
upload-transcripts:
description: Opt in to native JSONL and stdout/stderr logs, which may contain sensitive content.
default: 'false'
artifact-retention-days:
description: Structured artifact retention in days, subject to repository policy.
default: '7'
outputs:
outcome:
description: validated, passed, failed, error, cancelled, or not-run. Lint validation is not an evaluation pass.
plan-completeness:
description: Complete native terminal evidence, partial evidence, or unknown. No pre-execution plan is claimed.
executed:
description: Number of executed native trial records, excluding skipped records; absent when unknown.
passed:
description: Number of executed native eval/variant/model groups with a passing native verdict; absent when unknown.
failed:
description: Number of executed native eval/variant/model groups with a failing native verdict; absent when unknown.
vally-version:
description: Concrete acquired engine version; absent until acquisition.
result-path:
description: Local minimal structured report path.
artifact-id:
description: Uploaded structured report artifact identifier.
artifact-digest:
description: SHA-256 digest of the uploaded structured report artifact.
lint-artifact-id:
description: Companion lint artifact identifier in all mode after successful lint.
lint-artifact-digest:
description: SHA-256 digest of the companion lint artifact in all mode.
native-diagnostics-path:
description: Local native output directory. Raw logs are uploaded only when upload-transcripts is true.
runs:
using: node24
main: dist/index.cjs