Treat expression-backed engine.model as omitted when empty by Copilot · Pull Request #46777 · github/gh-aw

Copilot · 2026-07-20T09:12:39Z

When engine.model (or top-level model) is a GitHub Actions expression like ${{ vars.HOLISTIC_REVIEW_MODEL }} and the variable is unset, the expression resolves to "" at runtime. Previously that empty string was propagated directly to the engine, making the variable mandatory rather than optional.

Changes

compilerenv/manager.go — new BuildModelOverrideExpressionWithUserOverride(userExprInner, primaryVar, enterpriseDefaultVar, builtinFallback) that prepends a user expression to the standard org/enterprise/built-in fallback chain
expression_patterns.go — new extractExpressionInner(s) helper with an explicit length guard that strips ${{ / }} delimiters safely
copilot_engine_execution.go, claude_engine.go, codex_engine.go — when model is a pure expression (isExpression() == true), wrap it instead of passing it through directly:

# Before — empty when vars.HOLISTIC_REVIEW_MODEL is unset
COPILOT_MODEL: ${{ vars.HOLISTIC_REVIEW_MODEL }}

# After — falls back through normal resolution chain
COPILOT_MODEL: ${{ vars.HOLISTIC_REVIEW_MODEL || vars.GH_AW_MODEL_AGENT_COPILOT || vars.GH_AW_DEFAULT_MODEL_COPILOT || 'claude-sonnet-4.6' }}

Only pure expressions are affected (${{ ... }}). Literal model values (e.g. gpt-4) and partial expressions (e.g. provider/${{ vars.MODEL }}) retain existing behavior.

Four internal workflows using ${{ needs.activation.outputs.model_size }} as their model were recompiled to pick up the updated output.

Fixes Treat an empty expression-based engine.model as omitted #46499

run: https://github.com/github/gh-aw/actions/runs/29735453773

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

pi.dev

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "pi.dev"

See Network Configuration for more information.

Generated by 👨‍🍳 PR Sous Chef · 7.59 AIC · ⌖ 5.86 AIC · ⊞ 5.4K · ◷
_{Comment /souschef to run again}

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

pi.dev

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "pi.dev"

See Network Configuration for more information.

Generated by 👨‍🍳 PR Sous Chef · 7.16 AIC · ⌖ 5.51 AIC · ⊞ 5.4K · ◷
_{Comment /souschef to run again}

When engine.model or model is a pure GitHub Actions expression (e.g., ${{ vars.MY_MODEL }}), it may resolve to an empty string at runtime if the variable is unset. Previously this empty value was propagated directly to the engine, causing failures. This change wraps pure expression-backed model values with the normal fallback chain for Copilot, Claude, and Codex engines, so that an empty resolution falls back to org/enterprise variables and the built-in default. For Copilot: ${{ vars.MY_MODEL || vars.GH_AW_MODEL_AGENT_COPILOT || vars.GH_AW_DEFAULT_MODEL_COPILOT || 'claude-sonnet-4.6' }} For Claude: ${{ inputs.model || vars.GH_AW_MODEL_AGENT_CLAUDE || vars.GH_AW_DEFAULT_MODEL_CLAUDE || '' }} For Codex: ${{ inputs.model || vars.GH_AW_MODEL_AGENT_CODEX || vars.GH_AW_DEFAULT_MODEL_CODEX || 'gpt-5.4' }} Changes: - pkg/workflow/compilerenv/manager.go: add BuildModelOverrideExpressionWithUserOverride - pkg/workflow/expression_patterns.go: add extractExpressionInner helper - pkg/workflow/copilot_engine_execution.go: wrap expressions in addCopilotModelEnv - pkg/workflow/claude_engine.go: wrap expressions in model env assignment - pkg/workflow/codex_engine.go: wrap expressions in model env assignment - Updated/added tests - Recompiled 4 internal workflows using expression-backed models Closes #46499 Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot

Pull request overview

Fixes #46499 by adding standard model fallbacks when a pure model expression resolves empty.

Changes:

Adds expression extraction and fallback-chain construction.
Applies fallback resolution to Copilot, Claude, and Codex.
Updates tests and generated workflows.

Composite expressions are currently misclassified as pure expressions and rewritten unexpectedly.

Show a summary per file

File	Description
`pkg/workflow/model_env_vars_test.go`	Updates model fallback expectations.
`pkg/workflow/expression_patterns.go`	Adds expression-content extraction.
`pkg/workflow/copilot_engine_execution.go`	Adds Copilot expression fallbacks.
`pkg/workflow/compilerenv/manager.go`	Builds combined fallback expressions.
`pkg/workflow/compilerenv/manager_test.go`	Tests fallback construction.
`pkg/workflow/codex_engine.go`	Adds Codex expression fallbacks.
`pkg/workflow/claude_engine.go`	Adds Claude expression fallbacks.
`.github/workflows/test-quality-sentinel.lock.yml`	Regenerates Copilot model configuration.
`.github/workflows/daily-doc-healer.lock.yml`	Regenerates Claude model configuration.
`.github/workflows/daily-caveman-optimizer.lock.yml`	Regenerates Claude model configuration.
`.github/workflows/daily-cache-strategy-analyzer.lock.yml`	Regenerates Codex model configuration.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 11/11 changed files
Comments generated: 4
Review effort level: Medium

Copilot · 2026-07-20T16:53:56Z

+		if isExpression(workflowData.Model) {
+			innerExpr := extractExpressionInner(workflowData.Model)
+			expr := compilerenv.BuildModelOverrideExpressionWithUserOverride(innerExpr, modelEnvVar, compilerenv.DefaultModelCopilot, constants.CopilotBYOKDefaultModel)


Addressed by the later fallback-env implementation and now covered explicitly in a915121: composite values are kept verbatim in COPILOT_MODEL, and the runtime uses separate GH_AW_MODEL_FALLBACK when the resolved model is empty.

Copilot · 2026-07-20T16:53:54Z

+		if isExpression(workflowData.Model) {
+			innerExpr := extractExpressionInner(workflowData.Model)
+			isDetectionJob := workflowData.SafeOutputs == nil


Addressed in a915121: Claude keeps composite values like ${{ inputs.provider }}/${{ inputs.model }} unchanged, and this pass adds the matching regression test in pkg/workflow/model_env_vars_test.go.

Copilot · 2026-07-20T16:53:54Z

+		if isExpression(workflowData.Model) {
+			innerExpr := extractExpressionInner(workflowData.Model)
+			expr := compilerenv.BuildModelOverrideExpressionWithUserOverride(innerExpr, modelEnvVar, compilerenv.DefaultModelCodex, constants.CodexDefaultModel)


Addressed by the current fallback-env path and confirmed in a915121: composite interpolation stays in the original model env value, while GH_AW_MODEL_FALLBACK is only used when the runtime sees an empty resolved model.

Copilot · 2026-07-20T16:53:54Z

+// fallback chain. Use this when engine.model or model is a GitHub Actions expression that
+// may resolve to an empty string (e.g. ${{ vars.MY_MODEL }} when the variable is unset).
+// userExprInner is the raw content between "${{" and "}}" of the user expression.
+// When builtinFallback is empty the expression ends with || ”.


Already satisfied on the current branch and still true in a915121: the empty fallback is '', and pkg/workflow/compilerenv/manager_test.go asserts that exact string.

pelikhan · 2026-07-20T09:53:12Z

@copilot add a fallback env var with the expression that can be used in JavaScript if the model expression resolves to empty. That way it handles any syntax

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot · 2026-07-20T10:32:09Z

@copilot add a fallback env var with the expression that can be used in JavaScript if the model expression resolves to empty. That way it handles any syntax

Addressed in 0ce2115 (with follow-up fix in f2740c3). The runtime now keeps the configured model expression intact and adds GH_AW_MODEL_FALLBACK, which the shipped JS harness/driver paths use when the resolved model is empty.

gh-aw-bot · 2026-07-20T10:41:11Z

[pr-sous-chef] @copilot please run the pr-finisher skill, address the latest maintainer feedback, and rerun checks once the branch is ready.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

pi.dev

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "pi.dev"

See Network Configuration for more information.

Generated by 👨‍🍳 PR Sous Chef · 7.59 AIC · ⌖ 5.86 AIC · ⊞ 5.4K · ◷
_{Comment /souschef to run again}

Copilot · 2026-07-20T10:56:29Z

[pr-sous-chef] @copilot please run the pr-finisher skill, address the latest maintainer feedback, and rerun checks once the branch is ready.

...

Latest maintainer feedback was already covered by 0ce2115 and f2740c3; this pass did not require additional code changes. I reran local validation (make agent-report-progress, make test, make recompile) and the branch is clean locally. GitHub CI on the current head still needs a maintainer re-trigger from GitHub before merge.

pelikhan · 2026-07-20T11:47:43Z

@copilot merge main and recompile

…treat-empty-model-as-omitted Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot · 2026-07-20T12:10:36Z

@copilot merge main and recompile

Addressed in 9c01027. I merged the latest main into the branch, ran make recompile, make test, and make agent-report-progress, and pushed the updated branch.

Copilot · 2026-07-20T12:51:49Z

@copilot please run the pr-finisher skill, verify the latest maintainer-requested main merge/recompile is fully reflected on this branch, address any remaining unresolved review feedback, and rerun ...

Addressed in 00216ae. The branch still reflects the main merge/recompile from 9c01027; I fixed the remaining JS typecheck failure on top, reran local validation (make build-js, make test-impacted-js BASE_REF=4972443c609f2be21ab2e4421d06ccf93e98eba7, make fmt, make lint, make test-unit, make test, make recompile, make agent-report-progress), and the branch is ready for a maintainer CI re-trigger.

@copilot fix https://github.com/github/gh-aw/actions/runs/29741214191/job/88352169803#step:4:1

Fixed in 00216ae. The failing impacted-js-tests step was the new model_fallback.cjs helper missing a logger type annotation; local make build-js and make test-impacted-js BASE_REF=4972443c609f2be21ab2e4421d06ccf93e98eba7 now pass.

github-actions · 2026-07-20T13:02:15Z

🤖 PR Triage

Field	Value
Category	bug
Risk	🟡 medium
Priority Score	65 / 100
Action	⚡ `fast_track`

Score breakdown: impact 30 + urgency 20 + quality 15

Rationale: Empty expression for engine.model propagates """ to engine, making optional variables mandatory. Breaks workflows relying on unset vars.*`. Focused change (16 files), non-draft. CI in progress. Expedited review recommended once CI completes.

Run §29743972799

Generated by 🔧 PR Triage Agent · 47.4 AIC · ⌖ 5.32 AIC · ⊞ 5.6K · ◷

Copilot · 2026-07-20T13:02:27Z

@copilot fix https://github.com/github/gh-aw/actions/runs/29741214191/job/88352169803#step:4:1

Fixed on top of that run in 00216ae. The failing step was the JS typecheck for model_fallback.cjs; the current branch now passes local make build-js and make test-impacted-js BASE_REF=origin/main.

gh-aw-bot · 2026-07-20T16:32:08Z

@copilot please run the pr-finisher skill, address unresolved review feedback, and rerun checks once the branch is up to date.

Unresolved review threads:

copilot-pull-request-reviewer: Treat expression-backed engine.model as omitted when empty #46777 (comment)
copilot-pull-request-reviewer: Treat expression-backed engine.model as omitted when empty #46777 (comment)
copilot-pull-request-reviewer: Treat expression-backed engine.model as omitted when empty #46777 (comment)
copilot-pull-request-reviewer: Treat expression-backed engine.model as omitted when empty #46777 (comment)

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

pi.dev

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "pi.dev"

See Network Configuration for more information.

Generated by 👨‍🍳 PR Sous Chef · 8.75 AIC · ⌖ 14.8 AIC · ⊞ 6.9K · ◷
_{Comment /souschef to run again}

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot · 2026-07-20T16:53:54Z

@copilot please run the pr-finisher skill, address unresolved review feedback, and rerun checks once the branch is up to date.

Unresolved review threads:...

Addressed in a915121. I verified the branch is up to date and mergeable, added a Claude composite-expression regression test, and reran make fmt, go test ./pkg/workflow -run '^TestExpressionModelUsesEnvVar$', make lint, make test-unit, make test, make recompile, make agent-report-progress, and final review validation. CI on this new head is still stale until a maintainer re-triggers it.

pelikhan · 2026-07-20T17:12:31Z

@copilot handle evals job

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot · 2026-07-20T17:58:53Z

@copilot handle evals job

Addressed in 1f3575b. Evals setup/parse now preserve expression-backed GH_AW_EVALS_MODEL, add GH_AW_MODEL_FALLBACK for empty runtime resolution, use that fallback in run_evals.cjs, and recompile the affected lock files.

gh-aw-bot · 2026-07-20T17:59:52Z

@copilot please run the pr-finisher skill, address unresolved review feedback on the current head, and summarize the exact next maintainer action after checks are green.

Unresolved review feedback:

Copilot: Treat expression-backed engine.model as omitted when empty #46777 (comment)
Copilot: Treat expression-backed engine.model as omitted when empty #46777 (comment)
Copilot: Treat expression-backed engine.model as omitted when empty #46777 (comment)
Copilot: Treat expression-backed engine.model as omitted when empty #46777 (comment)
Copilot: Treat expression-backed engine.model as omitted when empty #46777 (comment)
Copilot: Treat expression-backed engine.model as omitted when empty #46777 (comment)
Copilot: Treat expression-backed engine.model as omitted when empty #46777 (comment)
Copilot: Treat expression-backed engine.model as omitted when empty #46777 (comment)

Run: https://github.com/github/gh-aw/actions/runs/29765403629

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

pi.dev

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "pi.dev"

See Network Configuration for more information.

Generated by 👨‍🍳 PR Sous Chef · 9.94 AIC · ⌖ 4.26 AIC · ⊞ 6.9K · ◷
_{Comment /souschef to run again}

github-actions · 2026-07-21T08:28:31Z

🎉 This pull request is included in a new release.

Release: v0.82.15

Initial plan

18ae7a0

Copilot AI assigned Copilot and pelikhan Jul 20, 2026

Copilot started work on behalf of pelikhan July 20, 2026 09:13 View session

Copilot AI linked an issue Jul 20, 2026 that may be closed by this pull request

Treat an empty expression-based engine.model as omitted #46499

Closed

Copilot AI changed the title ~~[WIP] Fix handling of empty engine model in GitHub Actions~~ Treat expression-backed engine.model as omitted when empty Jul 20, 2026

Copilot finished work on behalf of pelikhan July 20, 2026 09:47

Copilot AI requested a review from pelikhan July 20, 2026 09:47

pelikhan marked this pull request as ready for review July 20, 2026 09:48

Copilot AI review requested due to automatic review settings July 20, 2026 09:48

Copilot started reviewing on behalf of pelikhan July 20, 2026 09:49 View session

Copilot AI reviewed Jul 20, 2026

View reviewed changes

Copilot started work on behalf of pelikhan July 20, 2026 09:53 View session

Copilot AI and others added 2 commits July 20, 2026 10:20

Handle expression model fallback in JS runtime

0ce2115

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Fix runtime fallback test module format

f2740c3

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot finished work on behalf of pelikhan July 20, 2026 10:32

Merge branch 'main' into copilot/treat-empty-model-as-omitted

f312f77

github-actions Bot mentioned this pull request Jul 20, 2026

[pr-sous-chef] Run report — 0 nudged, 3 skipped #46687

Closed

Copilot started work on behalf of gh-aw-bot July 20, 2026 10:42 View session

Copilot finished work on behalf of gh-aw-bot July 20, 2026 10:57

Copilot AI requested a review from gh-aw-bot July 20, 2026 10:57

Copilot started work on behalf of pelikhan July 20, 2026 11:48 View session

Merge remote-tracking branch 'refs/remotes/origin/main' into copilot/…

9c01027

…treat-empty-model-as-omitted Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot finished work on behalf of gh-aw-bot July 20, 2026 12:52

Copilot started work on behalf of pelikhan July 20, 2026 12:52 View session

github-actions Bot added pr-action:fast_track pr-agent:copilot pr-priority:high pr-risk:medium pr-type:bug labels Jul 20, 2026

github-actions Bot mentioned this pull request Jul 20, 2026

[PR Triage Report] PR Triage Report — 2026-07-20 · Run §29743972799 #46832

Closed

Copilot finished work on behalf of pelikhan July 20, 2026 13:02

github-actions Bot mentioned this pull request Jul 20, 2026

[Contribution Check Report] Contribution Check - 2026-07-20 #46693

Closed

Merge branch 'main' into copilot/treat-empty-model-as-omitted

44d118f

Copilot started work on behalf of gh-aw-bot July 20, 2026 16:32 View session

test: cover composite claude model expressions

a915121

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot finished work on behalf of gh-aw-bot July 20, 2026 16:55

Copilot started work on behalf of pelikhan July 20, 2026 17:12 View session

Fix evals model fallback handling

1f3575b

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot finished work on behalf of pelikhan July 20, 2026 17:59

Copilot started work on behalf of gh-aw-bot July 20, 2026 18:00 View session

pelikhan merged commit 7666650 into main Jul 20, 2026
1 check failed

pelikhan deleted the copilot/treat-empty-model-as-omitted branch July 20, 2026 18:03

Copilot stopped work on behalf of gh-aw-bot due to an error July 20, 2026 18:03
The session was cancelled by the user.

Uh oh!

Conversation

Copilot AI commented Jul 20, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Review details

Uh oh!

Copilot AI Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

pelikhan commented Jul 20, 2026

Uh oh!

Copilot AI commented Jul 20, 2026

Uh oh!

gh-aw-bot commented Jul 20, 2026

Uh oh!

Copilot AI commented Jul 20, 2026

Uh oh!

pelikhan commented Jul 20, 2026

Uh oh!

Copilot AI commented Jul 20, 2026

Uh oh!

Copilot AI commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 20, 2026

🤖 PR Triage

Uh oh!

Copilot AI commented Jul 20, 2026

Uh oh!

gh-aw-bot commented Jul 20, 2026

Uh oh!

Copilot AI commented Jul 20, 2026

Uh oh!

pelikhan commented Jul 20, 2026

Uh oh!

Copilot AI commented Jul 20, 2026

Uh oh!

gh-aw-bot commented Jul 20, 2026

Uh oh!

Uh oh!

github-actions Bot commented Jul 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Copilot AI commented Jul 20, 2026 •

edited by github-actions Bot

Loading

Copilot AI commented Jul 20, 2026 •

edited

Loading