Skip to content

fix(build): honor PATH set in environment.environment - #2740

Open
theluckystrike wants to merge 5 commits into
chainguard-dev:mainfrom
theluckystrike:fix/honor-environment-path
Open

theluckystrike wants to merge 5 commits into
chainguard-dev:mainfrom
theluckystrike:fix/honor-environment-path

Conversation

@theluckystrike

Copy link
Copy Markdown

Fixes #1956.

A PATH set under environment.environment never reached the steps. runPipeline starts every step's envOverride with a fixed default PATH, and the runners (bubblewrap, docker, qemu) apply cfg.Environment first and envOverride after it. The default PATH therefore replaced the configured one, while other variables such as FOO in the issue came through as expected. A PATH set in a step's own environment already worked, because it is copied into envOverride after the default.

With this change runPipeline only adds the default PATH when the runner config has no PATH. The default value moves into a defaultPath constant and is the same string as before. Nested steps inherit from their parent as before, so they see the configured PATH too. Builds that do not set PATH get the same PATH as today.

TestRunPipelineHonorsConfiguredPath in pkg/build/pipeline_test.go parses a config, runs its steps through the build and the test pipeline runners with the existing env recording runner, and checks the PATH each step gets. With PATH set to /usr/bin in environment.environment, the step and its nested step must see /usr/bin and a step with its own PATH keeps it. Without any configured PATH those two steps get the default. On main (76b6e95) the first case fails because both steps get the default PATH instead of /usr/bin. With the change both cases pass.

I ran go test -count=1 ./... on main and on this branch with Go 1.27.3 on macOS. Both runs have 19 packages ok and the same 24 failing test lines, all in pkg/build (TestSplitManpagesSucceeds, TestSplitManpagesFailsLoudly and TestSplitAllDocsMergesRoots). They fail because the split pipeline scripts call GNU rmdir options that the BSD rmdir on macOS does not have. This branch has 946 passing test lines against 943 on main, which is the new test and its two cases. go vet ./pkg/build/ is clean, go build ./... works and gofmt reports nothing.

Functional changes. I did not build Wolfi or any package with a real runner, so I leave the box about building all of Wolfi unchecked. A package only sees a different PATH if it sets PATH in environment.environment, and right now that setting has no effect. The SCA and linter sections do not apply.

Not run. The e2e targets, the integration tag, -race and any container or qemu runner. The qemu runner comment still applies, since PATH there comes from the openssh build and this change does not touch it.

@theluckystrike
theluckystrike force-pushed the fix/honor-environment-path branch from 03ac708 to 00a9237 Compare November 10, 2026 09:19
@theluckystrike

Copy link
Copy Markdown
Author

Rebased onto current main as a single GitHub-signed commit (00a9237) so the commit-signing policy passes. Diff is unchanged: pkg/build/pipeline.go + pkg/build/pipeline_test.go, same tree as before.

javacruft and others added 3 commits November 11, 2026 15:07
An empty PATH makes the shell look commands up in the current
directory, which for a step is the workspace holding the fetched
upstream source. Fall back to the default PATH in that case, as the
configuration defaults already do for HOME and GOPATH.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@chainguard

chainguard Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

Code reviewers

1 finding (1 low) · reviewed edd9707

Reviewer Selected because Verdict Findings
melange paths: pkg/build/pipeline.go matches **, and 1 more file matches comment 1
  • melange · comment: Verdict: comment. 0 Blocking, 1 Questions, 0 Pre-existing. The fix in runPipeline is correct, and TestRunPipelineHonorsConfiguredPath (now with an empty-PATH case) fails without it. My earlier compatibility question about existing packages that set PATH is still open: nobody has replied and the body does not answer it.
Standards: 1 read, 0 left out
  • Read: AGENTS.md

Reviewers are configured in .chainguard/reviewers.yaml files on the default branch.

@chainguard chainguard Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reviewer/melange · Commented

Verdict: comment. 0 Blocking, 1 Questions, 0 Pre-existing. I checked the fix against all three runners: bubblewrap_runner.go:192-197, docker_runner.go:235-240 and qemu_runner.go:224-236 all apply cfg.Environment first and envOverride second, so the root cause is as described. Nothing in config parsing puts PATH into environment.environment (config.go:1891-1897), so builds that never set PATH keep today's PATH. TestRunPipelineHonorsConfiguredPath fails without the change. The open question is which existing packages now get a PATH they never had before.

Comment thread pkg/build/pipeline.go
// An empty PATH counts as none: a shell reads it as "search the current
// directory", which is the workspace holding the fetched source.
// NOTE: This does not currently override PATH in the qemu runner, that's set at openssh build time
if r.config.Environment["PATH"] == "" {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reviewer/melange · low

Question: This changes behaviour for every existing package that sets PATH under environment.environment, test.environment.environment, or a subpackage's test environment. It also reaches PATH values loaded from an --env-file, because those are merged into cfg.Environment.Environment (config.go:1857-1868). Until now that value was silently replaced by the default PATH. From this change on it is used as written. None of the runners expand it: bwrap --setenv, docker exec Env and ssh Setenv all pass the string through unchanged. So a package with PATH: /usr/lib/foo/bin:$PATH would get a literal $PATH entry and lose /usr/bin. That breaks a build that passes today. A build PATH also reaches every test container now, because test.go:498 copies the build's environment.environment into the test config. Did you search wolfi-dev/os (and the enterprise package repos, if you can) for PATH under environment.environment and the test environments? Did any hit rely on $PATH or on directories missing from the test image? If there are hits, list them in the PR body, or fix them before this merges, so the change is announced as AGENTS.md (Compatibility) and .chainguard/review-prompt.md require.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I measured it against wolfi-dev/os at 129c3a0 (shallow clone, today). I parsed all 3,914 package YAMLs and checked environment.environment, test.environment.environment and every subpackage's test.environment.environment for PATH.

One hit: infinispan-16.3.yaml:170, test environment only:

/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin   (infinispan)
/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin   (default, pipeline.go:238)

Same six directories; only /usr/sbin and /usr/bin swap order. So in Wolfi, no package build changes, and one test gets an equivalent PATH.

What I can't measure from here is --env-file usage outside that repo, or enterprise-packages. If you'd rather be careful, I can log a warning when a configured PATH differs from the default, so changes are visible in build logs. Happy to add that.

@dustinkirkland dustinkirkland left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 8bc7ab4 against AGENTS.md (Compatibility, Runners, Go and commits). 0 blocking, 1 suggestion. No design doc found beyond #1956; the change does what the issue asks.

Watch focus items checked:

  • Supply chain: the net diff is only pkg/build/pipeline.go and pkg/build/pipeline_test.go. Both merge commits (ecd67c1, 8bc7ab4) have an empty remerge diff, so they add nothing beyond main. No new deps, no CI or workflow edits.
  • PATH security: PATH reaches cfg.Environment only from the package's environment.environment (and, for tests, test.environment.environment) or the operator's --env-file (config.go:1856-1868). Whoever writes those already controls every runs: block, so honoring their PATH crosses no new trust boundary. apko's locked image configuration doesn't add a PATH to the struct; it only sets PATH in the OCI config (apko pkg/build/oci/image.go:175), so builds that never set PATH keep the default. An empty PATH falls back to the default (pipeline.go:247). A relative entry, or a leading, trailing or doubled :, is passed through as written. That only puts the workspace on the search path when the package author writes it that way, the same as a cd plus ./tool in runs:.
  • Runners: bubblewrap, docker and qemu (qemu_runner.go:224-236) all apply cfg.Environment first and envOverride second. Nothing else in pkg/ reads envOverride["PATH"].

I'm not seconding the open question in T1. The author's wolfi-dev/os scan answers it for Wolfi.

The required e2e tests complete check hasn't reported at this head yet.

autorev-by: dustinkirkland

autorev-by: dustinkirkland

Comment on lines +430 to +434
{
name: "nothing sets PATH",
yaml: head + steps,
want: []string{stdPath, stdPath, "/opt/step/bin"},
},

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion (non-blocking). 4a8609f made an empty configured PATH fall back to the default (pipeline.go:247). Its commit message gives a safety reason: an empty PATH makes the shell search the workspace. No test covers that. If someone reverts to the earlier if _, ok := r.config.Environment["PATH"]; !ok form, steps run with PATH="" and every case here still passes. AGENTS.md asks that a fix come with a test that fails without it. Adding one case pins this down:

Suggested change
{
name: "nothing sets PATH",
yaml: head + steps,
want: []string{stdPath, stdPath, "/opt/step/bin"},
},
},
{
name: "configuration sets an empty PATH",
yaml: head + "environment:\n environment:\n PATH: \"\"\n" + steps,
want: []string{stdPath, stdPath, "/opt/step/bin"},
},

@theluckystrike
theluckystrike force-pushed the fix/honor-environment-path branch from 2d2293c to 8bc7ab4 Compare November 16, 2026 03:33
@theluckystrike

Copy link
Copy Markdown
Author

Added the empty PATH case you suggested in edd9707, as a row in TestRunPipelineHonorsConfiguredPath.

It covers the fallback from 4a8609f. With that change reverted locally the new row fails, and with it in place all three rows pass:

--- FAIL: TestRunPipelineHonorsConfiguredPath/configuration_sets_an_empty_PATH
...
--- PASS: TestRunPipelineHonorsConfiguredPath/configuration_sets_an_empty_PATH (0.00s)
ok  	chainguard.dev/melange/pkg/build	1.059s

Nothing else in pkg/build changes. The rest of the package fails the same 24 split/manpage subtests on macOS with or without this commit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PATH set in environment.environment is not honored

3 participants