Skip to content

fix(compose): keep env behind an embedded variable the re-parse canno… - #677

Open
Baaaki wants to merge 1 commit into
oblien:mainfrom
Baaaki:fix/673-embedded-compose-var-provenance
Open

fix(compose): keep env behind an embedded variable the re-parse canno…#677
Baaaki wants to merge 1 commit into
oblien:mainfrom
Baaaki:fix/673-embedded-compose-var-provenance

Conversation

@Baaaki

@Baaaki Baaaki commented Aug 21, 2026

Copy link
Copy Markdown

Summary

A compose ${VAR} expression embedded in a larger string lost the record of
which variable it referenced, so a half-interpolated value like
postgres://u:@db was indistinguishable from a literal the author typed. The
parser now names those variables, and keepUnresolvedEnv stops a push-deploy
re-parse from overwriting the env the user configured behind one.

Behavior

  • Embedded ${VAR:?msg} with no value → the meta names the variable and marks it
    required; keepUnresolvedEnv keeps the stored value instead of overwriting it.
  • Embedded bare ${VAR} / $VAR with no value → named, not marked required.
  • Embedded ${VAR:-default} → resolved; nothing reported, and a genuine upstream
    edit still drifts normally.
  • Embedded ${VAR:+alt} that yielded "" → resolved exactly as authored;
    nothing reported.
  • Variable resolves from .env or the caller's env → nothing reported, the value
    interpolates as before.
  • Whole-value expression → unchanged (source: "missing" + variable +
    required).
  • Same variable twice with different operators → named once; required is the
    OR of them.
  • Value with no ${} at all → unchanged, no meta fields added.

Motivation

resolveComposeValue builds meta carrying variable / required only when the
expression is the entire value. An embedded one falls through to the
interpolateComposeString branch, which recorded source: "interpolated" and
nothing else:

# named and flagged today
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD in .env}
#   → { source: "missing", variable: "POSTGRES_PASSWORD", required: true }

# resolved to postgresql://username:@postgres:5432, naming nothing
DATABASE_URL: postgresql://username:${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD in .env}@postgres:5432
#   → { source: "interpolated" }

The half-interpolated value is non-empty, so emptiness — the signal
everything downstream keys off — says nothing about it.

The concrete damage is in keepUnresolvedEnv. That function exists precisely to
stop a push-deploy re-parse from overwriting a value the user configured in the
wizard: the repo's .env holds the secret and is not committed, so the re-parse
cannot resolve it. Both of its guards miss the embedded shape —

if (value !== "" || meta[key]?.source !== "missing") continue;
//     ^ value is "postgres://u:@db", not ""    ^ source is "interpolated", not "missing"

— so the working DATABASE_URL the user had typed is replaced with the broken
one on the next push that touches the compose file. Silently, and on a push that
had nothing to do with that variable.

Related issue

Refs #673.

Deliberately a plain reference rather than an auto-closing link. This fixes the
provenance loss and the overwrite above, but ${} is still resolved once at scan
time and frozen onto the service row, so the "dynamic env" the issue title asks
for is not delivered here. The issue should stay open. See Out of scope below.

Per CONTRIBUTING this is a bug fix, so it did not need prior scope agreement —
the issue is linked because it reported the symptom.

Changes

apps/api

  • src/lib/compose-parser.ts — interpolation optionally reports variables that
    contributed nothing (new UnresolvedSink), threaded through nested defaults.
    resolveComposeValue collects them onto the meta as unresolvedVariables
    (names only, never values) plus required: true when any used :? / ?.
    A satisfied ${VAR:-default}, or a ${VAR:+alt} that correctly yielded "",
    is not reported — those resolved as authored. The bare $VAR spelling
    reports too.
  • src/modules/deployments/build.service.tskeepUnresolvedEnv now decides
    via isUnresolvedParse. Its first clause is the previous condition verbatim
    (value === "" && source === "missing"); the embedded arm is pure addition,
    so nothing that was kept before stops being kept.
  • src/lib/secret-env.tsmaskEnvironmentMeta rebuilds the object from an
    allowlist, so a field not named there never reaches the client. The new one
    holds variable NAMES only — the same class as the existing variable, which
    is already kept unmasked — so it is passed through while the value-bearing
    fields (resolvedValue, defaultValue, expression) stay masked.

No schema, endpoint, or dependency changes.

Verification

7 new tests. 5 of them fail without the source change — verified by stashing
apps/api/src/ and re-running with the tests in place:

$ git stash push -- apps/api/src/ && bunx vitest run \
    test/lib/compose-parser.test.ts test/modules/deployments/build.service.test.ts

⎯⎯⎯ Failed Tests 5 ⎯⎯⎯
 FAIL  test/lib/compose-parser.test.ts > names the unresolved variable when the expression is embedded in a string
 FAIL  test/lib/compose-parser.test.ts > names an embedded variable that is merely unset, without marking it required
 FAIL  test/modules/deployments/build.service.test.ts > keeps the stored value when an embedded variable did not resolve
 FAIL  test/modules/deployments/build.service.test.ts > still keeps the stored value for a whole-value variable
 FAIL  test/modules/deployments/build.service.test.ts > lets a real upstream edit through - a resolved value is not a hole
 Test Files  2 failed (2)

$ git stash pop && bun run test
 @repo/api:test:  Test Files  404 passed (404)
 @repo/api:test:       Tests  4869 passed | 3 skipped (4872)
 Tasks:    7 successful, 7 total
  Time:    2m35.869s

$ bun run --cwd apps/api lint
$ tsc --noEmit          # (the script's own echo; no diagnostics, exit 0)

The two parser tests that pass either way are deliberate: they pin the cases
that must not start reporting (a resolved variable, a satisfied default).
Without them, widening the :- arm later would silently break
keepUnresolvedEnv. The last two failures above are regression guards for
behaviour that already worked — they fail here only because the function was not
exported before, and they are what proves the rewritten guard did not change it.

Before / after, same compose file, no .env present:

BEFORE  DATABASE_URL = "postgresql://username:@postgres:5432"
        meta = { source: "interpolated", resolvedValue, expression }

AFTER   DATABASE_URL = "postgresql://username:@postgres:5432"   (unchanged)
        meta = { source: "interpolated", resolvedValue, expression,
                 unresolvedVariables: ["POSTGRES_PASSWORD"], required: true }

        → keepUnresolvedEnv now keeps the user's stored DATABASE_URL instead of
          overwriting it on the next compose-touching push.

A note on bun format: run as CONTRIBUTING prescribes, then reviewed and the
unrelated parts dropped. It rewrites ~200 files in this repo — all three source
files I touch are already Prettier-drifted at HEAD — so committing its output
would have violated the "diff is scoped" rule. My added lines are Prettier-clean
(prettier --check on them passes); the diff contains no reformatting of lines
I was not otherwise changing.

Out of scope

Two things this deliberately does not do:

  • The wizard still shows no "needs a value" state for the embedded row.
    missingEnvCount filters on source === "missing", and widening it needs a
    decision about the masking/reveal interaction — an unedited value arrives at
    the client as the mask sentinel, so "has the user fixed this yet?" is not
    answerable from the value alone. That felt like yours to make. Worth noting
    the API already returns missingRequiredEnv for this case and nothing in the
    dashboard reads it.
  • ${} is still not dynamic. It is resolved once at scan time and frozen
    onto the service row; only {{publicUrl:…}} and {{env:svc:KEY}} are
    deploy-time. Making it dynamic means persisting the expression and
    re-interpolating against the merged env layers at deploy — a much larger
    change, and the reason this PR stops short of the issue's headline ask.

Diff

apps/api/src/lib/compose-parser.ts                         +50/-8   (UnresolvedSink, meta fields)
apps/api/src/lib/secret-env.ts                             +10/-4   (mask allowlist)
apps/api/src/modules/deployments/build.service.ts          +24/-3   (isUnresolvedParse)
apps/api/test/lib/compose-parser.test.ts                   +72      (4 parser tests)
apps/api/test/modules/deployments/build.service.test.ts    +74      (3 keepUnresolvedEnv tests)

…t resolve

A `${VAR}` expression embedded in a larger string lost its provenance.
`resolveComposeValue` carries `variable`/`required` only when the expression is
the entire value, so `postgres://u:${PW}@db` resolved to the non-empty
`postgres://u:@db` and named no variable at all.

`keepUnresolvedEnv` exists to stop a push-deploy re-parse from overwriting a
value the user configured in the wizard — the repo's `.env` holds the secret and
is not committed, so the re-parse cannot resolve it. Both of its guards
(`value === ""`, `source === "missing"`) miss the embedded shape, so a working
DATABASE_URL was replaced with the broken one on the next push that touched the
compose file.

Interpolation now reports the variables that contributed nothing, carried on the
meta as `unresolvedVariables` (names only, never values) plus `required` for the
mandatory operators. A satisfied `${VAR:-default}`, or a `${VAR:+alt}` that
correctly yielded "", is not reported — those resolved as authored.
`maskEnvironmentMeta` is an allowlist, so the new field is named there or it
never reaches the client.

Refs oblien#673
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant