Repository navigation
fix(scripts): escape flag values per output format in generate-federation-iac - #955
Conversation
…tion-iac
The generator put --account-name, --account-slug, --contact-email,
--cudly-api-url, --tenant-id, --project-id and --service-account-email
into the shared templates unescaped, so a quote, "${" or a newline could
change the structure of the generated tfvars, JSON or Bash file.
- renderTmpl now escapes every string field of iacData for the grammar
of the template it renders, chosen by file suffix: HCL for
*.tfvars.tmpl (\, ", and the "${" / "%{" template sequences), the
server's shellEscape rule for *.sh.tmpl, and JSON for *.json.tmpl. An
unknown suffix is an error. Fields are walked by reflection so a new
one cannot be missed.
- Any flag value containing a control character is rejected after the
existing field validators run. Values are also rendered into # comment
lines, where only a line break could change the structure.
- The server's bundle tfvars get the same HCL escaping. Their one
user-supplied value, ContactEmail (the session email), only has to
pass mail.ParseAddress and can carry quotes and "${" in a quoted local
part. CLI and CFN shell output keeps its existing shell escaping.
Output for values without metacharacters is byte-identical.
Closes LeanerCloud#952
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Note Currently processing new changes in this PR. This may take a few minutes, please wait... ⚙️ Run configuration
📒 Files selected for processing: 4
✨ Finishing Touches
Comment |
|
The Unit Tests failure on this PR is 🤖 Generated with Claude Code |
cristim
left a comment
There was a problem hiding this comment.
Independent review of 69d52ae: draft: blocks ready
No BLOCKING code findings. The script half of #952 is correct for the formats it can reach. The blocks below are gate items.
Findings
- NON-BLOCKING (security, follow-up, pre-existing):
internal/api/handler_federation.go:558-562rendersazure-wif-bicep-deploy.sh.tmplwith rawdataforformat=bicep|arm; onlycliand the CFN deploy script go throughshellEscapeData. Probe: session email"a$(id)id\"; touch X; \""@example.compassesmail.ParseAddressand lands asCONTACT_EMAIL="${CUDLY_CONTACT_EMAIL:-"a$(id)id\"; touch X; \""@example.com}"indeploy-azure.sh, so$(id)runs when the downloader runs the script. The victim is the owner of that email, so impact is low, but #952 asked the PR to confirm the server side and this path is not covered. Fix:shellEscapeData(data)at line 558, plus a test beside the new bundle test. - NON-BLOCKING:
--account-slugis unvalidated and used raw in output and zip entry names (scripts/generate-federation-iac.go:297-313,356,384).--account-slug ../../evilis accepted (exit 0). Escaping cannot fix this; file separately. - NON-BLOCKING: the Bash rule leaves
}unescaped inside${V:-...}, soa}brenders asab}(the test file documents this). On macOS/bin/bash3.2 an odd number of'inside"${V:-...}"is a syntax error. Both fail loud or corrupt only the value; I found no command execution (probed'"; touch X; "',$(), backticks on 3.2). - NON-BLOCKING: server
hclEscape(handler_federation.go:~414) does not reject control characters, unlike the script. Safe today becausemail.ParseAddressrejects CR/LF (probed) and the other fields are server-derived; note for later.hclEscape,shellEscapeand the reflect walker are duplicated between script and server (script is//go:build ignore, so sharing is not trivial); the cross-reference comment is acceptable. - NON-BLOCKING:
aws-cfn-deploy.sh.tmplneeds.OIDCIssuerHost, which the script'siacDatalacks, so--target aws --source gcp|azure --format bundlefails on main as well as on this head (pre-existing). The script's Bash escaper is therefore only reachable through that broken path; unit tests cover it.
What I attacked
- HCL: built
--account-name/--contact-emailpayloads with",\,${,%{,$${,%%{and$(...), rendered tfvars for all 5 target/source pairs, ranterraform fmtandterraform apply -var-fileon the aws-wif output; the variable round-trips byte for byte. The$${/%%{doubling stays correct when the input already contains$${. - JSON: slug
a"b\c</>érendered throughcf-params,jqparses it and the value is intact. - Shell: simulated
"${V:-<escaped>}"under bash for quote,$(), backtick, backslash and!payloads; no breakout. Findings 3. - Control chars: newline, CR, tab and ESC are rejected before any render;
checkNoControlCharsruns afterpopulateData, so the #930 validators (account ID shapes, subject claim, source account ID, Azure URL) still fire first. Escaping is defense in depth. - Server:
renderTemplate(tfvars, hclEscapeData(data))only affects tfvars; the CFN deploy script keeps its single shell escaping (the test asserts no$${). Finding 1 for the uncovered path. - Fail-before/pass-after: at the merge-base production code with the PR tests,
TestGenerator_Tfvars...,CFParams...,ShellTemplates...,RejectsControlCharactersand the serverBundleTfvarsEscapesSessionEmailall fail; at the headgo test -count=1 ./scripts/ ./internal/api/passes. Notego test ./scriptscan report a stale(cached)pass because the tests exec the generator; use-count=1.
Gate status (origin/main Review gate)
- Independent review: this comment, SHA above.
- CI: red on this SHA.
Unit Testsfails onTestScheduler_ListRecommendations_StaleCacheKicksBackgroundRefresh(author reports the same failure on main);Integration Testsalso failed andCI Successrolls up. Needs a re-run once main is green; not caused by this diff. Fork runs otherwise passed, pre-commit passed. - CodeRabbit: skipped (draft), no threads. Not pinged.
- Local verification: macOS, real generator binary and terraform, as above. No real-account verification.
- Labels: PR has none; #952 itself has no triage labels, so there is nothing to mirror. Maintainer triage needed.
- Size: +537 (non-blocking, external contributor).
Blocks ready
- CI green on this SHA (rerun after the main scheduler flake is fixed)
- Labels: triage #952, then mirror to the PR
- Mark ready for review (draft now)
- Optional before merge: finding 1 (one-line
shellEscapeDataat line 558) or a follow-up issue for it and finding 2
|
Thanks @cristim. I merged 🤖 Generated with Claude Code |
cristim
left a comment
There was a problem hiding this comment.
Independent delta review of 61be15a (previous: 69d52ae): no actionable findings
Delta: OLD_SHA is an ancestor of NEW_SHA. The only new commit is 61be15a, a merge of origin/main (parents: OLD_SHA and 2ef2f5d, picks up #960). No rebase, so range-diff is not needed. git diff OLD NEW on the PR's four files (internal/api/handler_federation.go, handler_federation_test.go, scripts/generate-federation-iac.go, scripts/generate_federation_iac_escape_test.go) is empty, so the escaping grammar for every output format and the injection payload tests are byte-identical to the reviewed code. The other 10 changed files in the delta come from main (scheduler/middleware/negative-cap tests). git merge-tree of origin/main and NEW_SHA is clean, and the PR diff against origin/main is still those 4 files (537 additions).
Previous non-blocking items, all unchanged and none addressed in this delta:
- handler_federation.go:558-562 raw bicep/arm deploy-azure.sh: NOT RESOLVED, tracked as #964.
--account-slug ../../evilaccepted: NOT RESOLVED (non-blocking).}in a shell value, odd'on bash 3.2: NOT RESOLVED (non-blocking).--target aws --source gcp|azure --format bundlefails on main too: NOT RESOLVED (pre-existing).
What I attacked: re-ran go test -count=1 ./scripts/ ./internal/api/ at NEW_SHA in a detached worktree: 3554 passed in 2 packages, which includes the escape payload tests.
CI on NEW_SHA: pending, not waited on. Run 38902343704 (CI - Build & Test) is queued/pending; Lint Workflows, Validate Terraform (azure) and the AWS Terraform state namespace check already pass. Run 38902343705 (pre-commit) is in progress. No action_required runs. The previous SHA's CI was green.
Gate: independent review covers this SHA (delta is main-only); local verification is tests above, labels unchanged from the prior review. The PR is still a draft and CI on NEW_SHA must go green before merge.
|
Gate record for 61be15a (origin/main CLAUDE.md "Review gate"):
Marking ready (owner-authorized) and merging at 61be15a with --match-head-commit. |
Root cause
scripts/generate-federation-iac.gocopies its flag values intoiacDataand renders the sharedinternal/iacfiles/templates/*.tmplwithtext/template, which does no escaping.--account-id,--source-account-id,--oidc-subject-claimand (for Azure)--cudly-api-urlare validated against strict allowlists. The other free-form flags are not:--account-name,--account-slug,--contact-email,--cudly-api-url(non-Azure),--tenant-id,--project-idand--service-account-email. A quote,${/%{or a newline in any of them could change the structure of the generated tfvars, CloudFormation params JSON or Bash file.Reproduced on
main(18ee477):Fix
I chose escaping by output format plus one generic rejection, not a per-field validator for each flag. A display name like
--account-namehas no grammar to validate against, and the same value lands in HCL, Bash and JSON, so a per-field rule would still need escaping behind it.renderTmplpicks an escaper from the template's file suffix and applies it to every string field ofiacData(walked by reflection, so a field added later is covered):*.tfvars.tmpl: HCL string escaping (\to\\,"to\",${to$${,%{to%%{).*.sh.tmpl: the same rule as the server'sshellEscape(\,",`,$).*.json.tmpl: JSON string escaping (encoding/json, HTML escaping off so<AZURE_TENANT_ID>stays as is).#comment lines, where only a line break can change the structure, and no flag can legitimately contain one. After the existing field validators run (so they still give their more specific errors), any flag whose value contains a control character (unicode.IsControl: newline, CR, tab, ESC and others) is rejected with an error that names the flag. Nothing is written.GET /api/federation/iaccan feed user-controlled values into its tfvars. It can, but only one:ContactEmailissession.Email. When an admin creates or edits a user, or a user changes their own email with their current password, the stored email only has to passmail.ParseAddressand is kept as typed. A quoted local part such as"o\"q${x}%{y}"@example.comtherefore reachescontact_email = "..."raw. That produces broken HCL or template evaluation inside that one string. It can't add attributes, becausemail.ParseAddressrejects CR/LF. The other tfvars fields are server-derived (AccountSlug: "target", the allowlisted source, OIDC constants, the STS account ID) or come from configuration (dashboardURLor the Lambda Function URL domain, not the Host header).addBundleTerraformnow renders the tfvars withhclEscapeData(data). The CLI scripts and the bundle's CFNdeploy-cfn.shkeep their existingshellEscapeDataand are not double-escaped. A test asserts this.Output is byte-identical for values with no metacharacters. I diffed every target/source tfvars combination plus
cf-params, rendered with benign values, betweenmainand this branch, and there were no differences. Hostile values now also show escaped inside the#header comments (e.g.# Target AWS account : pr\"od). That only changes how the comment looks.Verification
All Go commands ran with
AWS_CONFIG_FILE=/dev/null AWS_SHARED_CREDENTIALS_FILE=/dev/null, on macOS (Darwin 24.6, Go 1.26). Nothing touched a cloud API. Terraform was used only locally, asterraform consoleon scratch modules that declare variables and nothing else (no providers).Fail-before. The new tests were run on
mainwith the fix stashed (git stash push internal/api/handler_federation.go scripts/generate-federation-iac.go) and the new test file in place:TestGenerator_PlainValuesRenderVerbatim(the positive control) passes both before and after, as it should.Pass-after:
Mutation probes on the script. Each one was applied by hand and reverted. Every one fails at least one new test:
${rule fromhclEscapeTfvarsValuesCannotChangeStructure%{ruleTfvarsValuesCannotChangeStructurehclEscapefor.sh.tmplShellTemplatesEscapeValueshclEscapeorshellEscapefor.json.tmplCFParamsValuesCannotChangeStructurecheckNoControlCharsRejectsControlCharactersescapeDataentirelyhclEscapeDataBundleTfvarsEscapesSessionEmailReal-parser check. I generated hostile tfvars for gcp/azure, azure/aws and gcp/gcp and loaded each with
terraform console -var-file. Terraform returned the exact original--account-nameand--contact-emailstrings, including",\,${x}and%{y}. It also decoded the server test's expectedcontact_emailliteral back to the original email.How the tests work: the tfvars test checks that every non-comment line is exactly one quoted HCL attribute. It then checks that the set of attributes matches a benign run and that each value decodes back to its input; an unescaped
${/%{fails the decode. The JSON test unmarshals the output and checks there are exactly four parameters. The Bash test actually runs the rendered script withbashand compares what Bash sees against the literal inputs, which include$(printf INJECTED)and backticks.Out of scope / notes for follow-up
iacDatahas noOIDCIssuerHost, so--format bundlefor an AWS target with a non-AWS source fails to renderaws-cfn-deploy.sh.tmplonmaintoday. Because of that, the script cannot currently produce any.shoutput.TestGenerator_ShellTemplatesEscapeValuestherefore uses a probeaws-cfn-deploy.sh.tmpl(passed via--templates-dir) that has the same quoting shapes as the real templates. The.shescaping is in place now so that fixing chore(scripts): generate-federation-iac.go iacData has drifted out of sync with the templates it renders #153 doesn't reopen this issue.shellEscapedoes not handle}inside"${VAR:-default}". This applies to both the server and the script, since the script mirrors the server's rule. A}in the value ends the expansion early and corrupts the value. It can't run a command or break out of the quotes, because$, backticks,"and\are escaped. A fix needs a context-aware rule for the server's CLI templates too, so I left it alone. The shell test avoids}in that position and a comment explains why.deploy.shis rendered unescaped.writeAzureTemplateFilesrendersazure-wif-bicep-deploy.sh.tmplwith rawdatainstead ofshellEscapeData(data), unlikewriteCFNFiles. The template embedsContactEmailinCONTACT_EMAIL="${CUDLY_CONTACT_EMAIL:-...}", andmail.ParseAddressaccepts"a$(id)b"@example.com. This looks like a separate shell-injection bug in a server path that generate-federation-iac: CLI flag values reach tfvars/shell templates unescaped (HCL injection via --account-name) #952 doesn't cover, and it probably deserves its own issue.--account-slugin output file names. The script uses--account-slugverbatim in the default output file name (and in zip entry names), so a value with/or..can write outside the current directory. The operator controls this flag; I didn't change it.handler_federation.go(now about 865 lines) andhandler_federation_test.gowere already over the 500-line guideline before this change. The new script tests are in a new file,scripts/generate_federation_iac_escape_test.go, because both existing script test files are at the limit.Labels: I can't apply labels. Could a maintainer apply the triage labels? The issue has none yet; the issue suggests
type/security,type/bug,priority/p3,severity/low.Closes #952
🤖 Generated with Claude Code