Repository navigation
fix(localresource): do not render files when there is nothing to substitute - #8
Conversation
…titute
`copier.WithGoTemplate(values)` was installed unconditionally, so every file a
LocalResource commits went through text/template + sprig — including when
`placeholdersToOverride` was absent and `values` was nil. Rendering is not a
no-op on content that merely looks like a template, so a CR asking to copy bytes
did not get its bytes copied.
Found on krateo-057. LocalResource/publish-sock-shop-003, publishing a Helm chart
through the Blueprint Builder, is stuck at:
create failed: unable to copy files:
template: template:61: function "toYaml" not defined
`toYaml` is a Helm builtin, not a sprig one — as are `include`, `required` and
`tpl` — so a Helm chart template can never be committed verbatim through this
path. Which is precisely what the Blueprint Builder exists to do. The CR sets no
placeholders at all; it was asking for a byte-for-byte copy and got a render.
The loud failure is the lucky half. A template that merely PARSES is executed
against an empty value set: the added test shows `replicas: {{ .Values.replicas }}`
committed as `replicas: <no value>`. A corrupted chart that still looks like a
chart, with no error anywhere and nothing downstream able to tell.
The fix mirrors the sibling repo controller, which already installs a templating
pass only when it has values (`internal/controllers/repo/repo.go`, `if values !=
nil`). The two controllers simply disagreed about this. Behaviour with
placeholders present is unchanged.
Tests (internal/tools/copier/helm_verbatim_test.go), all four passing:
- a Helm chart copies verbatim with no templating pass installed
- with the pass installed it fails with the exact krateo-057 error string
- the silent case, asserted rather than described: `{{ .Values.replicas }}`
renders to `<no value>` and would be committed
- placeholders still substitute when the CR actually asks for them
No API, CRD or chart change: a LocalResource that wants substitution keeps
declaring placeholdersToOverride, and one that does not now gets what it asked
for. Consumers needing a choice of engine on LocalResource (repo has the
`krateo.io/templating-engine` annotation, localresource does not) is a separate
follow-up and is not needed to unblock this.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JumEqTGGJ2hf3vjE3xFQsv
…o/templating-engine
Follow-up on review feedback: the first commit inferred intent from
`placeholdersToOverride` being empty, and that inference is not always right.
git-provider templates whatever file it is given, not only Helm manifests, and a
file can legitimately template with no inputs at all — `{{ now | date
"2006-01-02" }}`, `{{ uuidv4 }}`, `{{ env "USER" }}`, `{{ randAlphaNum 6 }}`.
Under the first commit alone those stopped rendering, which is a regression for
anyone relying on them.
So the CR can now say what it means, and the annotation the repo controller has
carried all along is extended to localresource:
(absent) render only if placeholdersToOverride is set <- default
gotemplate always render, including with no placeholders
none never render, even with placeholders declared
Anything else is rejected with an error naming the offending value, rather than
silently treated as one of the above — guessing is how the original mismatch went
unnoticed.
The default is unchanged from the first commit and is the safe reading: a
resource with no values to inject can only be damaged by a render pass. It fixes
the Helm case that prompted this (`function "toYaml" not defined` on
LocalResource/publish-sock-shop-003) and matches the repo controller, which
likewise installs a pass only when it has values. `gotemplate` restores the
value-free behaviour for anyone who wants it; `none` states verbatim intent
explicitly, and keeps holding if placeholders are added later.
The annotation string now has ONE definition, in the new
internal/controllers/common/templating package, referenced by both controllers.
repo.AnnotationTemplatingEngine is kept as a deprecated alias so existing
references keep compiling. This is deliberate: the two controllers have now
shipped three production defects by disagreeing about a shared rule — which auth
methods consume a username (#5/#6), when to install a templating pass (this),
and whether a failed ConfigMap load should abort a sync (#9). A shared constant
is a small thing, but it is the seam where two of those started.
The decision itself is extracted into `templatingOptions(annotations, values)`
so it is readable and testable in one place instead of reconstructed from a
switch inside the sync path.
Tests (internal/controllers/localresource/templating_options_test.go) exercise
the real copier, so they assert what lands in the target repo rather than which
options were selected:
- default + no values -> verbatim (the sock-shop case)
- default + values -> substituted (unchanged)
- gotemplate + no values -> renders `{{ "x" | upper }}` to `X`
- none + values declared -> still verbatim
- unknown engine -> error naming the value
- gotemplate on a Helm builtin -> still fails, and should: the annotation is a
choice, not a promise that Helm syntax survives a Go render
docs/local-resource.md documents all three modes, with the Helm failure modes
spelled out (toYaml/include fail; `{{ .Values.x }}` silently becomes `<no
value>`; `{{/* */}}` headers are deleted).
go build ./..., go vet and go test ./internal/... clean — 8 packages ok.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JumEqTGGJ2hf3vjE3xFQsv
Update — the annotation, after review feedbackThe first commit inferred intent from
So the CR can now say what it means, using the annotation the repo controller has carried all along:
Anything else is rejected with an error naming the value, rather than silently treated as one of the above. The default is unchanged and still fixes the Helm case. One definition of the annotationThe string now lives once, in a new That is deliberate. These two controllers have now shipped three defects by disagreeing about a shared rule — which auth methods consume a username (#5/#6), when to install a templating pass (this), and whether a failed ConfigMap load aborts a sync (#9). A shared constant is small, but it is the seam where two of them started. TestsSix cases, exercising the real copier so they assert what lands in the target repo:
|
…that fail
Two changes, both consequences of the same root cause: git-provider rendered with
Go's `{{ }}`, a delimiter Helm already owns, and reported failures without saying
which file they came from.
## 1. Default delimiters are now `{% %}`
The files this provider publishes are overwhelmingly Kubernetes manifests and
Helm charts. Sharing `{{ }}` made it impossible to publish a chart WITH
substitution at all:
{{ toYaml x }} / {{ include "y" . }} fail — Helm builtins, not sprig ones
{{ .Values.replicas }} parses, renders to the literal `<no value>`
{{/* ... */}} deleted (helm create writes one into
every _helpers.tpl)
With `{% %}` both systems coexist: this provider substitutes its own placeholders
at publish time and leaves Helm's untouched for install time. That case was
previously impossible — `none` preserved Helm but never substituted, and
`gotemplate` substituted but destroyed the chart.
`{% %}` was chosen by testing candidates against real YAML rather than by taste:
<< >> breaks on the merge key `<<: *defaults` (parse error: expected :=)
[[ ]] breaks on flow sequences `[[1,2],[3,4]]` (parse error: unexpected ",")
{% %} safe against both, and not Helm syntax
`krateo.io/templating-delims: "left,right"` overrides it — `"{{,}}"` restores Go's
native syntax where the content is known not to be a chart. A malformed value is
rejected, not silently ignored: quietly using different delimiters than the author
asked for leaves their placeholders unsubstituted in a committed file.
This is a behaviour change, and the blast radius was measured before making it:
on the live cluster 0 of 13 LocalResources declare placeholdersToOverride and no
Repo CRs exist at all. The only `{{ }}` users in existence were this repo's own
fixtures and docs, both updated here. A CR still written with `{{ .name }}` now
gets it left LITERAL in the output — visible in the committed file, not silently
blanked.
## 2. status.templatingErrors lists every file that failed
Go reports template positions against an anonymous template
("template: template:61: ..."), which is useless when a sync copies more than one
file. The copier now attributes each failure to its source path, and — the part
that matters — collects them ALL rather than stopping at the first, so one sync
reports every offending file:
status:
templatingErrors:
- path: templates/a.yaml
message: 'template: template:1: function "toYaml" not defined'
- path: templates/b.yaml
message: 'template: template:1: function "include" not defined'
Copy still returns an error, so callers abort before committing; a failed sync
publishes nothing, and the list is cleared on the next successful one. Both
controllers populate it through one shared converter.
## Shared definitions
apis/common/v1alpha1 (TemplatingError) and internal/controllers/common/templating
(annotations, engines, delimiter parsing, status conversion) exist so repo and
localresource cannot describe the same concept differently. These two controllers
have now produced four defects by disagreeing about a shared rule — usernames
(#5/#6), when to template (#8), ConfigMap load failure (#9), and this.
## Verified
go build ./... · go vet · go test ./internal/... ./apis/... 8 packages ok
controller-gen object + crd (v0.19.0, the pinned version) reproducible
kubectl apply --dry-run=server -f crds/ on a live cluster both CRDs configured
New tests: delimiter coexistence (Helm syntax preserved while `{% %}` substitutes),
`{{,}}` opt-in still works and still collides with Helm (honest outcome of asking
for it), malformed delimiters rejected, multi-file error collection with paths,
single-failure message names the file, errors reset between copies.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JumEqTGGJ2hf3vjE3xFQsv
status.templatingErrors named each failing FILE, but carried only one message per
file, because that is all text/template gives: it stops at the first error. A file
with three undefined functions reported one, and the operator fixed it, re-synced,
and met the next.
Worse, the most damaging class was invisible. A reference to a value that was
never declared is not an error to the renderer — `{% .environment %}` with no
`environment` placeholder renders the literal `<no value>` and is committed with a
green status. Nothing in the previous version could report that, which made
templatingErrors blind to exactly the silent corruption this series has been
chasing.
tools/template.Validate finds both classes in one pass. It parses with
parse.SkipFuncCheck — which yields a tree even when the functions do not exist —
then walks it collecting:
- every function identifier absent from sprig AND from text/template's builtins
- every top-level field reference absent from the declared values
Before, for one file:
template: template:1: function "toYaml" not defined
After, same file:
template:2:8: no value declared for ".environment"
template:3:13: no value declared for ".a"
template:3:6: function "toYaml" not defined
template:4:6: function "include" not defined
with line:column, which the renderer's message does not carry.
Validate REPORTS, it does not gate: Render remains the source of truth for whether
a file renders, and a file that renders cleanly is never inspected. The scanner
runs only on the failure path, so the success path costs nothing.
Two details that matter for message quality:
- When Validate's own parse cannot continue (a genuine syntax error, or a `$var`
it cannot resolve) it returns a single UNLOCATED problem. The copier detects
that via Problem.Located() and keeps the RENDERER's error instead — otherwise
a `$svc` the scanner cannot resolve would mask the undefined function the
renderer would have named, which is a worse message, not a better one.
- RenderFailed.Error() now counts problems and distinct files separately.
"4 problems in 1 file" is a very different situation from "4 problems in
4 files", and the condition message is often all anyone reads.
Helm syntax is invisible to the scanner under the default delimiters, as it should
be: it is not ours to validate.
Tests: every undefined function reported (Render reports exactly one, asserted);
undeclared values caught where Render returns nil and emits `<no value>`
(asserted); both classes mixed; clean templates silent; Helm syntax ignored under
`{% %}`; syntax errors yield exactly one report; no false positives on builtins,
pipelines or command arguments; multi-problem single-file counted correctly in the
aggregate message.
go build ./... · go vet · go test ./internal/... ./apis/... 8 packages ok
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JumEqTGGJ2hf3vjE3xFQsv
Two gaps found by testing the design rather than trusting it.
## 1. The scanner only ran on files that had already failed
A reference to a value that was never declared is NOT an error to text/template —
it renders the literal `<no value>` and returns nil. So a file whose ONLY fault
was an undeclared value never entered the failure path, the scanner never saw it,
and it was committed silently. status.templatingErrors was blind to exactly the
corruption it was added to surface. Demonstrated with a four-file tree: the
syntax-error file and the undefined-function file were both reported, and the
undeclared-value file was not.
Validation now runs on every rendered file. A file with an undeclared value fails
the sync instead of publishing `<no value>`.
That IS a behaviour change and it is the intended one: a placeholder is declared
on the same CR that references it, so referencing an undeclared one is a typo
essentially every time, and the alternative is a corrupted file with a green
status. Render remains the source of truth for HOW a file renders; the scanner
decides whether it should have been published at all.
Also confirmed while testing, and worth recording because it was asked: a syntax
error in one file does NOT stop the others being checked. copyFile records and
returns, so the walk continues — a tree with a syntax error, an undefined
function and an undeclared value reports all three files in one sync.
## 2. Repo disagreed with LocalResource, again
Repo picked its engine with `if engine == "gotemplate" { ... } else { mustache }`,
so EVERY other value fell through to mustache:
- "none" silently got mustache — the opposite of what it asks for
- "gotmeplate" silently got mustache — and mustache renders an unresolved key
as EMPTY (AllowMissingVariables defaults to true), so a typo in an annotation
quietly gutted the copied files and reported success
Repo now uses the shared vocabulary: `none` is honoured, `gotemplate` gets the
delimiters and the scanner, an unrecognised value is rejected by name, and an
absent annotation still means mustache so existing Repos are unaffected. The
decision is extracted into templatingOptions(), mirroring localresource, so both
controllers are readable side by side.
mustache stays available and is now documented as LEGACY where someone will read
it — it uses `{{ }}` so it collides with Helm exactly as Go templates did, it
fails silently rather than loudly, and it cannot be scanned, so templatingErrors
carries no per-problem detail for it.
## Tests
New internal/controllers/repo/templating_options_test.go drives the real copier:
no values copies verbatim; gotemplate coexists with Helm syntax; the scanner
reaches the Repo path and names undeclared values; "none" is honoured rather than
becoming mustache; an unknown engine is rejected by name; mustache still works AND
the test pins WHY it is legacy (it eats `{{ .Values.replicas }}` to an empty
string); the default remains mustache; malformed delimiters are rejected.
go build ./... · go vet · go test ./internal/... ./apis/... 8 packages ok
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JumEqTGGJ2hf3vjE3xFQsv
…o end
CI caught a real break, not the flake this branch hit earlier: TC09 is the
integration test for the gotemplate annotation, and its fixture used `{{ .name }}`
with an expectation of substitution. With `{% %}` as the default that line is no
longer ours to render, so the file arrived verbatim and the assertion failed. The
test was right and the fixture was stale.
TC09 now uses the default delimiters AND carries a Helm expression beside them:
"Hello {% .name %}!\nreplicas: {{ .Values.replicas }}\n"
->
"Hello GoTemplate!\nreplicas: {{ .Values.replicas }}\n"
which makes it a stronger test than it was. The coexistence — our placeholder
substituted, Helm's untouched — is the whole point of the delimiter choice, and
it is only observable end to end through a real clone, render, commit and push.
The unit tests can show the copier's behaviour; only this shows what lands in the
remote.
TC11 pins the other direction: `krateo.io/templating-delims: "{{,}}"` restores
Go's native syntax for content known not to be a chart, and still substitutes.
Without it the opt-in path had no integration coverage at all.
go build ./... · go vet -tags integration · go test ./internal/... ./apis/...
8 packages ok. The integration suite needs Docker and Gitea, so TC09/TC11 are
verified by CI rather than locally — which is how the stale fixture surfaced.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JumEqTGGJ2hf3vjE3xFQsv
A LocalResource that asks to copy bytes does not get its bytes copied
copier.WithGoTemplate(values)is installed unconditionally, so every file goes throughtext/template+ sprig — including whenplaceholdersToOverrideis absent andvaluesisnil. Rendering is not a no-op on content that merely looks like a template.Found on krateo-057.
LocalResource/publish-sock-shop-003, publishing a Helm chart through the Blueprint Builder, is stuck at:toYamlis a Helm builtin, not a sprig one — as areinclude,requiredandtpl. So a Helm chart template can never be committed verbatim through this path, which is exactly what the Blueprint Builder exists to do. That CR sets no placeholders at all.The loud failure is the lucky half
A template that merely parses is executed against an empty value set. The added test asserts this rather than describing it:
That gets committed. A corrupted chart that still looks like a chart, no error anywhere, and nothing downstream able to tell. The
toYamlcrash is the case that at least announces itself.The fix
Install the templating pass only when there are values — which is what the sibling repo controller already does:
The two controllers simply disagreed about this. Behaviour when placeholders are present is unchanged.
Tests
internal/tools/copier/helm_verbatim_test.go— four cases, all passing:TestCopyLeavesHelmTemplateVerbatimWhenNotTemplatingTestGoTemplatePassRejectsHelmBuiltinsTestGoTemplatePassSilentlyEmptiesUnknownValues{{ .Values.replicas }}→<no value>TestPlaceholdersStillSubstituteWhenPresentgo build ./...,go vetandgo test ./internal/...clean — 7 packages ok.Scope
No API, CRD or chart change. A LocalResource wanting substitution keeps declaring
placeholdersToOverride; one that does not now gets what it asked for. No change is needed inbuilder-publish— it already sets no placeholders.Giving LocalResource a choice of engine (repo has the
krateo.io/templating-engineannotation, localresource does not) is a reasonable follow-up, but is not needed to unblock this and is deliberately not in scope here.Related: #5 / #6 (the bearer/usernameRef connect failure) — that one blocked all twelve LocalResources at connect; with it fixed via the chart side, eleven of twelve now sync and this is the last one standing.
🤖 Generated with Claude Code
https://claude.ai/code/session_01JumEqTGGJ2hf3vjE3xFQsv