Skip to content

Commit 80cb92a

Browse files
donislawdevclaude
andauthored
damage: one rule about accept, two callers, and a site that lists every command (#100)
Three defects that were all the same shape: a list written out by hand that nothing compared against its source. The refusal for a target that damages its files and declares they will be accepted lived in the recipe reader alone. The command line never skipped past it - it never reached it, so `--damage zero-head --expected accept` ended with code 0 and wrote a manifest saying a deliberately broken file should be accepted, while the identical recipe was refused with code 3. The condition is now `Chain.ConflictsWithExpectation` in internal/damage with two CALLERS rather than two copies: the recipe reader, which keeps reporting it beside the other problems of that recipe and with the address of the target, and the engine, which every surface passes through. The command line ends with code 2 and writes nothing. Only accept is refused - reject, sanitize and unspecified beside damage, and accept without damage, all still work, and the guard asserts that so this cannot become a wall. The website listed nine of the ten commands, because `tfg damage` arrived and the list was a hand copy of what `tfg --help` prints. site.Facts carried no commands at all, so nothing could compare. The block is rendered now: names from the help itself, summaries from the language file, both directions checked, and the parser refuses rather than returning an empty list - a guard that stopped finding the block would compare the page against nothing and pass. The site also gains a section on producing files that are broken on purpose, and README gains the --damage row its flag table never had. sign_release.py asked the certificate store through the Cert: drive, which does not exist when Windows PowerShell 5.1 is launched from pwsh. The error is non terminating, so the exit code was zero, and stderr was read only on a non-zero code - the script announced a missing card while the card was in the reader. It opens the store through .NET now and prints what PowerShell said whatever the code was. Guards: three new, eight mutations, all caught. Two came back NOT CAUGHT first and both were statements about the mutation rather than the code. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
1 parent be8c953 commit 80cb92a

20 files changed

Lines changed: 677 additions & 44 deletions

File tree

‎.github/scripts/sign_release.py‎

Lines changed: 48 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -114,6 +114,22 @@ def powershell(script):
114114
capture_output=True, text=True)
115115
if out.returncode != 0:
116116
raise SystemExit("sign_release: powershell failed:\n%s" % out.stderr.strip())
117+
# PowerShell errors are NON TERMINATING by default, so a script can print
118+
# a page of complaints and still exit zero. Reading stderr only on a
119+
# non-zero code therefore threw away the one sentence that said what went
120+
# wrong, and left the caller looking at empty output with no reason for it.
121+
#
122+
# It cost an hour on 2026-09-09 signing v0.3.0: the certificate lookup came
123+
# back empty and the script blamed a missing card, while the card was in
124+
# the reader and readable. The complaint was there the whole time and
125+
# nothing printed it. O200.
126+
#
127+
# A note rather than a failure, because a warning is not a refusal and the
128+
# caller may have asked something that legitimately produces one.
129+
if out.stderr.strip():
130+
print(" powershell also said:")
131+
for line in out.stderr.strip().splitlines():
132+
print(" %s" % line)
117133
return out.stdout
118134

119135

@@ -170,17 +186,41 @@ def signing_thumbprint(pin):
170186
the only selector it takes, and the repository pins SHA-256 because that is
171187
the digest worth pinning. Resolving one to the other here means the two can
172188
never drift apart in a configuration file.
189+
190+
THE STORE IS OPENED THROUGH .NET RATHER THAN THROUGH THE Cert: DRIVE, and
191+
that is a measurement rather than a preference. The drive is provided by
192+
Microsoft.PowerShell.Security, which Windows PowerShell 5.1 only loads when
193+
PSModulePath points at its own module directory - and a 5.1 launched from
194+
inside pwsh 7 is handed pwsh's PSModulePath instead. Measured 2026-09-09 on
195+
this machine, from a python started under pwsh:
196+
197+
Get-ChildItem Cert:\\CurrentUser\\My -> 0, plus
198+
"Cannot find drive. A drive with the name 'Cert' does not exist."
199+
X509Store('My','CurrentUser') -> 11
200+
201+
Both ended with code ZERO, because a PowerShell error is non terminating -
202+
so the script saw empty output and reported a missing card while the card
203+
was in the reader. It cost an hour signing v0.3.0 and the release went out
204+
through Git Bash as a workaround. X509Store is in the runtime rather than
205+
in a module, so it does not depend on which shell started which. O200.
173206
"""
174207
script = (
175208
"$out = @(); "
176-
"Get-ChildItem Cert:\\CurrentUser\\My, Cert:\\LocalMachine\\My "
177-
"-ErrorAction SilentlyContinue | Where-Object { "
178-
" $_.Extensions.EnhancedKeyUsages.Value -contains '%s' } | ForEach-Object { "
179-
" $h = [System.Security.Cryptography.SHA256]::Create().ComputeHash($_.RawData); "
180-
" $out += [pscustomobject]@{ "
181-
" sha256 = (($h | ForEach-Object { $_.ToString('x2') }) -join ''); "
182-
" thumb = $_.Thumbprint; subject = $_.Subject; "
183-
" notAfter = $_.NotAfter.ToString('s') } "
209+
"foreach ($where in 'CurrentUser', 'LocalMachine') { "
210+
" $store = New-Object System.Security.Cryptography.X509Certificates.X509Store('My', $where); "
211+
" try { $store.Open('ReadOnly') } catch { continue }; "
212+
" foreach ($c in $store.Certificates) { "
213+
" $eku = @(); "
214+
" foreach ($x in $c.Extensions) { "
215+
" if ($x -is [System.Security.Cryptography.X509Certificates.X509EnhancedKeyUsageExtension]) { "
216+
" foreach ($u in $x.EnhancedKeyUsages) { $eku += $u.Value } } }; "
217+
" if ($eku -notcontains '%s') { continue }; "
218+
" $h = [System.Security.Cryptography.SHA256]::Create().ComputeHash($c.RawData); "
219+
" $out += [pscustomobject]@{ "
220+
" sha256 = (($h | ForEach-Object { $_.ToString('x2') }) -join ''); "
221+
" thumb = $c.Thumbprint; subject = $c.Subject; "
222+
" notAfter = $c.NotAfter.ToString('s') } }; "
223+
" $store.Close() "
184224
"}; $out | ConvertTo-Json -Compress" % CODE_SIGNING_OID
185225
)
186226
entries = json.loads(powershell(script).strip() or "[]")

‎CHANGELOG.md‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,33 @@ because it turns other people's test suites red.
1414

1515
## [Unreleased]
1616

17+
### Fixed
18+
19+
- **Asking for damaged files and declaring they will be accepted is now refused
20+
on the command line too.** A damaged file is one a reader was measured to
21+
refuse, so `--expected accept` beside `--damage` asks for something nothing
22+
can deliver.
23+
24+
A recipe saying the same thing has always been refused. The command line was
25+
not: it wrote the files and recorded in the manifest that a deliberately
26+
broken file should be accepted, which is the one place this tool must not say
27+
something untrue. `tfg generate --damage zero-head --expected accept` now
28+
ends with exit code `2` and writes nothing, and a recipe still ends with `3`
29+
and names the target the problem is in.
30+
31+
Only `accept` is refused. `reject` is what damage already means, and
32+
`sanitize` and `unspecified` are both real questions to ask about a broken
33+
file - a system under test may be meant to repair it, or that may be the
34+
point of the test - so all three still work, as does `--expected accept` on
35+
files that are not damaged.
36+
37+
- **The documentation website lists every command the tool has.** `tfg damage`
38+
arrived in 0.3.0 and the page describing the commands still showed the other
39+
nine, because that list was written out by hand. The page now takes the list
40+
from the program itself, so a command added later cannot go missing from it,
41+
and the site has a section explaining how to produce a file that is broken on
42+
purpose.
43+
1744
## [0.3.0] - 2026-09-09
1845

1946
### Breaking

‎README.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -262,6 +262,7 @@ tfg generate --format txt --size 1mb settings come from the flags
262262
| `--out <dir>` | directory to write into. Default `.` |
263263
| `--seed <n>` | run seed. The same seed gives the same bytes |
264264
| `--set <k>=<v>` | a format setting, repeatable: `--set width=1920 --set height=1080` |
265+
| `--damage <name>` | break the files on purpose, repeatable and applied in order. Run `tfg damage` for the list |
265266
| `--expected <outcome>` | `accept`, `reject`, `sanitize` or `unspecified` |
266267
| `--expected-reason <r>` | why that outcome, from the closed list below |
267268
| `--preset <id>` | build the set a named test question calls for |
@@ -350,6 +351,11 @@ The smallest file a damage can be given follows its settings, so the column is
350351
measured with the defaults. Ask for less and the run is refused before anything is
351352
written, naming a size that would work.
352353

354+
Asking for `--expected accept` beside a damage is refused too, because nothing
355+
could meet it. Write `sanitize` if the system under test is meant to repair the
356+
file, or `unspecified` if that is the question you are asking - both of those,
357+
and `reject`, work as they always did.
358+
353359
## 📜 Recipes
354360

355361
A recipe is a YAML file describing a whole run. Commit it beside your tests and

‎internal/cli/errors.go‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -217,6 +217,19 @@ func classifyRequest(err error) (int, bool) {
217217
if errors.As(err, &tooSmall) {
218218
return ExitFormat, true
219219
}
220+
// Damaging a file and declaring it will be accepted is two flags that
221+
// cancel each other, which is a fault in the invocation rather than a
222+
// request no format can meet - the same conflict stands for all of them.
223+
// A script reading ExitFormat goes looking for another format or another
224+
// size, and neither of those is the fix. Owner's call on 2026-09-09.
225+
//
226+
// Anything arriving here came off the command line: a recipe declaring the
227+
// same pair is refused while the recipe is read, with the address of the
228+
// target and code 3 beside its other problems. O199.
229+
var impossible *damage.ExpectationConflictError
230+
if errors.As(err, &impossible) {
231+
return ExitUsage, true
232+
}
220233
if code, ok := classifyFormat(err); ok {
221234
return code, true
222235
}

‎internal/damage/refusals.go‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -114,6 +114,52 @@ func (e *NoChangeError) Error() string {
114114
return e.What() + ". " + e.Why()
115115
}
116116

117+
// RuledOutExpectation is the one declared outcome a damaged file cannot have.
118+
//
119+
// It is spelled here rather than imported because internal/manifest sits
120+
// beside this package rather than under it, so the two cannot see each other.
121+
// TestTheOutcomeDamageRulesOutIsTheOneTheManifestKnows compares this against
122+
// manifest.OutcomeAccept and against the list a recipe accepts, which is what
123+
// stops three spellings of one word from drifting apart.
124+
const RuledOutExpectation = "accept"
125+
126+
// ConflictsWithExpectation is the refusal a target earns by damaging its files
127+
// and declaring they will be accepted, or nil when there is no conflict.
128+
//
129+
// It lives here, on the chain, because it is a fact about damage rather than
130+
// about either surface - and both surfaces ask it. Measured on 2026-09-09 with
131+
// the check living in the recipe reader alone: a recipe was refused with code
132+
// 3 while the identical run off the command line ended with code 0 and wrote a
133+
// manifest saying a deliberately broken file should be accepted. The recipe
134+
// reader still asks first, so it keeps reporting this beside every other
135+
// problem of that recipe and with the address of the target - what changed is
136+
// that the engine asks too, so no surface can get past it. See O199.
137+
// Two shapes of one rule, and the pair is deliberate. The engine wants an
138+
// error to hand upwards, while the recipe reader wants the parts - What, Why
139+
// and Instead - to lay out beside the other problems of that recipe.
140+
//
141+
// Written as two functions rather than one returning the concrete type,
142+
// because that one would be the typed nil trap: a nil *ExpectationConflictError
143+
// placed in an error interface is NOT a nil error. Measured 2026-09-09 on a
144+
// four case program - "reject", "sanitize", "" and "accept" all came back
145+
// err != nil - so the engine would have refused EVERY target, damaged or not,
146+
// and the guard beside this one asserts exactly that it does not.
147+
func (c Chain) ConflictsWithExpectation(expected string) error {
148+
if bad := c.ExpectationConflict(expected); bad != nil {
149+
return bad
150+
}
151+
return nil
152+
}
153+
154+
// ExpectationConflict is the same question answered with the refusal itself,
155+
// or nil. For a caller that needs the parts rather than an error.
156+
func (c Chain) ExpectationConflict(expected string) *ExpectationConflictError {
157+
if len(c) == 0 || expected != RuledOutExpectation {
158+
return nil
159+
}
160+
return &ExpectationConflictError{Outcome: expected}
161+
}
162+
117163
// ExpectationConflictError is a target that damages a file and expects it to
118164
// be accepted.
119165
//

‎internal/engine/damage.go‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,24 @@ import (
1313
"github.com/donislawdev/TestingFilesGenerator/internal/manifest"
1414
)
1515

16+
// checkDamage is every refusal a damaged target can earn during planning.
17+
//
18+
// One entry rather than two calls side by side in engine.go, and that is the
19+
// same measurement this file was cut out for: adding the second call put
20+
// engine.go at 411 lines of code against a ceiling of 408, and the answer to a
21+
// ceiling is a cut rather than a larger number. These two belong together
22+
// anyway - both are "what makes this target impossible before a byte is
23+
// written", which is one subject.
24+
//
25+
// The floor first, because it names a number a person can act on. A target
26+
// earning both refusals gets that one.
27+
func checkDamage(t *Target) error {
28+
if err := checkDamageFloor(t); err != nil {
29+
return err
30+
}
31+
return checkDamageExpectation(t)
32+
}
33+
1634
// checkDamageFloor refuses a file smaller than the damage it was given.
1735
//
1836
// Here rather than at the moment of writing, and that is the point: a file
@@ -46,6 +64,24 @@ func checkDamageFloor(t *Target) error {
4664
return nil
4765
}
4866

67+
// checkDamageExpectation refuses a target that breaks its files and declares
68+
// they will be accepted.
69+
//
70+
// Here as well as in the recipe reader, and that is the whole point of it. The
71+
// condition is one function on the chain, so this is a second CALLER rather
72+
// than a second copy - what it buys is that the command line reaches it, and
73+
// the command line never reads a recipe. Measured on 2026-09-09 before this
74+
// existed: the recipe was refused with code 3 while
75+
// --damage zero-head --expected accept ended with code 0 and wrote a manifest
76+
// claiming a deliberately broken file should be accepted. O199.
77+
//
78+
// The window cannot reach this today - damage sits on the generate screen and
79+
// the expectation on the recipe screen - and being here rather than in the
80+
// reader is what covers it on the day those two meet.
81+
func checkDamageExpectation(t *Target) error {
82+
return t.Damage.ConflictsWithExpectation(t.Expected)
83+
}
84+
4985
// damageFor records what was broken about this file, with the settings
5086
// resolved rather than as written.
5187
//

‎internal/engine/engine.go‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -278,7 +278,7 @@ func settleTarget(t *Target, opt Options, seen map[string]bool) (format.Descript
278278
// After the draw, because a range arrives here carrying only a count and a
279279
// damage floor has to be judged against the sizes that will really be
280280
// written.
281-
if err := checkDamageFloor(t); err != nil {
281+
if err := checkDamage(t); err != nil {
282282
return format.Descriptor{}, err
283283
}
284284
return desc, nil

‎internal/guard/damagerefused_test.go‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,16 @@
11
package guard
22

33
import (
4+
"bytes"
5+
"context"
46
"errors"
57
"os"
68
"path/filepath"
9+
"slices"
710
"strings"
811
"testing"
912

13+
"github.com/donislawdev/TestingFilesGenerator/internal/cli"
1014
"github.com/donislawdev/TestingFilesGenerator/internal/damage"
1115
"github.com/donislawdev/TestingFilesGenerator/internal/engine"
1216
"github.com/donislawdev/TestingFilesGenerator/internal/format"
@@ -313,6 +317,91 @@ targets:
313317
}
314318
}
315319

320+
// The command line refuses that pair as well, and writes nothing.
321+
//
322+
// The guard above asks recipe.Parse and only recipe.Parse, and that was enough
323+
// to be green through a build where this was broken. Measured on 2026-09-09 on
324+
// the 0.3.0 binary: the recipe was refused with code 3 while
325+
// --damage zero-head --expected accept ended with code 0 and left a manifest
326+
// on disk saying a deliberately damaged file should be accepted - the tool
327+
// lying in the one place its value lives. O199.
328+
//
329+
// Both halves are needed. Asking only the refusal would pass for a build that
330+
// refuses damage beside any expectation at all, and asking only that files are
331+
// written would pass for one that refuses nothing - so the second loop is the
332+
// wall detector and the first is the hole detector.
333+
//
334+
// The count of files is asked rather than the exit code alone: a refusal that
335+
// arrives after the writing has started is a refusal that came too late, and
336+
// the code by itself cannot tell those apart.
337+
func TestTheCommandLineRefusesDamageBesideAcceptToo(t *testing.T) {
338+
run := func(t *testing.T, extra ...string) (int, string, int) {
339+
t.Helper()
340+
dir := t.TempDir()
341+
var out, errOut bytes.Buffer
342+
args := append([]string{
343+
"generate", "--format", "txt", "--size", "100",
344+
"--damage", damage.ZeroHead, "--out", dir,
345+
}, extra...)
346+
code := cli.Run(context.Background(), args, &out, &errOut)
347+
written, err := os.ReadDir(dir)
348+
if err != nil {
349+
t.Fatalf("reading the output directory: %v", err)
350+
}
351+
return code, errOut.String(), len(written)
352+
}
353+
354+
code, said, files := run(t, "--expected", damage.RuledOutExpectation)
355+
if code != cli.ExitUsage {
356+
t.Errorf("damage beside %q ended with %d, expected %d - two flags that cancel each other are a fault in the invocation\nstderr: %s",
357+
damage.RuledOutExpectation, code, cli.ExitUsage, said)
358+
}
359+
if files != 0 {
360+
t.Errorf("the run was refused and still left %d file(s) behind", files)
361+
}
362+
if !strings.Contains(said, damage.RuledOutExpectation) {
363+
t.Errorf("the refusal does not name the word it turned down: %s", said)
364+
}
365+
366+
// The other three are legitimate questions about a broken file, and a
367+
// build refusing them would be a wall rather than this rule.
368+
for _, outcome := range []string{"reject", "sanitize", "unspecified"} {
369+
if code, said, _ := run(t, "--expected", outcome); code != cli.ExitOK {
370+
t.Errorf("--expected %s beside damage ended with %d rather than %d: %s",
371+
outcome, code, cli.ExitOK, said)
372+
}
373+
}
374+
// And the expectation on its own is untouched. Every case above carries a
375+
// damage, so a build that refused accept for every run whatsoever would
376+
// look correct from all of them.
377+
dir := t.TempDir()
378+
var out, errOut bytes.Buffer
379+
if code := cli.Run(context.Background(), []string{
380+
"generate", "--format", "txt", "--size", "100",
381+
"--expected", damage.RuledOutExpectation, "--out", dir,
382+
}, &out, &errOut); code != cli.ExitOK {
383+
t.Errorf("--expected %s with nothing damaged ended with %d rather than %d, which makes this a wall rather than a rule about damage: %s",
384+
damage.RuledOutExpectation, code, cli.ExitOK, errOut.String())
385+
}
386+
}
387+
388+
// The outcome damage rules out is the one the manifest and the recipe know.
389+
//
390+
// Three spellings of one word live in three packages that cannot import each
391+
// other - damage sits beside manifest rather than under it - so this compares
392+
// them rather than leaving them to drift. A rename in one place turns this red
393+
// instead of quietly producing a build where nothing is ever refused.
394+
func TestTheOutcomeDamageRulesOutIsTheOneTheManifestKnows(t *testing.T) {
395+
if damage.RuledOutExpectation != manifest.OutcomeAccept {
396+
t.Errorf("damage rules out %q and the manifest calls it %q, so nothing would ever match",
397+
damage.RuledOutExpectation, manifest.OutcomeAccept)
398+
}
399+
if !slices.Contains(recipe.Outcomes(), damage.RuledOutExpectation) {
400+
t.Errorf("damage rules out %q and a recipe does not accept that word at all: %v",
401+
damage.RuledOutExpectation, recipe.Outcomes())
402+
}
403+
}
404+
316405
// A damage the build does not know is refused while the recipe is read, and
317406
// the refusal names what there is.
318407
//

0 commit comments

Comments
 (0)