Skip to content

Commit e3dc83e

Browse files
donislawdevclaude
andcommitted
format: a zip can be locked with ZipCrypto, which is the lock worth having for what it breaks
tfg generate --format zip --size 30kb --set entries=3 \ --set password=Secret123 --set encryption=zipcrypto It is here for what it does to a reader rather than for what it protects. Measured on our own output: .NET's ZipFile opens one of these, reports the entry at its true length of 8192, hands back a stream and fills it with ff c7 04 3e where the file holds "tfg - txt". It never says the entry was encrypted at all, so an application built on that library processes noise and calls it data. AES in the same library throws, which is the safer defect and the less interesting one. This is the fixture PRESETS.md section 4.12 has been describing since before there was anything to build it with. The cipher is not ours and it was not taken on trust. It went into tools/probes/zipcrypto BEFORE it went into the generator, pointed at an archive 7-Zip had written, and asked to decrypt it - check byte, plaintext CRC and the bytes themselves all came back right. That order was chosen because of what the AES work had shown a few hours earlier: a mutation proved the archiver guard could not see a keystream running backwards, since an AE-2 entry signs its ciphertext rather than its contents, so a file of noise passes every check a reader makes. A defect got through anyway, and the probe is what diagnosed it. Every locked entry declared method 99 - WinZip AES - while a ZipCrypto entry has to stay stored, because it changes nothing about how the bytes sit and only puts twelve in front of them. 7-Zip reported "Data Error in encrypted file. Wrong password?" which points at the password, the one thing that was right. The probe decrypted the same file perfectly, and that is what moved the suspicion off the cryptography and onto the header. So there is a guard for the header shape now, and it asks both schemes in both directions. ZipCrypto stores, carries the real plaintext CRC and no extra field. AES declares 99, carries a CRC of nought and a 0x9901 field. Getting either backwards produces a file that opens and then fails on something that sounds like the user's fault. The cost is named rather than hidden: ZipCrypto puts the high byte of the plaintext CRC in its header, so the contents have to be known before the first byte of the entry goes out - and the contents arrive as a stream. Each entry is therefore generated twice, once to be checksummed and once to be encrypted. Twice the processor and not a byte more memory, because buffering the entry would break the guard that says a generator does not hold a whole file. Three guards stopped carrying a hand written list of methods and read the registry instead. A list in a test is one somebody has to remember on the day a fourth scheme arrives, and this was that day. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 27ee61b commit e3dc83e

8 files changed

Lines changed: 364 additions & 35 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,14 @@ because it turns other people's test suites red.
7070

7171
### Added
7272

73+
- **A zip can be locked with ZipCrypto, the old scheme.** `--set encryption=zipcrypto`.
74+
75+
It is here for what it does to a reader rather than for what it protects. Measured: .NET's own `ZipFile` opens one of these, reports the entry at its true length, hands back a stream and fills it with the ENCRYPTED bytes - and never says the entry was encrypted at all. An application built on that library processes noise and calls it data. AES fails loudly in the same library, which is the safer defect and the less interesting one.
76+
77+
So this is the fixture for finding out whether something in a pipeline waves an encrypted archive through.
78+
79+
**It is not protection and it is not offered as any.** ZipCrypto has been broken for decades. Use `aes-256` when the point is that the contents are hard to read.
80+
7381
- **A zip can be locked with a password.**
7482

7583
tfg generate --format zip --size 30kb --set entries=3 \

‎internal/format/archive/archive.go‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,7 @@ const (
6161
// The encryption methods, spelled the way a recipe writes them.
6262
const (
6363
NoEncryption = "none"
64+
ZipCrypto = "zipcrypto"
6465
AES128 = "aes-128"
6566
AES192 = "aes-192"
6667
AES256 = "aes-256"
@@ -162,10 +163,11 @@ var axes = map[string]format.Property{
162163
Encryption: {
163164
Name: Encryption, Kind: format.PropertyChoice,
164165
// Sorted, because a closed set has one order on every surface.
165-
Choices: []string{AES128, AES192, AES256, NoEncryption},
166+
Choices: []string{AES128, AES192, AES256, NoEncryption, ZipCrypto},
166167
Default: NoEncryption,
167-
Detail: "How the archive is locked. This is the WinZip AES scheme, which 7-Zip and WinZip open " +
168-
"and some other readers cannot open at all - .NET lists the files and then fails on reading one.",
168+
Detail: "How the archive is locked. AES is the WinZip scheme, and some readers cannot open it " +
169+
"at all. ZipCrypto is the old one every reader opens and nothing modern trusts, and some " +
170+
"of them hand back the encrypted bytes without saying so.",
169171
},
170172
}
171173

‎internal/format/archive/lock.go‎

Lines changed: 50 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,12 @@ const (
4141
authLen = 10
4242
iterations = 1000
4343

44+
// zipStore is a stored entry, and winZipAES is the number an AES entry
45+
// declares instead. The second is not a compression at all: the real method
46+
// sits in the 0x9901 field beside it, and this build always stores.
47+
zipStore = 0
48+
winZipAES = 99
49+
4450
// aesExtraLen is the 0x9901 field: two bytes of id, two of length and
4551
// seven of body. It is written into the local header AND the central
4652
// directory, so an entry pays for it twice.
@@ -64,6 +70,22 @@ type Lock struct {
6470
// On says whether anything is encrypted.
6571
func (l Lock) On() bool { return l.Method != "" && l.Method != NoEncryption }
6672

73+
// NeedsPlaintextCRC says whether an entry cannot be started until its
74+
// contents are known.
75+
//
76+
// True for ZipCrypto and false for AES, and the difference is measured rather
77+
// than assumed. Read out of what 7-Zip writes: a ZipCrypto entry carries the
78+
// real CRC of the plaintext and puts its high byte in the header, so a reader
79+
// can reject a wrong password without decrypting anything. An AE-2 entry
80+
// carries a CRC of zero, because its authentication code does that job.
81+
//
82+
// The cost falls on the caller and it is real: an entry that needs this is
83+
// generated twice, once to be counted and once to be encrypted. Twice the
84+
// processor and not a byte more memory, which is the trade this project takes
85+
// every time - holding the file to hash it would break the guard that says a
86+
// generator does not.
87+
func (l Lock) NeedsPlaintextCRC() bool { return l.Method == ZipCrypto }
88+
6789
// keyLen is the AES key in bytes, and the salt is half of it. Zero for an
6890
// archive that is not locked with AES.
6991
func (l Lock) keyLen() int {
@@ -98,6 +120,7 @@ func (l Lock) strength() byte {
98120
// Measured, and it agrees from two directions - the size of the whole file and
99121
// the compressed size field in the local header:
100122
//
123+
// ZipCrypto +12 (eleven bytes that vary and one check byte)
101124
// AES-128 +20 (8 salt, 2 verifier, 10 authentication)
102125
// AES-192 +24
103126
// AES-256 +28
@@ -106,12 +129,34 @@ func (l Lock) strength() byte {
106129
// headers are written for real during the counting pass, so the writer counts
107130
// those eleven bytes twice over on its own.
108131
func (l Lock) EntryOverhead() int64 {
109-
if !l.On() {
132+
switch {
133+
case !l.On():
110134
return 0
135+
case l.Method == ZipCrypto:
136+
return zipCryptoHeader
111137
}
112138
return int64(l.saltLen() + pwvLen + authLen)
113139
}
114140

141+
// ZipMethod is the compression method the entry declares. Named for what it
142+
// answers rather than for the field it reads, because Method is that field.
143+
//
144+
// AES entries declare 99, which is not a compression at all - the real
145+
// method sits in the 0x9901 field beside it. ZipCrypto declares what it
146+
// really is, because it changes nothing about how the bytes are stored, it
147+
// only puts twelve bytes in front of them and scrambles what follows.
148+
//
149+
// Getting this wrong is quiet. An entry declaring 99 with no 0x9901 field
150+
// beside it gets past the check byte and fails on the checksum, and 7-Zip
151+
// reports "Data Error in encrypted file. Wrong password?" - which points at
152+
// the password, the one thing that was right.
153+
func (l Lock) ZipMethod() uint16 {
154+
if l.keyLen() == 0 {
155+
return zipStore
156+
}
157+
return winZipAES
158+
}
159+
115160
// Extra is the 0x9901 field an AES entry carries, and nil for anything else.
116161
func (l Lock) Extra() []byte {
117162
if l.keyLen() == 0 {
@@ -185,7 +230,10 @@ func ReadLock(id string, props map[string]string) (Lock, error) {
185230
// would give two runs of one recipe different bytes, which is untouchable rule
186231
// 3 - and the same recipe producing the same file is more of the product here
187232
// than the encryption is.
188-
func (l Lock) NewEntryWriter(w io.Writer, seed uint64, index int) (io.WriteCloser, error) {
233+
func (l Lock) NewEntryWriter(w io.Writer, seed uint64, index int, crc uint32) (io.WriteCloser, error) {
234+
if l.Method == ZipCrypto {
235+
return l.newZipCryptoWriter(w, seed, index, crc)
236+
}
189237
if l.keyLen() == 0 {
190238
return nil, fmt.Errorf("archive: %q is not an encryption this build can write", l.Method)
191239
}
Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
package archive
2+
3+
import (
4+
"hash/crc32"
5+
"io"
6+
7+
"github.com/donislawdev/TestingFilesGenerator/internal/core"
8+
)
9+
10+
// The old ZIP encryption, the one everything opens and nothing modern trusts.
11+
//
12+
// It is here for what it does to a reader rather than for what it protects.
13+
// Measured on 2026-09-01: .NET's own ZipFile opens a ZipCrypto archive, reports
14+
// the entry at its true length, hands back a stream and fills it with
15+
// CIPHERTEXT - 'ea 59 89 8e' where the file holds 'AAAA' - and never says the
16+
// entry was encrypted at all. An application built on that library processes
17+
// noise and calls it data. AES fails loudly in the same library, which is the
18+
// safer defect and the less interesting one.
19+
//
20+
// So this is the fixture that finds a real class of fault, and it is the one
21+
// the presets have been waiting for: PRESETS.md section 4.12 names "an archive
22+
// with a password waved through without warning" as a thing to catch.
23+
//
24+
// The cipher itself is not ours and was not taken on trust. It was written into
25+
// a probe first, pointed at an archive 7-Zip produced, and asked to decrypt it -
26+
// the check byte, the plaintext CRC and the bytes themselves all came back
27+
// right. tools/probes/zipcrypto, and it stays runnable, because a keystream
28+
// that is subtly wrong produces a file some readers still accept.
29+
30+
const (
31+
// The three keys PKWARE starts from and the multiplier it steps them with.
32+
// Constants of the scheme rather than choices of ours.
33+
key0Init = 305419896
34+
key1Init = 591751049
35+
key2Init = 878082192
36+
multiplier = 134775813
37+
38+
// zipCryptoHeader is the twelve bytes that precede an entry's data: eleven
39+
// that vary and one that lets a reader reject a wrong password without
40+
// decrypting anything else. It is the whole of what this scheme adds, which
41+
// is why the overhead is a constant.
42+
zipCryptoHeader = 12
43+
)
44+
45+
var crcTable = crc32.MakeTable(crc32.IEEE)
46+
47+
// pkware is the stream cipher, keyed by a password and then by every byte of
48+
// plaintext that passes through it.
49+
//
50+
// The plaintext updates the keys in both directions, which is why encrypting
51+
// and decrypting are two functions here rather than one - a stream cipher that
52+
// is its own inverse would not need the distinction, and this one is not.
53+
type pkware struct{ k0, k1, k2 uint32 }
54+
55+
func newPKWARE(password string) *pkware {
56+
c := &pkware{k0: key0Init, k1: key1Init, k2: key2Init}
57+
for i := 0; i < len(password); i++ {
58+
c.update(password[i])
59+
}
60+
return c
61+
}
62+
63+
func (c *pkware) update(p byte) {
64+
c.k0 = crcTable[(c.k0^uint32(p))&0xff] ^ (c.k0 >> 8)
65+
c.k1 += c.k0 & 0xff
66+
c.k1 = c.k1*multiplier + 1
67+
c.k2 = crcTable[(c.k2^(c.k1>>24))&0xff] ^ (c.k2 >> 8)
68+
}
69+
70+
func (c *pkware) keyByte() byte {
71+
t := uint16(c.k2|2) & 0xffff
72+
return byte((t * (t ^ 1)) >> 8)
73+
}
74+
75+
func (c *pkware) encrypt(p byte) byte {
76+
x := p ^ c.keyByte()
77+
c.update(p)
78+
return x
79+
}
80+
81+
// zipCryptoWriter encrypts on the way through, header first.
82+
type zipCryptoWriter struct {
83+
out io.Writer
84+
c *pkware
85+
}
86+
87+
// newZipCryptoWriter starts an entry, writing the twelve byte header before
88+
// anything else.
89+
//
90+
// The eleven bytes that vary come from the run seed and never from crypto/rand,
91+
// for the reason every other draw in this tool avoids it: two runs of one
92+
// recipe have to give one file. The twelfth is the high byte of the plaintext
93+
// CRC, and it is why this scheme needs the contents known before the first byte
94+
// goes out - the whole reason a ZipCrypto entry is generated twice.
95+
func (l Lock) newZipCryptoWriter(w io.Writer, seed uint64, index int, crc uint32) (io.WriteCloser, error) {
96+
c := newPKWARE(l.Password)
97+
rng := core.NewRand(core.FileSeed(seed, index))
98+
99+
header := make([]byte, zipCryptoHeader)
100+
for i := range header[:zipCryptoHeader-1] {
101+
header[i] = byte(rng.Uint32())
102+
}
103+
header[zipCryptoHeader-1] = byte(crc >> 24)
104+
for i := range header {
105+
header[i] = c.encrypt(header[i])
106+
}
107+
if _, err := w.Write(header); err != nil {
108+
return nil, err
109+
}
110+
return &zipCryptoWriter{out: w, c: c}, nil
111+
}
112+
113+
func (z *zipCryptoWriter) Write(p []byte) (int, error) {
114+
out := make([]byte, len(p))
115+
for i := range p {
116+
out[i] = z.c.encrypt(p[i])
117+
}
118+
if _, err := z.out.Write(out); err != nil {
119+
return 0, err
120+
}
121+
return len(p), nil
122+
}
123+
124+
// Close has nothing to finish. ZipCrypto signs nothing - the entry's own CRC is
125+
// what a reader checks, after decrypting, if it checks at all.
126+
func (z *zipCryptoWriter) Close() error { return nil }

‎internal/format/zip/zip.go‎

Lines changed: 53 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ import (
99
stdzip "archive/zip"
1010
"context"
1111
"fmt"
12+
"hash/crc32"
1213
"io"
1314
"strings"
1415
"time"
@@ -48,11 +49,6 @@ const (
4849
fillerName = "tfg-padding.bin"
4950

5051
writeChunk = 32 * 1024
51-
52-
// winZipAES is the compression method a locked entry declares. It is
53-
// not a compression at all - the real method sits in the 0x9901 field
54-
// and is store, like everything else here.
55-
winZipAES = 99
5652
)
5753

5854
// A fixed timestamp on every entry. Taking one from the clock would make two
@@ -414,6 +410,11 @@ type entryPlan struct {
414410
index int
415411
// withContents is false during the pass that only measures.
416412
withContents bool
413+
// crc is the checksum of the contents, and it is only ever filled for a
414+
// lock that needs the contents known before the first byte. Zero
415+
// otherwise, which is what an AE-2 entry carries anyway and what the
416+
// counting pass writes into a header nobody reads.
417+
crc uint32
417418
}
418419

419420
// openEntry starts the next entry and gives back what its contents go to.
@@ -447,14 +448,14 @@ func openEntry(zw *stdzip.Writer, m memo, e entryPlan) (io.Writer, func() error,
447448

448449
h := &stdzip.FileHeader{
449450
Name: e.name,
450-
Method: winZipAES,
451+
Method: m.lock.ZipMethod(),
451452
Modified: fixedTime,
452453
Extra: m.lock.Extra(),
453454
}
454455
// Bit 0 says the entry is encrypted. The CRC stays zero because AE-2
455456
// carries none - which is what lets the contents be written in one pass.
456457
h.Flags |= 1
457-
h.CRC32 = 0
458+
h.CRC32 = e.crc
458459
h.CompressedSize64 = uint64(e.plain + m.lock.EntryOverhead())
459460
h.UncompressedSize64 = uint64(e.plain)
460461

@@ -470,13 +471,42 @@ func openEntry(zw *stdzip.Writer, m memo, e entryPlan) (io.Writer, func() error,
470471
// which is what the engine caught when this was written the other way.
471472
return raw, nothingToShut, nil
472473
}
473-
locked, err := m.lock.NewEntryWriter(raw, m.seed, e.index)
474+
locked, err := m.lock.NewEntryWriter(raw, m.seed, e.index, e.crc)
474475
if err != nil {
475476
return nil, nil, err
476477
}
477478
return locked, locked.Close, nil
478479
}
479480

481+
// plaintextCRC is the checksum of what an entry is about to hold, worked out
482+
// by generating it once and throwing the bytes away.
483+
//
484+
// Only ZipCrypto asks for this, and only when the contents are really being
485+
// written. That scheme puts the high byte of the plaintext CRC in the twelve
486+
// byte header it prepends, so a reader can turn away a wrong password without
487+
// decrypting anything - which means the checksum has to be known before the
488+
// first byte of the entry goes out, and the contents arrive as a stream.
489+
//
490+
// Generating twice rather than buffering, and that is the trade taken
491+
// deliberately. Holding the entry to hash it would break the guard that says
492+
// a generator does not keep a whole file in memory, and the second pass is
493+
// free of risk because a generator producing different bytes on two calls in
494+
// one process is itself a guarded impossibility. It costs processor time on
495+
// locked archives and nothing at all on open ones.
496+
func plaintextCRC(ctx context.Context, m memo, withContents bool, write func(io.Writer) error) (uint32, error) {
497+
if !withContents || !m.lock.NeedsPlaintextCRC() {
498+
return 0, nil
499+
}
500+
sum := crc32.NewIEEE()
501+
if err := write(sum); err != nil {
502+
return 0, err
503+
}
504+
if err := ctx.Err(); err != nil {
505+
return 0, err
506+
}
507+
return sum.Sum32(), nil
508+
}
509+
480510
// build writes the archive.
481511
//
482512
// withContents says whether the files inside are actually generated. The
@@ -500,8 +530,14 @@ func build(ctx context.Context, w io.Writer, m memo, withContents bool) error {
500530
default:
501531
}
502532

533+
crc, err := plaintextCRC(ctx, m, withContents, func(w io.Writer) error {
534+
return c.desc.Generator.Write(ctx, w, c.plan)
535+
})
536+
if err != nil {
537+
return fmt.Errorf("zip: the %s file inside could not be checksummed: %w", c.desc.ID, err)
538+
}
503539
entry, shut, err := openEntry(zw, m, entryPlan{
504-
name: c.name, plain: c.plan.Bytes, index: i, withContents: withContents,
540+
name: c.name, plain: c.plan.Bytes, index: i, withContents: withContents, crc: crc,
505541
})
506542
if err != nil {
507543
return err
@@ -520,8 +556,15 @@ func build(ctx context.Context, w io.Writer, m memo, withContents bool) error {
520556
// The filler is locked with everything else. An archive where one entry
521557
// opens without the password and the rest do not is a file nobody
522558
// asked for, and the arithmetic is the same either way.
559+
crc, err := plaintextCRC(ctx, m, withContents, func(w io.Writer) error {
560+
return writeFiller(ctx, w, m.seed, m.fillerSize)
561+
})
562+
if err != nil {
563+
return err
564+
}
523565
entry, shut, err := openEntry(zw, m, entryPlan{
524-
name: fillerName, plain: m.fillerSize, index: len(m.children), withContents: withContents,
566+
name: fillerName, plain: m.fillerSize, index: len(m.children),
567+
withContents: withContents, crc: crc,
525568
})
526569
if err != nil {
527570
return err

0 commit comments

Comments
 (0)