Skip to content

Commit 96aa4af

Browse files
donislawdevclaude
andcommitted
format: JPEG XL is the twenty fourth format, and its encoder is frugal with nothing
Adds jxl: one frame, 8 bit, RGB, written as a JPEG XL codestream inside the container the format defines for it. width, height and quality are settings, the picture goes up to 40 megapixels, and left alone it is the largest of a fixed ladder that fits the size asked for, up to 640x480 - the same top rung JPG and AVIF use, so the picture formats answer one request with one size of picture. Every size from the minimum of 147 B upwards is reachable, a byte at a time. The padding travels in a free box, which is the box the container sets aside for space that means nothing. The second format here whose pixels are coded by somebody else's encoder. The road AVIF surveyed was walked again rather than assumed, and three of the answers were not the ones the survey expected. The encoder emits a BARE codestream and cannot write a container at all - it reads one. So the container is ours, built around its bytes. Both channels were measured and both work: trailing bytes after a bare codestream are taken by every reader, down to a single byte, with no dead zone. They are not used. Trailing bytes are not a structure the format defines, only something readers tolerate, and a free box says "nothing here" in the format's own words. The choice costs 48 B of minimum and is written down where the measurement is. Two readers rather than one, and that is what caught the defect worth catching: a file type box with no compatible brand is REFUSED by libjxl and accepted by the pure Go decoder. Had the only witness been the module we build against, that file would have left the tool and failed on somebody else's machine. Both readers refuse a truncated file and a corrupted one, so both are witnesses. exiftool reads this format and accepts both, so it is not one. The encoder is expensive where AVIF's is cheap: about 618 000 objects for one 640x480 picture against about a hundred. A flat ceiling of 128 objects cannot describe both, and raising it would stop it saying anything about the other twenty three, so a format may now declare its own. The check that ceiling stands in for - whether allocation grows with the size of the FILE rather than with the picture - is untouched and still applies to every format. Its tolerance now scales with what a format allocates, which is zero change for every format that allocates less than a thousand objects, and it was proved still to catch the real defect: a write buffer moved inside its loop grows by 2018 against an allowance of 634. Measured and worth keeping: - the assembly is clean, unlike AVIF's. 240 picture sizes, three runs, no crash, and identical bytes with it and without. - the same bytes on Windows amd64, Linux amd64, emulated Linux arm64 and macOS 26.6.2 on arm64, over five cases including an odd size and the minimum. Thread count does not move them either, at 1 through 16 - which mattered, because the library's default is GOMAXPROCS. - the library's default effort is both slower and larger for these pictures, so the lowest is used. It does not generalise and the exception is recorded. - the ladder's SHAPE depended on how many seeds were swept. At four the one pixel rung came out larger than the eight by six one, which would have left rungs unreachable and the fallback wrong. The ceilings are from 256. - planning does not code the picture: fifty plans allocate 39 kB against 19.8 MB for one write. - no socket in the command line binary. It grows by 1 249 280 B, the window by 1 629 067 B. Opened in XnView MP, which reports 640x480x24 and 300.00 KiB and draws the label, so the fidelity checklist row is filled in from a screen rather than from a decoder. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 2392c40 commit 96aa4af

41 files changed

Lines changed: 1417 additions & 102 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎CHANGELOG.md‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,24 @@ because it turns other people's test suites red.
3434

3535
### Added
3636

37+
- **JPEG XL, the twenty fourth format.** One frame, 8 bit, RGB.
38+
`tfg generate --format jxl --size 300kb` writes a JPEG XL picture in the
39+
container the format defines for it. `width`, `height` and `quality` can be
40+
set, and the picture goes up to 40 megapixels, so Full HD and 4K are both in
41+
reach. Left alone, the picture is the largest of a fixed set that fits the
42+
size asked for, up to 640x480 - the same as JPG and AVIF, so the picture
43+
formats answer the same request with the same sized picture.
44+
45+
**Every size from its minimum of 147 B upwards is reachable, with no gaps.**
46+
The padding travels in a `free` box, which is the box the container sets aside
47+
for space that means nothing, and it takes any length at all.
48+
49+
The second format here whose pixels are coded by somebody else's encoder. It
50+
is pinned, so raising it is a breaking change like any other, and it is pure
51+
Go: no C compiler, no shared library and no socket. The files were read back
52+
by two independent decoders, one of them libjxl, and both refuse a file that
53+
has been truncated or corrupted.
54+
3755
- **AVIF, the twenty third format.** One frame, 8 bit, 4:2:0.
3856
`tfg generate --format avif --size 300kb` writes an AV1 picture in an ISO base
3957
media container. `width`, `height` and `quality` can be set, and the picture

‎README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010

1111
**Testing Files Generator** is a tool for QA engineers and developers who need real
1212
files to test against - an upload form, a parser, anything that takes a file and
13-
has an opinion about it. You pick one of its 23 formats and the size you want,
13+
has an opinion about it. You pick one of its 24 formats and the size you want,
1414
and you get **exactly that**: ask for a 10 MB PDF and you get a PDF that a reader
1515
will open, at 10 MB to the byte. Every run also leaves a manifest saying **what
1616
your system should do with each file**, which is the part other generators leave
@@ -23,7 +23,7 @@ needs it finds out it exists.
2323

2424
- **Hit an exact size, to the byte** - ask for 10485761 bytes and get exactly
2525
that, never a silently rounded file.
26-
- **Write 23 real formats** - a generated PNG opens in an image viewer, a DOCX
26+
- **Write 24 real formats** - a generated PNG opens in an image viewer, a DOCX
2727
opens in Word, a ZIP extracts. Not padded zeros with an extension.
2828
- **Say what should happen to each file** - the manifest carries an expected
2929
outcome, so your test reads the assertion instead of you writing it out.

‎THIRD-PARTY-NOTICES.md‎

Lines changed: 47 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,14 +17,14 @@ first, so the difference is worth stating rather than leaving to be assumed.
1717

1818
| binary | what it is | third party code in it |
1919
|---|---|---|
20-
| `tfg` | the command line | the Go runtime, and **three** modules: `github.com/goccy/go-yaml`, `github.com/gen2brain/gav1d` and `golang.org/x/text` |
20+
| `tfg` | the command line | the Go runtime, and **four** modules: `github.com/goccy/go-yaml`, `github.com/gen2brain/gav1d`, `github.com/gen2brain/jxl` and `golang.org/x/text` |
2121
| `tfg-gui` | the desktop window | the same, plus **27** more for the graphics toolkit, one of them on Linux only |
2222

2323
The window is a separate binary because its toolkit needs a C compiler and
2424
OpenGL, neither of which the command line uses. A server or a build agent
2525
running `tfg` therefore carries none of the 27, and that is checked rather than
2626
asserted: a guard in the source compares what the command line binary actually
27-
links against that list of three.
27+
links against that list of four.
2828

2929
---
3030

@@ -159,6 +159,51 @@ Source: <https://github.com/gen2brain/gav1d>
159159

160160
---
161161

162+
## github.com/gen2brain/jxl
163+
164+
Version 0.2.0. Codes JPEG XL. It is written in Go and assembly with no C in
165+
it and no module of its own behind it, so it brings nothing else along.
166+
167+
```
168+
Copyright (c) the JPEG XL Project Authors.
169+
All rights reserved.
170+
171+
Redistribution and use in source and binary forms, with or without
172+
modification, are permitted provided that the following conditions are met:
173+
174+
1. Redistributions of source code must retain the above copyright notice, this
175+
list of conditions and the following disclaimer.
176+
177+
2. Redistributions in binary form must reproduce the above copyright notice,
178+
this list of conditions and the following disclaimer in the documentation
179+
and/or other materials provided with the distribution.
180+
181+
3. Neither the name of the copyright holder nor the names of its
182+
contributors may be used to endorse or promote products derived from
183+
this software without specific prior written permission.
184+
185+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
186+
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
187+
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
188+
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
189+
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
190+
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
191+
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
192+
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
193+
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
194+
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
195+
```
196+
197+
It also ships a `PATENTS` file carrying a patent grant from Google: a
198+
perpetual, worldwide, non-exclusive, no-charge, royalty-free licence to the
199+
Google patents necessarily infringed by this implementation of JPEG XL,
200+
withdrawn from anybody who sues over it. The full text is in the module, at
201+
`PATENTS`.
202+
203+
Source: <https://github.com/gen2brain/jxl>
204+
205+
---
206+
162207
## The window binary only
163208

164209
These 26 modules are the graphics toolkit and what it brings with it. They are

‎go.mod‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@ require github.com/goccy/go-yaml v1.19.2
3434

3535
require (
3636
github.com/gen2brain/gav1d v0.2.5
37+
github.com/gen2brain/jxl v0.2.0
3738
golang.org/x/text v0.41.0
3839
)
3940

‎go.sum‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,8 @@ github.com/fyne-io/oksvg v0.2.0 h1:mxcGU2dx6nwjJsSA9PCYZDuoAcsZ/OuJlvg/Q9Njfo8=
2424
github.com/fyne-io/oksvg v0.2.0/go.mod h1:dJ9oEkPiWhnTFNCmRgEze+YNprJF7YRbpjgpWS4kzoI=
2525
github.com/gen2brain/gav1d v0.2.5 h1:Zg5/DE1JBdf8kxZqIvNo2ZEizF9DVF9jdPTAKJQZbOY=
2626
github.com/gen2brain/gav1d v0.2.5/go.mod h1:ReFKjHq7pvRhEk+pgRPAo9gOlcrf51sdzVcAbbX7e0E=
27+
github.com/gen2brain/jxl v0.2.0 h1:cXSQnfbcC0JGFzaI344VzPMbe7mh6rYc0tjPcBUsE4U=
28+
github.com/gen2brain/jxl v0.2.0/go.mod h1:VjJHRai/8I8Es01mULqyiOihRBOjJr1Zo+NoN/MyIwo=
2729
github.com/go-gl/gl v0.0.0-20260331235117-4566fea9a276 h1:IO5P06Pcj9K04d+l4nrf3c2U56+dAotIFG6u4P1wAHI=
2830
github.com/go-gl/gl v0.0.0-20260331235117-4566fea9a276/go.mod h1:9YTyiznxEY1fVinfM7RvRcjRHbw2xLBJ3AAGIT0I4Nw=
2931
github.com/go-gl/glfw/v3.4/glfw v0.1.0-pre.1.0.20260707082822-2a407d02d01a h1:HWK0MBggT/T6YH7VffE10xBIhqeTq8JzIUPJXrRy87g=

‎internal/format/all/all.go‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ import (
1717
_ "github.com/donislawdev/TestingFilesGenerator/internal/format/ico"
1818
_ "github.com/donislawdev/TestingFilesGenerator/internal/format/jpg"
1919
_ "github.com/donislawdev/TestingFilesGenerator/internal/format/jsonfile"
20+
_ "github.com/donislawdev/TestingFilesGenerator/internal/format/jxl"
2021
_ "github.com/donislawdev/TestingFilesGenerator/internal/format/logfile"
2122
_ "github.com/donislawdev/TestingFilesGenerator/internal/format/md"
2223
_ "github.com/donislawdev/TestingFilesGenerator/internal/format/pdf"

‎internal/format/format.go‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -257,6 +257,26 @@ type Descriptor struct {
257257
// its own. Empty for every format that has none.
258258
JointLimits []JointLimit
259259

260+
// AllocCeiling is how many objects this format may allocate producing one
261+
// file, when the flat ceiling every other one meets does not describe it.
262+
// Zero means the flat one applies, which is the case for all but one.
263+
//
264+
// It exists because a borrowed encoder allocates on its own account. The
265+
// hand written generators here sit between 3 and 128 objects a file, and
266+
// gav1d, the AVIF encoder, sits at about a hundred - but gen2brain/jxl
267+
// allocates per block, about 618 000 of them for one 640x480 picture. A
268+
// single ceiling has to fit the heaviest format, so one that fits that one
269+
// would say nothing about the other twenty three.
270+
//
271+
// What the ceiling stands in for is untouched by this: the guard also asks
272+
// each format whether its allocation GROWS with the size of the file
273+
// asked for, and that question is the real one. Every format answers it,
274+
// this one included. Owner's decision, 2026-08-31.
275+
//
276+
// A ratchet, like the coverage threshold and the code shape ceilings: it
277+
// goes down when work makes it lowerable, never up to turn a run green.
278+
AllocCeiling int64
279+
260280
// Container says this format holds other files, so a recipe may declare
261281
// contains for it.
262282
//

‎internal/format/jxl/codec.go‎

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
// The codec: the single call that turns pixels into a JPEG XL codestream.
2+
package jxl
3+
4+
import (
5+
"bytes"
6+
"fmt"
7+
"image"
8+
9+
"github.com/gen2brain/jxl"
10+
)
11+
12+
const (
13+
// encodeEffort is how hard the encoder searches, and it is set to the
14+
// lowest setting on a measurement rather than on taste.
15+
//
16+
// The library's own default is 7, the highest it implements. For the
17+
// picture this generator builds - a gradient with a label on it - that
18+
// setting is both slower AND larger, which is the opposite of what the
19+
// knob is for. Measured on 2026-08-31, five interleaved rounds, ranges
20+
// that do not overlap: at 320x240 effort 1 takes 25 ms against 63 ms, at
21+
// 640x480 it takes 77 ms against 233 ms. Asked across sixteen seeds at
22+
// 320x240, effort 1 produced the smaller file in fourteen of them.
23+
//
24+
// Worth writing down because it does not generalise: at 80x60 effort 7 is
25+
// the smaller file, 240 B against 286 B. Search pays off on a picture with
26+
// few blocks to search. Ours are mostly not that.
27+
encodeEffort = 1
28+
29+
// encodeThreads keeps one frame on one goroutine.
30+
//
31+
// The library's default is GOMAXPROCS, and the bytes were measured
32+
// identical at 1, 2, 3, 4, 8 and 16 threads, so this is not fixing a
33+
// determinism bug that exists. It is removing a machine dependent input
34+
// from a component whose output is the contract (D11), which is cheaper
35+
// than trusting that it will stay harmless.
36+
//
37+
// The price is measured and real, and it is paid at the top of the ladder
38+
// only: 640x480 takes 77 ms here against 43 ms with every core, while
39+
// 320x240 takes 25.5 ms against 22.3 ms. One frame per goroutine also
40+
// halves what the encode allocates.
41+
encodeThreads = 1
42+
43+
minQuality = 1
44+
maxQuality = 100
45+
46+
// defaultQuality is the library's own default, kept because it is the
47+
// setting the ladder ceilings below were measured at.
48+
defaultQuality = 90
49+
)
50+
51+
// encode runs the picture through the encoder and hands back the codestream.
52+
//
53+
// What comes back is the bare codestream, starting FF 0A, not a container -
54+
// measured, rather than taken from the encoder's documentation. The container
55+
// this format writes is built in jxl.go, around these bytes.
56+
func encode(m image.Image, quality int) ([]byte, error) {
57+
var buf bytes.Buffer
58+
err := jxl.Encode(&buf, m, jxl.EncodeOptions{
59+
Quality: quality,
60+
Effort: encodeEffort,
61+
Threads: encodeThreads,
62+
})
63+
if err != nil {
64+
return nil, fmt.Errorf("jxl: the encoder could not code the picture: %w", err)
65+
}
66+
return buf.Bytes(), nil
67+
}

0 commit comments

Comments
 (0)