Skip to content

Commit 3bcb429

Browse files
donislawdevclaude
andcommitted
format: add HTML and SVG, and close the text group
Twelve of the twenty five Tier 1 formats now work end to end. What is left is entirely binary - images, media, archives, Office - where the record pattern does not apply and the padding channel has to be measured from scratch again. Both formats fill the same way as the four before them, so the record pattern has stopped being a deviation from the written channel and has become the rule. A comment carries the label or carries nothing. SVG gets the strongest reference tool in this project, and it is the first that does not parse but RENDERS. Inkscape draws the file to a bitmap, Pillow opens it and counts the colours. A drawing that parses and paints nothing passes every other check here and fails that one. Two things about Inkscape were measured before it could be trusted, and both surprised. It exits ZERO on a malformed file and says so only on standard error, so the exit code alone would bless a broken drawing. And it will not write the bitmap to standard output, so the only way to know one exists is to open the file it wrote. Both are why the checker is a script rather than a plain command line. HTML is the opposite case and it is the format's own doing. A parser is required to recover from almost anything, so the one tolerant reader here says very little, and the standard library has no HTML parser - adding a dependency would need a decision and would trip the dependency gate. So the scanner is written to the specification, with the HTML5 void element list in it, and it carries the weight the reader cannot. Measured at 1 MiB + 1 B, an odd size: SVG rendered 800 x 600 with 16412 colours and 16897 drawable shapes. HTML gave 41504 elements with all six kinds of block and 9883 blocks, every tag balanced. Three mutations went in and all three leave the size and the exit code right: a list left unclosed, an ampersand left bare, and a viewBox written in lowercase - the last one the same length as what it replaced, so the file does not move by a single byte and only a guard that knows SVG is case sensitive notices. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 0fce8fa commit 3bcb429

11 files changed

Lines changed: 960 additions & 5 deletions

File tree

‎CHANGELOG-DEV.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,27 @@ belongs in one place.
1717

1818
### Added
1919

20+
- **The first reference tool that renders rather than parses.** SVG is checked
21+
by Inkscape drawing it to a bitmap, which is then opened and its colours
22+
counted. A drawing that parses and paints nothing passes every other check in
23+
this project, and fails that one.
24+
Two things about it were measured before it could be trusted, and both
25+
surprised. Inkscape 1.4.4 **exits zero on a malformed file** and says so only
26+
on standard error, so the exit code alone would bless a broken drawing. And
27+
it will not write the bitmap to standard output, so the only way to know one
28+
exists is to open the file it wrote. Both are why this checker is a script
29+
rather than a plain command line.
30+
- **HTML is the weakest format here to check, and that is the format's doing.**
31+
A parser is required to recover from almost anything, so the tolerant reader
32+
says very little, and the standard library has no HTML parser - adding a
33+
dependency would need a decision and would trip the dependency gate in CI. So
34+
the scanner is written to the specification, with the HTML5 void element list
35+
in it, and it carries the weight the reader cannot.
36+
Both new formats got mutations that leave the size and the exit code right:
37+
a list left unclosed, an ampersand left bare, and a `viewBox` written in
38+
lowercase - the last one **the same length as what it replaced**, so the file
39+
does not move by a single byte and only a guard that knows SVG is case
40+
sensitive notices.
2041
- **One primitive for every record based format, in `core.FillRecords`.** Whole
2142
records while another still leaves room, then a closing record built to the
2243
byte. That rule was written out by hand for the third and fourth time when

‎CHANGELOG.md‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,20 @@ because it turns other people's test suites red.
1717

1818
### Added
1919

20+
- **Two more formats: HTML and SVG.** `--format html` produces a complete HTML5
21+
document - doctype, character set, title and language - with five kinds of
22+
block in the body: headings, lists, tables, quotes and paragraphs. One quote
23+
carries an escaped ampersand, so a parser under test meets an entity rather
24+
than only plain text. `--format svg` produces a drawing with a viewBox and
25+
four kinds of shape, where lines are stroked and closed shapes are filled -
26+
a shape painted the wrong way round is invisible, and the size never notices.
27+
Both hit the size you ask for to the byte, and the filling goes through whole
28+
blocks and whole shapes. Nothing is ever cut in the middle, so a list, a table
29+
or a shape is always complete.
30+
The smallest HTML is 118 B and the smallest SVG is 193 B, which is the
31+
skeleton plus one whole block. Asking for less says so and names the number.
32+
**This finishes the text and structured formats.** Twelve of the twenty five
33+
now work end to end.
2034
- **Three more formats: CSV, JSON and XML.** `--format csv` produces a table
2135
with a header and six columns, where the description column is quoted and
2236
carries commas - the case a CSV reader has to get right. `--format json`

‎internal/format/all/all.go‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,11 +9,13 @@ package all
99
import (
1010
// Each format registers itself when its package is loaded.
1111
_ "github.com/donislawdev/TestingFilesGenerator/internal/format/csvfile"
12+
_ "github.com/donislawdev/TestingFilesGenerator/internal/format/htmlfile"
1213
_ "github.com/donislawdev/TestingFilesGenerator/internal/format/jsonfile"
1314
_ "github.com/donislawdev/TestingFilesGenerator/internal/format/logfile"
1415
_ "github.com/donislawdev/TestingFilesGenerator/internal/format/md"
1516
_ "github.com/donislawdev/TestingFilesGenerator/internal/format/pdf"
1617
_ "github.com/donislawdev/TestingFilesGenerator/internal/format/png"
18+
_ "github.com/donislawdev/TestingFilesGenerator/internal/format/svgfile"
1719
_ "github.com/donislawdev/TestingFilesGenerator/internal/format/txt"
1820
_ "github.com/donislawdev/TestingFilesGenerator/internal/format/wav"
1921
_ "github.com/donislawdev/TestingFilesGenerator/internal/format/xmlfile"

‎internal/format/htmlfile/html.go‎

Lines changed: 269 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,269 @@
1+
// Package htmlfile generates HTML documents.
2+
//
3+
// The package is not called "html" so that it cannot be confused with the
4+
// standard library package of that name at a glance, the same reason logfile is
5+
// not called log. The format id is "html".
6+
package htmlfile
7+
8+
import (
9+
"context"
10+
"fmt"
11+
"io"
12+
"math/rand/v2"
13+
"strconv"
14+
"strings"
15+
16+
"github.com/donislawdev/TestingFilesGenerator/internal/core"
17+
"github.com/donislawdev/TestingFilesGenerator/internal/format"
18+
)
19+
20+
// Measured on 2026-08-01, a comment holds arbitrary bytes to 1 MiB both after
21+
// the closing tag and inside the body. That says where the format tolerates
22+
// arbitrary bytes, and the filling still does not go there - a five megabyte
23+
// page that is one comment renders as an empty page.
24+
//
25+
// Whole blocks instead, and the remainder to the exact byte in the text of a
26+
// closing paragraph. Fifth format built this way.
27+
//
28+
// HTML is the weakest case in the group for checking. There is one tolerant
29+
// reader on this machine and it is lenient by design - the format itself says
30+
// a parser must recover from almost anything, so "it parsed" says very little.
31+
// The guards written against this format carry more of the weight than they do
32+
// for SVG, which has a renderer behind it.
33+
34+
const (
35+
generatorVersion = "1"
36+
37+
prologue = `<!DOCTYPE html>` + "\n" +
38+
`<html lang="en">` + "\n" +
39+
`<head>` + "\n" +
40+
`<meta charset="utf-8">` + "\n" +
41+
`<title></title>` + "\n" +
42+
`</head>` + "\n" +
43+
`<body>` + "\n"
44+
45+
// The title is filled in per file when a label is wanted, so the empty pair
46+
// above is what the smallest document carries.
47+
emptyTitle = `<title></title>`
48+
49+
paraOpen = "<p>"
50+
paraClose = "</p>\n"
51+
bodyClose = "</body>\n</html>\n"
52+
53+
tailLast = paraClose + bodyClose
54+
55+
// fixedWidth is every literal byte of the closing record.
56+
fixedWidth = len(paraOpen) + len(tailLast)
57+
)
58+
59+
func init() {
60+
format.Register(format.Descriptor{
61+
ID: "html",
62+
Extension: ".html",
63+
Fidelity: format.FidelityFull,
64+
Determinism: format.DeterminismByte,
65+
66+
// A document with an empty body is legal HTML and renders as a blank
67+
// page. That is a shape request rather than a byte count. The minimum
68+
// here is the skeleton and one whole block.
69+
MinBytes: minimumBytes(),
70+
71+
Padding: format.PaddingChannel{
72+
Name: "the text of the closing paragraph",
73+
Where: format.PlacementEnd,
74+
Capacity: 0,
75+
},
76+
77+
// A page shows what it is, so the label is a visible heading rather
78+
// than a comment.
79+
Label: format.LabelVisible,
80+
Oracle: "python-html",
81+
// Fragment mode, element counts, inline CSS and JS, images, forms and
82+
// the "every HTML5 tag" variant come later. Declaring none now makes a
83+
// recipe asking for them fail loudly.
84+
Properties: nil,
85+
GeneratorVersion: generatorVersion,
86+
Generator: generator{},
87+
})
88+
}
89+
90+
type generator struct{}
91+
92+
type memo struct {
93+
head string // the whole skeleton down to <body>, title included
94+
labelLine string // the visible heading, empty when absent
95+
seed uint64
96+
}
97+
98+
func (generator) Plan(r format.Request) (format.Plan, error) {
99+
min := minimumBytes()
100+
if r.Bytes < min {
101+
return format.Plan{}, &format.BelowMinimumError{
102+
Format: "HTML",
103+
Requested: r.Bytes,
104+
Minimum: min,
105+
Reason: "a page holds a head, a body and whole blocks, and one of each needs that much",
106+
Hint: fmt.Sprintf("Ask for %d B or more.", min),
107+
}
108+
}
109+
110+
p := format.Plan{
111+
Bytes: r.Bytes,
112+
Exact: true,
113+
Determinism: format.DeterminismByte,
114+
Properties: map[string]any{
115+
"encoding": "utf-8",
116+
"line_ending": "lf",
117+
"doctype": "html",
118+
"language": "en",
119+
},
120+
}
121+
122+
m := memo{seed: r.Seed, head: prologue}
123+
if r.Label {
124+
// The title and the heading say the same thing, which is what a page
125+
// does - one for the tab and one for the reader.
126+
label := core.Label("html", r.Bytes, r.Seed)
127+
withTitle := strings.Replace(prologue, emptyTitle, "<title>"+label+"</title>", 1)
128+
heading := "<h1>" + label + "</h1>\n"
129+
if int64(len(withTitle)-len(prologue)+len(heading))+minimumBytes() <= r.Bytes {
130+
m.head = withTitle
131+
m.labelLine = heading
132+
} else {
133+
p.Notes = append(p.Notes, format.Note{
134+
Code: "label_omitted",
135+
Detail: fmt.Sprintf(
136+
"The label needs %d B and this file has no room for it beside a whole block. Its name and the manifest still identify it.",
137+
len(heading)),
138+
})
139+
}
140+
}
141+
142+
p.Properties["label_embedded"] = m.labelLine != ""
143+
p.Memo = m
144+
return p, nil
145+
}
146+
147+
func (generator) Write(ctx context.Context, w io.Writer, p format.Plan) error {
148+
m, ok := p.Memo.(memo)
149+
if !ok {
150+
return fmt.Errorf("html: the plan was not produced by this generator")
151+
}
152+
153+
head := m.head + m.labelLine
154+
if err := core.WriteAll(w, []byte(head)); err != nil {
155+
return err
156+
}
157+
158+
rng := core.NewRand(m.seed)
159+
return core.FillRecords(ctx, w, rng, p.Bytes-int64(len(head)), blocks{})
160+
}
161+
162+
// blocks builds the body. A natural record is one complete block element and
163+
// the closing record is a paragraph stretched to land the byte count.
164+
//
165+
// Whole blocks only, for the same reason Markdown writes whole blocks: a table
166+
// or a list cut in the middle still renders, and it says something other than
167+
// it meant.
168+
type blocks struct{}
169+
170+
// Shortest is the smallest closing record: an empty paragraph plus the bytes
171+
// that close the body and the document.
172+
func (blocks) Shortest() int64 { return int64(fixedWidth) }
173+
174+
func (blocks) Append(dst []byte, rng *rand.Rand) []byte {
175+
switch rng.IntN(5) {
176+
case 0:
177+
dst = append(dst, "<h2>"...)
178+
dst = appendPhrase(dst, rng, 2+rng.IntN(3))
179+
return append(dst, "</h2>\n"...)
180+
case 1:
181+
dst = append(dst, "<ul>\n"...)
182+
for i, n := 0, 2+rng.IntN(3); i < n; i++ {
183+
dst = append(dst, "<li>"...)
184+
dst = appendPhrase(dst, rng, 3+rng.IntN(4))
185+
dst = append(dst, "</li>\n"...)
186+
}
187+
return append(dst, "</ul>\n"...)
188+
case 2:
189+
dst = append(dst, "<table>\n<tr><th>id</th><th>field</th><th>value</th></tr>\n"...)
190+
for i, n := 0, 2+rng.IntN(3); i < n; i++ {
191+
dst = append(dst, "<tr><td>"...)
192+
dst = strconv.AppendInt(dst, int64(i+1), 10)
193+
dst = append(dst, "</td><td>"...)
194+
dst = append(dst, words[rng.IntN(len(words))]...)
195+
dst = append(dst, "</td><td>"...)
196+
dst = append(dst, words[rng.IntN(len(words))]...)
197+
dst = append(dst, "</td></tr>\n"...)
198+
}
199+
return append(dst, "</table>\n"...)
200+
case 3:
201+
// A real page carries entities, and an ampersand left raw is the
202+
// classic way a document stops being well formed. The fixture carries
203+
// one on purpose.
204+
dst = append(dst, "<blockquote><p>"...)
205+
dst = appendPhrase(dst, rng, 4+rng.IntN(4))
206+
dst = append(dst, " &amp; "...)
207+
dst = appendPhrase(dst, rng, 2)
208+
return append(dst, "</p></blockquote>\n"...)
209+
default:
210+
dst = append(dst, paraOpen...)
211+
dst = appendPhrase(dst, rng, 12+rng.IntN(12))
212+
return append(dst, paraClose...)
213+
}
214+
}
215+
216+
func (blocks) AppendExact(dst []byte, rng *rand.Rand, n int64) []byte {
217+
start := len(dst)
218+
dst = append(dst, paraOpen...)
219+
used := int64(len(dst)-start) + int64(len(tailLast))
220+
dst = appendFiller(dst, n-used)
221+
return append(dst, tailLast...)
222+
}
223+
224+
func appendPhrase(dst []byte, rng *rand.Rand, n int) []byte {
225+
for i := 0; i < n; i++ {
226+
if i > 0 {
227+
dst = append(dst, ' ')
228+
}
229+
dst = append(dst, words[rng.IntN(len(words))]...)
230+
}
231+
return dst
232+
}
233+
234+
// appendFiller writes exactly n bytes of paragraph text out of readable words.
235+
//
236+
// It never emits an ampersand or an angle bracket, the two characters that
237+
// would have to be escaped - an escape would make the text longer than the
238+
// count asked for.
239+
func appendFiller(dst []byte, n int64) []byte {
240+
if n <= 0 {
241+
return dst
242+
}
243+
start := len(dst)
244+
for i := 0; int64(len(dst)-start) < n; i++ {
245+
if len(dst) > start {
246+
dst = append(dst, ' ')
247+
}
248+
dst = append(dst, words[i%len(words)]...)
249+
}
250+
return dst[:start+int(n)]
251+
}
252+
253+
// minimumBytes is the skeleton and one whole block, computed rather than
254+
// written down so it cannot drift away from the template.
255+
func minimumBytes() int64 {
256+
var b blocks
257+
return int64(len(prologue)) + b.Shortest()
258+
}
259+
260+
// words is the filler vocabulary. English by default, like the rest of the text
261+
// group.
262+
var words = []string{
263+
"account", "anchor", "banner", "browser", "caption", "content", "control",
264+
"default", "element", "feature", "footer", "gallery", "handler", "header",
265+
"heading", "inline", "layout", "listing", "margin", "marker", "message",
266+
"module", "navigation", "notice", "option", "padding", "preview", "profile",
267+
"section", "selector", "sidebar", "summary", "template", "toolbar",
268+
"tooltip", "viewport", "widget", "wrapper",
269+
}

0 commit comments

Comments
 (0)