diff --git a/README.md b/README.md index f6cf912..1a45a89 100644 --- a/README.md +++ b/README.md @@ -206,7 +206,7 @@ call it after a reply to catch a stray byte or a second status line (e.g. a NUL | `(*Server) TLSHandshakes()`, `TLSFailed()`, `TLSServerNames()` | completed and failed handshakes (implicit TLS and STARTTLS), SNI of every ClientHello | | `LineHandler(f func(c *Conn, line string) (reply []string, closeConn bool)) func(*Conn)` | answer each client line with what `f` returns, in any order (what `Script` cannot); `f` may `c.ReadDotBlock()` for an article; `Faults` apply | | `PeerHandler(*PeerOptions) func(*Conn)`, `type PeerOptions` | transit peer: MODE STREAM, CHECK, TAKETHIS (RFC 4644), IHAVE (RFC 3977 6.3.2), AUTHINFO; options below | -| `ReplyCorpus() []ReplyCase`, `ReplyCaseByName(name) (ReplyCase, bool)` | 112 named wrong and malformed server replies for the test suite of a client, see below | +| `ReplyCorpus() []ReplyCase`, `ReplyCaseByName(name) (ReplyCase, bool)` | 125 named wrong and malformed server replies for the test suite of a client, see below | | `(*Conn) Wait(ch <-chan struct{}) bool`, `Sleep(d) bool` | for a Handler or `PeerOptions.BeforeReply`: block until `ch` fires (true) or the Deadline passes / the server closes (false); `Sleep` is bounded the same way | | `NewStore() *Store` (the zero value works too) | stateful store; safe for concurrent use; returns copies | | `(*Store) AddGroup`, `AddGroupAt`, `SetPosting`, `AddArticle(group, Article) int64` | setup | @@ -396,27 +396,28 @@ Faults on the COMPRESS reply (`DropAfterBytes`, `Split` offsets) count the `Empt `With206` bytes too. `compress/flate` cannot flush partially and its reader holds `Z_PARTIAL_FLUSH` data back until more arrives; the kit writes the partial flush itself, and its client inflater reads it. -**Reply corpus.** `fakeserver.ReplyCorpus()` is 112 named wrong and malformed server replies (GROUP +**Reply corpus.** `fakeserver.ReplyCorpus()` is 125 named wrong and malformed server replies (GROUP answered 215, a 211 without numbers, a greeting of `hello there`, an OVER row with a letter for the article number) for the test suite of an NNTP client, so each client project does not write the table again; `ReplyCaseByName(name)` finds one. A `ReplyCase` is `{Name; Family; Cmd, Line string; -Lines []string; Why string; Want Outcome; NeedsTimeout bool}`: +Lines []string; Why string; Want Outcome; NeedsTimeout, Final bool}`: | field / method | what | |---|---| | `Name`, `Why` | a unique short identifier fit for `t.Run` (`group-211-nofields`); what is wrong, with the RFC where there is one | -| `Family` | `FamilyCode` (well-formed reply, wrong code or shape for the command: 37 cases), `FamilyStatus` (malformed status line: 37), `FamilyGreeting` (bad greeting: 10), `FamilyBlock` (bad rows or terminator of a multi-line reply: 28) | +| `Family` | `FamilyCode` (well-formed reply, wrong code or shape for the command: 39 cases), `FamilyStatus` (malformed status line: 37), `FamilyGreeting` (bad greeting: 10), `FamilyBlock` (bad rows or terminator of a multi-line reply: 39) | | `Cmd`, `Line` | the `Faults` key (upper-cased command) whose reply is replaced, and a sample client line (`GROUP g`); both empty for a greeting case | | `Lines` | written instead of the normal reply, each plus CRLF (a block carries its own `.` line); may hold bytes that are no valid NNTP (NUL, 8-bit, bare LF) | -| `Want` | the generic expected client outcome: `WantError` (the call fails cleanly: no panic, no hang, no data handed back as good; then the connection is in sync or closed; 74 cases), `WantEither` (a strict client fails, a lenient one may accept; neither may panic, hang or desynchronise; 34), `WantAccept` (odd but legal, e.g. `high` below `low` for an empty group, 281 to AUTHINFO USER; the client must accept it; 4). Error types and reuse of the connection afterwards are for each client suite to agree on | +| `Want` | the generic expected client outcome: `WantError` (the call fails cleanly: no panic, no hang, no data handed back as good; then the connection is in sync or closed; 81 cases), `WantEither` (a strict client fails, a lenient one may accept; neither may panic, hang or desynchronise; 38), `WantAccept` (odd but legal, e.g. `high` below `low` for an empty group, 281 to AUTHINFO USER, an address in the 240 text of POST; the client must accept it; 6). Error types and reuse of the connection afterwards are for each client suite to agree on | | `NeedsTimeout` | the client waits for more data than the reply holds (a terminator that never comes): the test needs a read timeout and the call ends with one | -| `Apply(*Options)`, `ApplyLine(*Options)` | `Faults.Override[Cmd]` (every line of that command) or `Faults.OverrideLine[Line]` (that line only); `Options.Greeting` for a greeting case; they add to the maps already in `o`. A command case needs a mode that applies `Faults`, e.g. `Options.Store` | +| `Final` | `Lines` replace the reply after the article of POST or IHAVE (`Faults.OverrideFinal`): the client gets its normal 340 or 335, sends an article, then reads the case. The corpus's Final cases are POST and need a Store that allows posting | +| `Apply(*Options)`, `ApplyLine(*Options)` | `Faults.Override[Cmd]` (every line of that command) or `Faults.OverrideLine[Line]` (that line only); `Faults.OverrideFinal[Cmd]` for a `Final` case (both); `Options.Greeting` for a greeting case; they add to the maps already in `o`. A command case needs a mode that applies `Faults`, e.g. `Options.Store` | | `Bytes() []byte`, `IsGreeting() bool` | exactly what the server writes for the case; whether it is a greeting case (the reply to connecting) | A missing `.` terminator is `Faults.Endless`, an overlong block line `Faults.LongLine`, a reply cut short `Faults.DropInMultiline`: not repeated in the corpus. A reply that depends on an earlier answer of the same exchange (POST answered 340 and then 340 again after the article) needs a Script or a Handler and is not -in it. +in it; a fixed final reply after the article is a `Final` case. ```go for _, rc := range fakeserver.ReplyCorpus() { @@ -658,6 +659,7 @@ into a `fakeserver.Article` body like any other: no CRLF and no dot-stuffing (th | `EncodeParts(name, data, partSize, Options) [][]string` | one post per part: `=ybegin part= total=`, `=ypart begin= end=`, the data, `=yend size= part= pcrc32=` (and `crc32=` of the whole data on the last part) | | `Data(Profile, size, seed) []byte`; `Random`, `Plain`, `Escapes`, `Edges` | reproducible data: random bytes (most of Usenet), no escapes, an escape in every second byte, a third of the bytes at `.`, tab and space | | `Generator(Profile, low, high, size func(n int64) int, Options) fakeserver.Generator` | a generated group: article n is `Data(p, size(n), n)` named `

-.bin`, with `GeneratedArticle`'s headers and a yEnc Subject | +| `Corpus() []Case`, `CaseByName(name) (Case, bool)`; `type Case struct { Name string; Lines []string; Data []byte; Why string; Want fakeserver.Outcome }` | broken and odd posts for the test suite of a decoder, with the reply corpus's outcome classes: a stray `=yend` after a complete block and text after `=yend` (accept, decode to `Data`), a bad CRC or size (error), no `=yend` (either) | `RawDots` and `RawSpaces` model posting tools that leave those characters to the server's dot-stuffing and to the reader. The broken-post options model corrupt uploads; `Faults.DropInMultiline` diff --git a/fakeserver/doc.go b/fakeserver/doc.go index f2eb14c..9e67629 100644 --- a/fakeserver/doc.go +++ b/fakeserver/doc.go @@ -254,7 +254,7 @@ // wrong; Apply puts it into the Options of a fake server (a command case needs a mode that // applies Faults, e.g. a Store): // -// func ReplyCorpus() []ReplyCase // 112 cases +// func ReplyCorpus() []ReplyCase // 125 cases // func ReplyCaseByName(name string) (ReplyCase, bool) // type ReplyCase struct { // Name string // unique, fit for t.Run, e.g. "group-211-nofields" @@ -265,9 +265,10 @@ // Why string // what is wrong, with the RFC where there is one // Want Outcome // WantError (fail cleanly), WantEither (strict fails, lenient may accept; no panic, hang or desync), WantAccept (odd but legal: must be accepted) // NeedsTimeout bool // the client waits for more data than the reply holds: the test needs a read timeout +// Final bool // Lines replace the reply after the article of POST/IHAVE (Faults.OverrideFinal): the client gets 340/335, sends an article, then reads the case // } -// func (rc ReplyCase) Apply(o *Options) // Faults.Override[Cmd]; Options.Greeting for a greeting case; adds to the maps already in o -// func (rc ReplyCase) ApplyLine(o *Options) // Apply for Line only (Faults.OverrideLine[Line]) +// func (rc ReplyCase) Apply(o *Options) // Faults.Override[Cmd]; OverrideFinal[Cmd] for a Final case; Options.Greeting for a greeting case; adds to the maps already in o +// func (rc ReplyCase) ApplyLine(o *Options) // Apply for Line only (Faults.OverrideLine[Line]); a Final case as Apply // func (rc ReplyCase) Bytes() []byte // exactly what the server writes: every line + CRLF // func (rc ReplyCase) IsGreeting() bool // diff --git a/fakeserver/replycorpus.go b/fakeserver/replycorpus.go index a9c78ac..14b8e8f 100644 --- a/fakeserver/replycorpus.go +++ b/fakeserver/replycorpus.go @@ -42,7 +42,9 @@ import "strings" // A missing "." terminator is Faults.Endless, an overlong block line Faults.LongLine, a reply // cut short Faults.DropInMultiline: not repeated here. A reply that depends on an earlier // answer of the same exchange (POST answered 340 and then, after the article, 340 again) needs -// a Script or a Handler, not an Override, and is not in the corpus. +// a Script or a Handler, not an Override, and is not in the corpus. A case with Final set +// replaces the reply that follows the article of POST or IHAVE (240/441, 235/437) instead: the +// first reply is the normal 340 or 335, and the test sends an article before it looks. // Family groups the cases of the reply corpus. type Family string @@ -97,6 +99,13 @@ type ReplyCase struct { // terminator that never comes): the test needs a read timeout in the client, and the // call ends with a timeout error. NeedsTimeout bool + // Final is set when Lines replace the reply that follows the article of POST or IHAVE + // (240/441, 235/437; Faults.OverrideFinal), not the first reply: the client gets its + // normal 340 or 335, sends an article and then reads the case. ApplyLine treats it like + // Apply, since OverrideFinal is keyed by the command only. The corpus's Final cases are + // POST: they need a Store that allows posting (not ReaderOnly); Script, LineHandler and + // Handler modes do not apply OverrideFinal. + Final bool } // IsGreeting reports whether the case is a bad greeting (the reply to connecting, not to a @@ -107,7 +116,8 @@ func (rc ReplyCase) IsGreeting() bool { return rc.Family == FamilyGreeting } func (rc ReplyCase) Bytes() []byte { return joinCRLF(rc.Lines) } // Apply makes the server of o write the case: Faults.Override[Cmd] for a command case (every -// line of that command gets the reply), Options.Greeting for a greeting case. It adds to the +// line of that command gets the reply), Faults.OverrideFinal[Cmd] for a Final case, +// Options.Greeting for a greeting case. It adds to the // maps already in o and keeps the other faults. The server still needs a mode that applies // Faults: a Store (Options.Store) for a command case; a greeting case works in any mode that // writes Options.Greeting. @@ -116,6 +126,13 @@ func (rc ReplyCase) Apply(o *Options) { o.Greeting = strings.Join(rc.Lines, "\r\n") return } + if rc.Final { + if o.Faults.OverrideFinal == nil { + o.Faults.OverrideFinal = make(map[string][]string) + } + o.Faults.OverrideFinal[rc.Cmd] = append([]string(nil), rc.Lines...) + return + } if o.Faults.Override == nil { o.Faults.Override = make(map[string][]string) } @@ -125,7 +142,7 @@ func (rc ReplyCase) Apply(o *Options) { // ApplyLine is Apply for the sample client line only (Faults.OverrideLine[Line]): other lines // of the same command get their normal reply. func (rc ReplyCase) ApplyLine(o *Options) { - if rc.IsGreeting() { + if rc.IsGreeting() || rc.Final { rc.Apply(o) return } @@ -166,6 +183,11 @@ func rcGreeting(name string, want Outcome, why string, lines ...string) ReplyCas return ReplyCase{Name: name, Family: FamilyGreeting, Lines: lines, Why: why, Want: want} } +func rcFinal(rc ReplyCase) ReplyCase { + rc.Final = true + return rc +} + func rcTimeout(rc ReplyCase) ReplyCase { rc.NeedsTimeout = true return rc @@ -219,6 +241,9 @@ func ReplyCorpus() []ReplyCase { rcCase(c, "ihave-340-first", "IHAVE ", WantError, "340 is the POST send code; IHAVE answers 335 (6.3.2)", "340 ok"), rcCase(c, "quit-200", "QUIT", WantError, "QUIT answers 205 (5.4)", "200 hm"), rcCase(c, "authinfo-user-250", "AUTHINFO USER u", WantError, "AUTHINFO USER answers 381, 281 or 481 (RFC 4643 2.3)", "250 hm"), + // code, Final: the reply after the article of POST, a well-formed 240 with odd text + rcFinal(rcCase(c, "post-240-address-in-text", "POST", WantAccept, "the 240 text is free and not defined to carry a message-id (6.3.1); an address in angle brackets there is not the Message-ID of the article", "240 posted ok; report abuse to ")), + rcFinal(rcCase(c, "post-240-empty-brackets", "POST", WantAccept, "the 240 text is free (6.3.1); '<>' in it is no message-id (RFC 5536 3.1.3 needs content between the brackets)", "240 <> posted")), rcCase(c, "authinfo-user-215-block", "AUTHINFO USER u", WantError, "a 215 block where AUTHINFO USER answers one line (RFC 4643 2.3)", rcBlock("215 what", "111 19990101000000")...), rcCase(c, "authinfo-user-281", "AUTHINFO USER u", WantAccept, "281 to USER with no PASS asked: legal when the server needs no password (RFC 4643 2.3.1); a client must not send PASS", "281 ok"), @@ -285,6 +310,17 @@ func ReplyCorpus() []ReplyCase { rcCase(b, "over-non-utf8-subject", "OVER 1-2", WantEither, "bytes that are not UTF-8 in the Subject field; a client keeps them or reports an error", rcBlock("224 overview follows", "1\tsubj \xff\xfe\ta@b\td\t<1@x>\t\t1\t1")...), rcCase(b, "hdr-nonnumeric-row", "HDR Subject 1", WantError, "a word for the article number of an HDR row (8.5)", rcBlock("225 hdr", "abc x")...), rcCase(b, "hdr-row-no-space", "HDR Subject 1", WantEither, "an HDR row of the number only, no value and no space (8.5)", rcBlock("225 hdr", "1")...), + rcCase(b, "hdr-msgid-row-for-range", "HDR Subject 1-2", WantEither, "a message-id where the article number belongs: rows of a range request carry numbers, the 0 form is for a message-id argument only (8.5.2); a client that checks the row syntax but not which request it answers may accept it", rcBlock("225 hdr", " x")...), + rcCase(b, "hdr-msgid-unclosed-row", "HDR Subject ", WantError, "a row that starts with '<' but has no closing '>'; the row for a message-id argument starts with 0 (8.5.2)", rcBlock("225 hdr", "", WantError, "'<>' where the 0 of a message-id argument belongs (8.5.2)", rcBlock("225 hdr", "<> value")...), + rcCase(b, "hdr-negative-row", "HDR Subject 1-2", WantError, "a negative article number in an HDR row (8.5.2: numbers are digits only)", rcBlock("225 hdr", "-1 x")...), + rcCase(b, "hdr-plus-sign-row", "HDR Subject 1-2", WantEither, "a '+' before the article number (8.5.2: digits only); common number parsers (strconv.Atoi, strtol, int()) accept it and get the right number", rcBlock("225 hdr", "+1 x")...), + rcCase(b, "hdr-underscore-row", "HDR Subject 1-2", WantError, "'1_0' for the article number: some number parsers (Python int(), Go ParseInt with base 0) accept the underscore and read 10, a wrong number handed back as good (8.5.2: digits only)", rcBlock("225 hdr", "1_0 x")...), + rcCase(b, "hdr-non-ascii-digit-row", "HDR Subject 1-2", WantError, "U+0661 (Arabic-Indic digit one) for the article number: Unicode-aware parsers read it as 1 (8.5.2: ASCII digits only)", rcBlock("225 hdr", "\u0661 x")...), + rcCase(b, "list-overview-fmt-empty-name", "LIST OVERVIEW.FMT", WantError, "a field of ':' alone, with no header name (8.4.2)", rcBlock("215 fmt", "Subject:", "From:", "Date:", "Message-ID:", "References:", ":bytes", ":lines", ":")...), + rcCase(b, "list-overview-fmt-bare-full", "LIST OVERVIEW.FMT", WantError, "' :full' with no header name before the suffix, also after trimming the space (8.4.2: ':full' follows a header name)", rcBlock("215 fmt", "Subject:", "From:", "Date:", "Message-ID:", "References:", ":bytes", ":lines", " :full")...), + rcCase(b, "list-overview-fmt-space-before-colon", "LIST OVERVIEW.FMT", WantEither, "'Xref :full', a space between the header name and the colon (8.4.2); a lenient client trims it to Xref:full, none may key the field 'Xref '", rcBlock("215 fmt", "Subject:", "From:", "Date:", "Message-ID:", "References:", ":bytes", ":lines", "Xref :full")...), + rcCase(b, "list-overview-fmt-leading-space", "LIST OVERVIEW.FMT", WantEither, "' Subject:', a space before the header name (8.4.2); a lenient client trims it, none may key the field ' Subject'", rcBlock("215 fmt", " Subject:", "From:", "Date:", "Message-ID:", "References:", ":bytes", ":lines")...), rcCase(b, "list-active-too-few-fields", "LIST ACTIVE", WantError, "a LIST ACTIVE row without the status field (7.6.3)", rcBlock("215 list", "alt.test 5 1")...), rcCase(b, "list-active-nonnumeric", "LIST ACTIVE", WantError, "letters for high and low (7.6.3)", rcBlock("215 list", "alt.test x y y")...), rcCase(b, "list-active-empty-group", "LIST ACTIVE", WantAccept, "high below low is how an empty group reads (7.6.3): not an error", rcBlock("215 list", "alt.test 0 1 y")...), diff --git a/fakeserver/replycorpus_test.go b/fakeserver/replycorpus_test.go index 2899647..ea6d5bd 100644 --- a/fakeserver/replycorpus_test.go +++ b/fakeserver/replycorpus_test.go @@ -46,9 +46,43 @@ func rcExpect(t *testing.T, rc ReplyCase, apply func(ReplyCase, *Options)) strin if _, err := io.WriteString(conn, rc.Line+"\r\n"); err != nil { t.Fatalf("write: %v", err) } + if rc.Final { + rcUpload(t, conn, rc.Cmd) + } return rcRead(t, conn, len(rc.Bytes())) } +// rcUpload reads the normal first reply of POST (340) or IHAVE (335) and sends an article. +func rcUpload(t *testing.T, conn net.Conn, cmd string) { + t.Helper() + want := map[string]string{"POST": "340", "IHAVE": "335"}[cmd] + if want == "" { + t.Fatalf("a Final case for %q, want POST or IHAVE", cmd) + } + line := rcReadLine(t, conn) + if !strings.HasPrefix(line, want+" ") { + t.Fatalf("first reply %q, want %s", line, want) + } + art := "From: a@b.invalid\r\nNewsgroups: g\r\nSubject: s\r\nMessage-ID: \r\n\r\nbody\r\n.\r\n" + if _, err := io.WriteString(conn, art); err != nil { + t.Fatalf("write article: %v", err) + } +} + +// rcReadLine reads one CRLF line, a byte at a time so nothing after it is consumed. +func rcReadLine(t *testing.T, conn net.Conn) string { + t.Helper() + var b []byte + var c [1]byte + for !strings.HasSuffix(string(b), "\r\n") { + if _, err := io.ReadFull(conn, c[:]); err != nil { + t.Fatalf("read line: %v (got %q)", err, b) + } + b = append(b, c[0]) + } + return strings.TrimSuffix(string(b), "\r\n") +} + func TestReplyCorpusShape(t *testing.T) { cases := ReplyCorpus() if len(cases) < 40 { @@ -90,6 +124,9 @@ func TestReplyCorpusShape(t *testing.T) { if rc.Want != WantError && rc.Want != WantEither && rc.Want != WantAccept { t.Errorf("%s: Want %q", rc.Name, rc.Want) } + if rc.Final && rc.Cmd != "POST" && rc.Cmd != "IHAVE" { + t.Errorf("%s: Final for %q, want POST or IHAVE", rc.Name, rc.Cmd) + } if rc.NeedsTimeout && rc.Want != WantError { t.Errorf("%s: NeedsTimeout with Want %q", rc.Name, rc.Want) } @@ -171,7 +208,7 @@ func TestReplyCorpusApplyLine(t *testing.T) { // Nothing but the case's bytes is written: a sample of cases, then a probe after them. func TestReplyCorpusNoExtraBytes(t *testing.T) { - for _, name := range []string{"group-211-negative", "article-222-block", "over-nonnumeric-row", "status-bare-lf", "greeting-400"} { + for _, name := range []string{"group-211-negative", "article-222-block", "over-nonnumeric-row", "status-bare-lf", "greeting-400", "post-240-address-in-text"} { rc, ok := ReplyCaseByName(name) if !ok { t.Fatalf("no case %q", name) @@ -181,6 +218,9 @@ func TestReplyCorpusNoExtraBytes(t *testing.T) { if !rc.IsGreeting() { rcRead(t, conn, len(DefaultGreeting)+2) _, _ = io.WriteString(conn, rc.Line+"\r\n") + if rc.Final { + rcUpload(t, conn, rc.Cmd) + } } rcRead(t, conn, len(rc.Bytes())) _ = conn.SetReadDeadline(time.Now().Add(150 * time.Millisecond)) @@ -209,6 +249,13 @@ func TestReplyCaseApplyKeepsOptions(t *testing.T) { t.Fatalf("OverrideLine = %v", o.Faults.OverrideLine) } + f, _ := ReplyCaseByName("post-240-address-in-text") + var of Options + f.ApplyLine(&of) + if got := of.Faults.OverrideFinal["POST"]; len(got) != 1 || got[0] != f.Lines[0] || of.Faults.Override != nil || of.Faults.OverrideLine != nil { + t.Fatalf("Final ApplyLine: %+v", of.Faults) + } + g, _ := ReplyCaseByName("greeting-two-lines") var og Options g.Apply(&og) diff --git a/yenc/corpus.go b/yenc/corpus.go new file mode 100644 index 0000000..29ad6b2 --- /dev/null +++ b/yenc/corpus.go @@ -0,0 +1,69 @@ +package yenc + +import ( + "slices" + + "github.com/go-while/nntp-testkit/fakeserver" +) + +// Case is one broken or odd yEnc post body for the test suite of a yEnc DECODER (a client +// library, a downloader, a SIMD decoder). Every decoder needs the same table, so it lives +// here once. A suite loops over Corpus, decodes c.Lines, or serves them as the body of a +// fakeserver.Article and fetches that, and checks the outcome against c.Want: +// +// for _, c := range yenc.Corpus() { +// t.Run(c.Name, func(t *testing.T) { +// got, err := decode(c.Lines) +// // WantAccept: err == nil and got == c.Data +// // WantError: err != nil (no panic, no hang) +// // WantEither: either; with err == nil, got == c.Data +// }) +// } +// +// Want uses the classes of the reply corpus (fakeserver.WantError, WantEither, WantAccept). +// For a decoder, WantError means it reports the post as broken; one that hands back the data +// with a CRC or size warning passes when the warning reaches the caller. +// The cases are facts about yEnc 1.3 posts, written from scratch. +type Case struct { + // Name is a unique short identifier fit for t.Run, e.g. "stray-yend-after-block". + Name string + // Lines are the body lines as Encode writes them: no CRLF, not dot-stuffed. + Lines []string + // Data are the decoded bytes a decoder returns when it accepts the case; nil when no + // decoder may accept it (Want is WantError). + Data []byte + // Why says what is odd or broken. + Why string + // Want is the generic expected decoder outcome. + Want fakeserver.Outcome +} + +// CaseByName returns the case of that name. +func CaseByName(name string) (Case, bool) { + for _, c := range Corpus() { + if c.Name == name { + return c, true + } + } + return Case{}, false +} + +// Corpus returns the yEnc cases, in a fixed order, a new slice each call. +func Corpus() []Case { + data := Data(Escapes, 300, 1) + good := Encode("a.bin", data, Options{Line: 64}) + with := func(lines ...string) []string { return append(slices.Clone(good), lines...) } + bad := func(o Options) []string { o.Line = 64; return Encode("a.bin", data, o) } + return []Case{ + {Name: "stray-yend-after-block", Lines: with("=yend size=0"), Data: slices.Clone(data), Want: fakeserver.WantAccept, + Why: "a second =yend after a complete block whose size and CRC match: yEnc 1.3 says nothing about lines after the block, decoders in practice stop at the first =yend, and the stray line must not be checked against the block"}, + {Name: "text-after-yend", Lines: with("", "-- ", "posted with a newsreader"), Data: slices.Clone(data), Want: fakeserver.WantAccept, + Why: "plain text (a signature) after =yend: yEnc data ends at =yend, what follows is not decoded"}, + {Name: "bad-crc32", Lines: bad(Options{BadCRC: true}), Want: fakeserver.WantError, + Why: "=yend gives a crc32 that does not match the data"}, + {Name: "bad-size", Lines: bad(Options{BadSize: true}), Want: fakeserver.WantError, + Why: "=yend gives a size one byte more than the data and than =ybegin says"}, + {Name: "no-yend", Lines: bad(Options{NoEnd: true}), Data: slices.Clone(data), Want: fakeserver.WantEither, + Why: "the post ends without =yend: the data may be cut short and nothing can be checked; a lenient decoder returns what it has"}, + } +} diff --git a/yenc/corpus_test.go b/yenc/corpus_test.go new file mode 100644 index 0000000..ab7d9d9 --- /dev/null +++ b/yenc/corpus_test.go @@ -0,0 +1,122 @@ +package yenc + +import ( + "bytes" + "fmt" + "hash/crc32" + "strings" + "testing" + + "github.com/go-while/nntp-testkit/fakeserver" +) + +func TestCorpusShape(t *testing.T) { + seen := map[string]bool{} + for _, c := range Corpus() { + if c.Name == "" || strings.ContainsAny(c.Name, " \t/") || seen[c.Name] { + t.Errorf("bad or duplicate name %q", c.Name) + } + seen[c.Name] = true + if c.Why == "" || len(c.Lines) == 0 { + t.Errorf("%s: empty Why or Lines", c.Name) + } + switch c.Want { + case fakeserver.WantError: + if c.Data != nil { + t.Errorf("%s: WantError with Data", c.Name) + } + case fakeserver.WantAccept, fakeserver.WantEither: + if c.Data == nil { + t.Errorf("%s: Want %q without Data", c.Name, c.Want) + } + default: + t.Errorf("%s: Want %q", c.Name, c.Want) + } + for _, l := range c.Lines { + if strings.ContainsAny(l, "\r\n") { + t.Errorf("%s: line %q holds CR or LF", c.Name, l) + } + } + } + if _, ok := CaseByName("stray-yend-after-block"); !ok { + t.Error("no stray-yend-after-block") + } + if _, ok := CaseByName("no-such-case"); ok { + t.Error("CaseByName found a case that does not exist") + } +} + +// The accepted cases decode to Data up to their first =yend (a minimal decoder for the test). +func TestCorpusData(t *testing.T) { + for _, c := range Corpus() { + if c.Data == nil { + continue + } + if got := decodeUntilYend(c.Lines); !bytes.Equal(got, c.Data) { + t.Errorf("%s: decodes to %d bytes, want the %d of Data", c.Name, len(got), len(c.Data)) + } + } + c, _ := CaseByName("stray-yend-after-block") + if n := len(c.Lines); c.Lines[n-1] != "=yend size=0" || !strings.HasPrefix(c.Lines[n-2], "=yend size=300 ") { + t.Errorf("stray-yend-after-block ends %q", c.Lines[len(c.Lines)-2:]) + } +} + +// The broken cases differ from the good block where they claim to; the stray =yend follows a +// block whose size and CRC match Data. +func TestCorpusTrailers(t *testing.T) { + c, _ := CaseByName("stray-yend-after-block") + want := fmt.Sprintf("=yend size=%d crc32=%08x", len(c.Data), crc32.ChecksumIEEE(c.Data)) + if got := c.Lines[len(c.Lines)-2]; got != want { + t.Errorf("stray-yend-after-block block ends %q, want %q", got, want) + } + last := func(name string) string { c, _ := CaseByName(name); return c.Lines[len(c.Lines)-1] } + if l := last("bad-crc32"); l == want || !strings.HasPrefix(l, fmt.Sprintf("=yend size=%d ", len(c.Data))) { + t.Errorf("bad-crc32 ends %q", l) + } + if l := last("bad-size"); !strings.HasPrefix(l, fmt.Sprintf("=yend size=%d crc32=%08x", len(c.Data)+1, crc32.ChecksumIEEE(c.Data))) { + t.Errorf("bad-size ends %q", l) + } + if l := last("no-yend"); strings.HasPrefix(l, "=yend") { + t.Errorf("no-yend ends %q", l) + } +} + +func TestCorpusFresh(t *testing.T) { + a := Corpus() + // no two cases of one call share Data + for i := range a { + for j := i + 1; j < len(a); j++ { + if len(a[i].Data) > 0 && len(a[j].Data) > 0 && &a[i].Data[0] == &a[j].Data[0] { + t.Errorf("%s and %s share Data", a[i].Name, a[j].Name) + } + } + } + a[0].Lines[0] = "changed" + a[0].Data[0]++ + b := Corpus() + if b[0].Lines[0] == "changed" || bytes.Equal(a[0].Data, b[0].Data) { + t.Fatal("Corpus shares state between calls") + } +} + +func decodeUntilYend(lines []string) []byte { + var out []byte + for _, l := range lines { + switch { + case strings.HasPrefix(l, "=ybegin "), strings.HasPrefix(l, "=ypart "): + continue + case strings.HasPrefix(l, "=yend"): + return out + } + for i := 0; i < len(l); i++ { + c := l[i] + if c == '=' && i+1 < len(l) { + i++ + c = l[i] - 64 + } + out = append(out, c-42) + } + } + return out +} diff --git a/yenc/doc.go b/yenc/doc.go index 5b23917..ea9d21d 100644 --- a/yenc/doc.go +++ b/yenc/doc.go @@ -2,8 +2,9 @@ // body lines, ready for a fakeserver.Article, and a fakeserver.Generator that serves a group of // them. Options cover the choices posting tools differ in (line length, which characters to // escape) and broken posts (no =ybegin or =yend line, a wrong size or CRC). Data makes -// reproducible test data in a few profiles. It is an encoder for tests, not a decoder; only the -// standard library is used. +// reproducible test data in a few profiles. Corpus is a shared list of broken and odd posts +// with the outcome a decoder's test suite expects. It is an encoder for tests, not a decoder; +// only the standard library is used. // // Contract (plan nntp-testkit). Implemented in yenc.go: // @@ -49,4 +50,16 @@ // // of Data(p, size(n), uint64(n)), with the headers, Message-ID and Date of // // fakeserver.GeneratedArticle (so Lookup stays nil) and the Subject posting tools write: // // ` - "" yEnc (1/1) `. size nil: 750000 bytes for every article. +// +// Implemented in corpus.go: +// +// type Case struct { +// Name string // unique, fit for t.Run: "stray-yend-after-block" +// Lines []string // body lines as Encode writes them +// Data []byte // the decoded bytes when a decoder accepts the case; nil for WantError +// Why string +// Want fakeserver.Outcome // WantError, WantEither or WantAccept, as in the reply corpus +// } +// func Corpus() []Case // fixed order, a new slice each call +// func CaseByName(name string) (Case, bool) package yenc