Skip to content

NNTP Proxy/Server Core: Initial Implementation (Server.go) - #62

Merged
go-while merged 157 commits into
mainfrom
testing
Oct 18, 2026
Merged

NNTP Proxy/Server Core: Initial Implementation (Server.go)#62
go-while merged 157 commits into
mainfrom
testing

Conversation

@go-while

@go-while go-while commented Jun 10, 2025 •
edited
Loading

Copy link
Copy Markdown
Owner

NNTP proxy, web job queue, review fixes, pure-Go builds (testing → main)

This PR started in June 2025 as the first version of the NNTP proxy (Server.go). Since then testing gained the web job queue, and in October 2026 a full review of the codebase with fixes for everything it found (the plan and every result are in review-the-codebase-for-jolly-petal.md). This description covers the state of testing now.

All verification ran against local go-while/nntp-server instances with generated test articles (tools/nntpseed), never against live Usenet providers. CI on testing is green.

📡 NNTP proxy (-proxytcp / -proxytls)

Protocol

  • Commands: CAPABILITIES, AUTHINFO USER/PASS, MODE READER, DATE, QUIT, GROUP, NEXT, LAST, LIST, XOVER/OVER, XHDR/HDR, and ARTICLE/BODY/HEAD/STAT by message-id, by number, or for the current article.
  • Follows RFC 3977: GROUP selects the first article, the article pointer only moves on success, replies to requests by number carry the real number and message-id, NEXT/LAST forward the server's answer, DATE uses the RFC format.
  • Errors no longer drop the connection: unknown commands, syntax errors and failed lookups are answered; only QUIT, a failed login or a failed write close it. Every write is checked.
  • Articles are relayed unchanged (no added Path:, no stripped headers). A damaged or bad-CRC article from one provider is tried on the next; the client gets 430 instead of an empty article.

Limits and robustness

  • 60 s to log in, 5 min idle between commands (then 400), at most 64 connections waiting to log in.
  • Command lines capped at 512 bytes before they are buffered; list and overview replies at 64 MiB.
  • The upstream connection is parked before writing to a slow client.
  • Negative cache keyed by group + number, with a size cap and a cleanup job.
  • The accept loop backs off on errors (running out of file descriptors used to spin 3 cores and write 190 MB of log in seconds).
  • The proxy listens only once its providers are loaded; TLS 1.2+; a TLS failure doesn't take down the TCP listener.

🔐 Users

  • bcrypt hashes, account expiry, per-user connection limits. An unknown user costs the same bcrypt time as a wrong password (no timing oracle). A failed or empty reload of the passwd file keeps the users loaded before.
  • AUTHINFO PASS pw junk no longer logs in as pw.

Proxy flags

  • -proxytcp (int): start the proxy on this TCP port, e.g. -proxytcp=1119. Default 0 (off).
  • -proxytls (int): start the proxy on this TLS port, e.g. -proxytls=1563. Default 0 (off).
  • -tlscrt / -tlskey (string): TLS certificate and key for the proxy. Default fullchain.pem / privkey.pem in the current directory.
  • -proxyadduser (string): add a proxy user, 'username|password|maxconns|expires|(no)post'. The password is given in cleartext and hashed when written. Expiry is a Unix timestamp or a duration (7d, 12h, 30m). Example: -proxyadduser="HelloWorld|NotAsecurePassword|5|42d|nopost" (5 connections, expires in 42 days).
  • -proxypasswdfile (string): the proxy's passwd file. Default .proxypasswd.

🌐 Web interface and job queue (-web)

  • Upload NZBs (.nzb, .nzb.gz, up to 128 MiB), queue them, start one or all, stop a running job or cancel a waiting one, delete queued or finished jobs. Jobs run one at a time in queue order: queued → waiting → running → completed/failed.
  • Live progress per job and provider statistics (/jobs, /stats, /api/jobs, /api/stats). Each job's counters come from its own session.
  • Login: HTTP basic auth against its own passwd file (-webpasswd, users via -webadduser; proxy users get no web access). Without a login the server refuses to start on anything but a loopback address.
  • Brute force: after 10 failed logins within 10 minutes a client address gets 429 (with Retry-After) for 10 minutes, answered before any password check. Logins that already succeeded keep working, so users behind the same address (reverse proxy) aren't locked out.
  • Protection: state-changing requests need the X-NZBreX-Request: 1 header and a same-site Origin/Sec-Fetch-Site (CSRF); the Host header is checked against DNS rebinding (-webhosts). Uploads go to a private temp dir with unique names and O_EXCL.
  • Read-header and idle timeouts; a busy port or a bad certificate is reported at startup; SIGINT/SIGTERM stops the running job, cancels waiting ones and exits.
  • Per-job settings are shown read-only: jobs run with the startup flags, the config endpoints answer 501, and the docs say so (WEB_JOB_QUEUE.md, rewritten from the code).
  • Bootstrap is served from the binary (no CDN).
  • Runs on Go's net/http (no gin): binary 33.5 → 17.1 MB, third-party packages 114 → 27.

Web flags: -web, -webaddr (default 127.0.0.1:11911), -webssl / -webssl-crt / -webssl-key, -webpasswd, -webadduser 'user|password' (or auto|auto), -webhosts.

🛠 Engine fixes (races, hangs, leaks)

Every fix was checked with the race detector, most against the old build as well.

  • Data races: band-aid locks replaced with proper publishing of the segment list, provider list and channels; one owner closes each channel; progress counters are atomics; job data is copied, not shared. Web race test: 13 races → 0.
  • Hangs: a run no longer hangs on a dead or silent provider, a server that stops mid-run, a provider with MaxConns: 0, a 436 on IHAVE, or a -nzbdir watcher; blocking waits can be stopped; the NNTP handshake and CAPABILITIES have timeouts (a silent server hung > 180 s, now gives up at 105 s).
  • Leaks: memory slots (MemLimiter: one owner record per slot, no double return), connection counts, goroutines, the decoder pool (use after release in -yenctest=4), one log file per -log session.
  • Process-wide state: jobs no longer change -chansize, the yEnc CPU limiters or the log output for the whole process.
  • Crashes: a pre-login CAPABILITIES, -crc32 in the proxy, a nil connection pool in the stats, a retryIn computed from the seconds field instead of a timestamp.
  • Startup: broken mode combinations (-nzbdir with -web, the proxy with -web, ...) exit with an error instead of half-starting.
  • Finished runs: re-uploaded articles count as done (an upload marked them uploaded but not available, so every refresh run ended through a fallback close case); the web progress no longer adds uploads on top.
  • Download-only runs: new flags -dlmissing (articles missing on any provider) and -dlall (every article) download into the cache (-cd), also without an upload provider; -dlall -crc32 -yencout -yencmerge rebuilds the files. Without a flag such a run now ends cleanly with a hint (it used to download nothing and end in a safety exit).
  • Timeouts during a session: every provider connection has a 60 s idle timeout, so a provider that stops answering mid-command fails that command instead of blocking a worker forever; a slow but steady article is not cut off. Download and upload workers no longer quit after 15 s without work, which left retries (after another provider timed out) with nobody to run them.
  • -nzbdir works: it watches a directory (-dirrefresh, default 60 s) and processes its NZBs one at a time, oldest first, once a file is unchanged between two scans; afterwards the file moves to done/ or failed/. SIGINT/SIGTERM stops the running NZB, which stays for the next start.

⚡ yEnc decoding: pure Go by default, C optional

  • New decoder go-while/go-yenc-simd (own module): pure Go, AVX2 and AVX-512 (VBMI2) with GOEXPERIMENT=simd, plain Go otherwise. Same results as rapidyenc's C code, call for call (golden outputs, differential tests, fuzzing). Through NZBreX's full decoder it reaches 61–81 % of the C speed, far above download rates.
  • Two build variants:
    Variant Build -yenctest=4 decoder
    purego (default) go build . go-yenc-simd; no cgo, cross-compiles anywhere, static on Linux
    rapidyenc go build -tags rapidyenc . rapidyenc's C code; -yencgo switches to go-yenc-simd
  • -yenctest=4 logs the decoder in use. Fixed: -rapidyencbufsize had no effect.

🧱 Crafted NZBs and dependencies

📦 Build, CI and releases

  • Go 1.27.2. The rapidyenc C sources are a git submodule (animetosho/rapidyenc, pinned).
  • CI (ci.yml, every push and PR): go vet and go test -race for both variants with and without SIMD, purego cross builds for linux/windows/darwin amd64+arm64 and freebsd/amd64, and a native macOS build and test of the C variant.
  • Releases (release.yml, on v* tags): purego for all targets from one runner, rapidyenc for linux amd64/arm64, windows amd64 and macOS arm64; one publish job (the only one with write access) uploads everything with SHA256SUMS. scripts/build-release.sh builds and packs a target the same way locally: flat archives with README, LICENSE and samples, and a .deb named nzbrex.
  • Replaces build_and_publish_test.yml, whose "musl" build was linked against glibc, whose "purego" assets used cgo, and whose release had no checksum files.

🧪 Tests

New tests for the proxy, web routes (every route, login, CSRF, Host, uploads), job queue, auth, memory limiter, connection pool, teardown and the decoders; go test -race ./... passes for both variants. Local test setup and tools: tools/nntpseed (generates yEnc articles, including damaged ones, and feeds them to local servers) and nzbs/LOCAL_TESTING.md.

📝 Known issues (not fixed here)

  • Per-job settings in the web interface are not implemented yet (shown read-only, jobs use the startup flags).

🤖 Generated with Claude Code

Copilot AI review requested due to automatic review settings June 10, 2025 22:42

This comment was marked as outdated.

Sign in to view

@go-while
go-while requested a review from Copilot June 10, 2025 23:11

This comment was marked as outdated.

Sign in to view

go-while and others added 16 commits October 18, 2026 00:46
@go-while @claude
Give up on dead check groups and down providers instead of hanging
68ba5d7 
- Count live check routines per group; the feeder closes its channel and the
  divider quits ("checkersGone") once a group has none left.
- A failed STAT is retried once, then recorded as an error on the group so the
  item still counts as checked; a check routine quits after 3 items without
  a connection.
- Replace the -verbose-only watchdog (which killed progressing runs that had
  dead segments) with an always-on 3 min no-progress check that also counts
  failed attempts.
- GetConn failures in the downs/reups routines now flag the item, so it no
  longer stays "in flight" forever; uploads give up a provider after 3 fails,
  and the FlagError retry delay works (.Unix(), retryOn set).
- Failed uploads keep their mem slot: returning it deadlocked downloads and
  re-queued uploads on MemLockWait.
- After the stop, queued items are dropped and GetConn no longer dials.
- A provider whose pool gave up on dial errors is skipped for the rest of the
  session (never in proxy mode).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@go-while @claude
Fix CloseConn double decrement, bound KillConnPool, stop leaked gorou…
ce726f9 
…tines

- CloseConn: nil-safe; the open counters drop once per conn (a second close
  drifted openConns negative), the shared-conn token is always sent.
- KillConnPool: no global lock, bounded wait (5 s), only idle conns closed.
  The unused ConnPools map and PoolsLock are gone (and the lock inversion).
- Worker tick goroutines are time.Tickers; WatchOpenConnsThread and the
  per-pool Speedmeter stop when KillConnPool closes the pool's stop channel.
- readDotLines: early returns close the rapidyenc pipe / decodeBodyChan and
  wait for the decoder goroutine (with -yenctest=4 the decoder was released
  while still in use); the async decoder keeps draining after an error.

4 web jobs in a row: 122 goroutines left before, 10 after.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@go-while @claude
MemLimiter: one owner record per slot, no double return, no exit on a…
0c3bd6e 
… second lock

MemReturn checked memlocked and returned the slot in separate steps, so two
concurrent returns of one item handed back two slots (or hit log.Fatalf on a
full chan). memdata is now the single ownership record; it changes together
with item.memlocked under m.mux, and only once the slot is taken. A second
return is a logged no-op. A second MemLockWait on an item still waiting for
a slot waits for the first one instead of calling os.Exit(1).

MemAvail keeps its meaning (bound the downloads waiting for a slot): gating
on free slots cost 25% throughput with -mem 4. waiting is atomic so MemAvail
takes no lock, since pushDL calls it while holding item.mux.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@go-while @claude
Guard session preBoot/active with s.mux only
2b1bf9b 
GoBootWorkers wrote them under globalmux while LaunchSession and
closeSession use s.mux, and the web stats poll read active with no
session lock. All now use s.mux; the poll reads active together with
providerList. waitDivider.Done is deferred before the early returns.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@go-while @claude
Bound the NNTP handshake and CAPABILITIES; accept 201 greetings
23454e8 
connect() read the greeting and AUTHINFO replies with no deadline
(UseNoDeadline makes ExtendConn a no-op), so a server that accepts and
then stays silent blocked GetConn forever. The handshake now has a 30 s
deadline, cleared before the conn is handed out; a timeout counts as a
failed dial so the provider can be marked down for the session.

checkCapabilities held provider.mux across two unbounded reads. It now
has its own deadline and takes the lock only to store the result.

The greeting was read with ReadCodeLine(200): a 201 greeting came back
with an error and, without a login, the conn was leaked. Accept any 2xx.

clearSession read len(sessMap) without the processor lock.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@go-while @claude
Web: report bind errors, copy jobs instead of sharing them, no scans …
4f1aeb2 
…under locks

StartWebServer binds and loads the TLS cert before serving and returns the
error; main exits on it (a busy port used to leave the process running
without a web server).

The job queue moves to webjobs.go. Readers get copies and never write.
Each job creates and keeps its own session (SessionID is set), so nothing
matches sessions by path. GoWorkDivider publishes its per-pass counts in
s.progress; running jobs read them and finished jobs keep the final
counts, also when nobody polled. AddJob parses outside the lock and
rejects broken NZBs. collectStats no longer holds globalmux or walks
segment lists. New waiting state, panic recovery on the job goroutine,
queue caps, 404/409 status codes.

Per-job settings (D2): the config endpoints return 501 with the startup
settings, /upload ignores the fields, and the dialogs are read-only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@go-while @claude
Web login, CSRF guard, timeouts and safe uploads; shared passwd code
453edcd 
Login (D1): HTTP basic auth against its own -webpasswd file, users via
-webadduser. Without it the server only starts on loopback. Verified
logins are cached briefly, the file is reloaded when it changes.

The passwd parsing moves to passwd.go and is shared with the proxy, which
keeps its own map: a reload swaps in a new map only when it parsed and has
users (it used to clear the live map first), and unknown users cost a
bcrypt comparison like wrong passwords (timing oracle), expiry checked.

Every state-changing request needs X-NZBreX-Request: 1, a same-site
Sec-Fetch-Site and a matching Origin. Without a login the Host header is
checked against an allowlist (DNS rebinding).

Server timeouts, 128 MiB upload cap, no trusted proxies. Uploads go to a
private MkdirTemp dir under unique names, written with O_EXCL instead of
gin's SaveUploadedFile, and are deleted with their job. Bootstrap is
embedded (Debian libjs-bootstrap5 5.3.5) instead of loaded from a CDN.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@go-while @claude
Web: stop/cancel jobs, run them in queue order, graceful shutdown; re…
a690206 
…ject broken modes

Jobs ran in random order: each started job had its own goroutine racing
for a mutex. One runner now takes the oldest waiting job.

POST /api/jobs/:id/stop cancels a waiting job or stops a running one.
The engine now reacts to s.stop() before the work is done: the divider
quits (close case "stopped"), the check feeders stop and the check
routines drain without checking; the rest tears down as usual.

SIGINT/SIGTERM in web mode shuts the server down, stops the running job,
removes the upload dir and exits 0.

D3: -nzb with -web or the proxy, -web with the proxy, and -nzbdir (with
anything, or alone while it is unfinished) are rejected at startup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@go-while @claude
Proxy: deadlines, bounded lines, non-fatal errors, RFC fixes; no empt…
4a1042e 
…y articles on broken yEnc

Hangs and exhaustion (5.1): per-command read deadlines (60 s to log in,
5 min idle), write deadlines, 400 before closing, a cap of 64 connections
that haven't logged in, 512-byte command lines, upstream conns parked
before writing to the client, 64 MiB cap on buffered replies, a bounded
negative cache keyed by group for numbers, accept-loop backoff (EMFILE
used to spin at 100% CPU and log millions of lines), shutdown on signal.

Correctness (5.2): only QUIT, an expired login or a failed client write
close the connection; syntax errors and unknown commands are answered.
Every client write is checked. NEXT/LAST forward the upstream reply.
GROUP selects the first article, message-ID requests keep the group,
the pointer moves only on success, numeric replies carry the real
number and message-ID. Listeners return errors instead of log.Fatalf
and start only after the providers are loaded.

Shared readDotLines: a broken yEnc article returns bad crc (99932)
instead of success with no content (empty 220s to proxy clients, empty
articles to workers); a plain-text article in mode 4 is delivered as is.
The proxy relays articles unchanged (no Path/header rewriting).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@go-while @claude
Jobs no longer change process-wide state; rewrite WEB_JOB_QUEUE.md
cf4fd36 
- -chansize <= 0 is sized per session instead of written back to cfg.opt
- yEnc CPU limiters are set up once at startup, never resized mid-run
- -log redirects only for NZB sessions (no logs/.log for web/proxy) and
  closes the previous log file
- /api/stats lists the startup providers while no job runs
- WEB_JOB_QUEUE.md rewritten to match the code (plan 6.3, D2)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@go-while @claude
Base-session constructor, no lock on every park, queue counts before …
39fff72 
…the send

- newBaseSession builds the proxy/web sessions with all their fields
- ParkConn only locks for its debug line
- dl/up queue counts go up before the channel send and back down if it
  was full, so a fast worker can't decrement first and leave them at 1

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@go-while @claude
ydec: pure-Go yEnc decoder with Go SIMD (AVX2, AVX-512 VBMI2), opt-in…
c46ddd9 
… via -yencgo

- ydec.DecodeIncremental is a drop-in for rapidyenc_decode_incremental:
  a state machine matching rapidyenc's scalar reference, plus vector loops
  with GOEXPERIMENT=simd on amd64 (VPSHUFB tables on AVX/AVX2, VPCOMPRESSB
  on AVX-512 VBMI2); plain Go elsewhere
- tested against the C library: random and chunked inputs, encoded
  articles, sample files, fuzzing; rapidyenc's decoder tests and the
  broken-yEnc readDotLines tests also run with it
- rapidyenc.UseGoDecoder / -yencgo switch the Decoder's yEnc step only;
  C stays the default
- fix: -rapidyencbufsize had no effect (AcquireDecoder used a copy of
  DefaultBufSize made at init)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@go-while @claude
Use go-yenc-simd (own module) instead of the in-tree ydec package
f290ba1 
- ydec moved to github.com/go-while/go-yenc-simd (package yencsimd),
  imported via a replace to the sibling checkout until it is published
- the C cross-check tests and benchmarks move to rapidyenc
- go mod tidy merged the require blocks (no version changes)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@go-while @claude
Don't trust subject counts or names: segment list size, merge path; u…
11b6573 
…se subjectparser fork

- the segment list was sized by TotalSegments, the count the subjects
  claim: "(1/9999999999999999999)" panicked (makeslice), a large plausible
  count allocated gigabytes. Sized by the NZB's segments now (web job
  count too)
- -yencmerge joined the filename from the subject into the target path:
  "../../../x" wrote outside the cache dir. Only its last element is used
  now (safeFileName); unusable names are refused
- github.com/Tensai75/subjectparser is replaced by go-while's fork
  (../subjectparser, branch review-fixes): filenames of subjects without
  segment numbers, overflowing numbers, regexps compiled once

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@go-while @claude
Switch to javi11/nzbparser (fork with fixes), validate message-ids
77255c4 
- github.com/Tensai75/nzbparser -> github.com/javi11/nzbparser (active fork
  with its own subject parser), replaced by ../nzbparser (branch
  review-fixes) until the fixes are published or merged; NzbSegment.Id is
  ID there. The subjectparser fork is no longer imported.
- Segment ids are normalised to "<...>" and must be valid RFC 3977
  message-ids (printable ASCII, no spaces, at most 250 bytes) before they
  are used in NNTP command lines; invalid ones are skipped with an error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@go-while @claude
Use the published go-yenc-simd and nzbparser fork, no local replaces
bca90ce 
- github.com/go-while/go-yenc-simd is on GitHub: required at 50d56c3
  (pseudo-version), the replace to ../go-yenc-simd is gone
- github.com/javi11/nzbparser is replaced by the pushed review-fixes
  branch of github.com/go-while/nzbparser (7c20501, open as
  javi11/nzbparser#2) instead of ../nzbparser

A fresh clone builds without sibling checkouts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@go-while go-while self-assigned this Oct 18, 2026
go-while and others added 12 commits October 18, 2026 20:06
@go-while @claude
Pure Go by default, rapidyenc's C decoder behind -tags rapidyenc
3ab3c81 
- rapidyenc/decoder_c.go (cgo, the C DecodeIncremental, DecodeKernel) and
  encoder.go build only with -tags rapidyenc; without it,
  decoder_purego.go decodes with go-yenc-simd and nothing needs cgo
- State/End constants have fixed Go values, checked against the C headers
  in the C build (TestStateValuesMatchC)
- rapidyenc.Backend() names the decoder; -yenctest=4 logs it, -yencgo
  only matters in the C build
- C-only tests carry the tag; a test-only Go encoder serves the others
- local_build_linux-amd64.sh builds with -tags rapidyenc; README build
  section covers both variants

Both builds pass the race tests; CGO_ENABLED=0 cross-compiles to
linux/windows/darwin amd64+arm64 and freebsd/amd64.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@go-while @claude
New CI and release workflows for the purego and rapidyenc variants
856b13c 
- ci.yml (push, PR): go vet + go test -race for both variants, with and
  without GOEXPERIMENT=simd; purego cross-compiles for every target; a
  native macOS check of the C variant. Read-only permissions.
- release.yml (v* tags, or by hand for a tag): purego for linux/windows/
  darwin amd64+arm64 and freebsd/amd64 from one runner; rapidyenc for
  linux amd64/arm64, windows amd64, darwin arm64; one publish job (the
  only one with contents: write) uploads everything with SHA256SUMS via gh
- scripts/build-release.sh builds and packs one target (flat tar.gz/zip
  with README, LICENSE, samples; .deb "nzbrex" with docs in
  /usr/share/doc), the same locally and in CI
- build_and_publish_test.yml is removed: its musl build linked glibc, its
  "purego" assets used cgo, no checksums reached the release, the .deb
  broke Debian naming, and every matrix job wrote to the release

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@go-while @claude
Web server on net/http instead of gin
9761e03 
- routes with Go 1.22+ ServeMux patterns ("POST /api/jobs/{id}/start"),
  the Host/CSRF guard and the login as handler wrappers, writeJSON,
  pages rendered into a buffer, no directory listings for /static/,
  panic recovery (500), access log with -debug only
- same routes, JSON, status codes and content types; a wrong method is
  405 now (was 404)
- webserver_test.go covers every route, uploads, login, CSRF and Host;
  written against the gin router first, passes unchanged
- binary 33.5 -> 17.1 MB, third-party packages 114 -> 27

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@go-while @claude
CI/release: actions on Node 24 (checkout v7, setup-go v7, artifacts v…
9cbb613 
…7/v8)

checkout@v4 and setup-go@v5 target Node 20, which GitHub is retiring
(forced onto Node 24, with a warning per job). No breaking change applies:
setup-go keeps go-version-file, release.yml downloads all artifacts with
merge-multiple (not by ID), and download-artifact@v8 now fails on a
checksum mismatch instead of warning.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@go-while @claude
build-release.sh: no '~' in .deb file names
9d32151 
GitHub replaces '~' with '.' in release asset names, so the .deb names
in SHA256SUMS didn't match the uploaded files (v0.0.7-test-p1). The file
name now uses the tag's version (nzbrex_0.0.7-test-p1_amd64_purego.deb);
the Debian Version field keeps '~' (0.0.7~test~p1) so test releases sort
before the release.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@go-while @claude
release.yml: write the release text, target the tag's branch
0e98086 
v0.0.7-test-p1 got GitHub's generated notes (unrelated PRs merged into
main, compared against v0.0.5-rapidyenc) and targeted main. Now the text
is the annotated tag's message, the commits since the previous release
(next lower version tag, collapsed, at most 100), and
.github/release-notes.md (which file to download, how to check it). The
release targets testing when the tagged commit is on it, otherwise main.
A re-run updates the text and target too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@go-while @claude
Count uploaded articles as done
63da2d4 
A successful upload sets uploadedTo, not availableOn, but the divider's
done test needed availableOn on every provider. Uploaded articles were
never done: each pass counted them as "no upload target", and every
refresh run ended through the fallback close case #8.

Done is now onAllProviders(item): on every provider, confirmed there or
uploaded there by this session. availableOn keeps meaning "confirmed"
(pushDL picks download sources from it). The web's ArticlesDone is done
alone; it added uploaded to make up for this.

On the local servers: a normal run ends through #0 (was #8), one with
damaged articles through #6 (was #8); result tables unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@go-while @claude
-dlmissing and -dlall: download without an upload target
ae86dc3 
Download-only runs (-cd, -yencout, no provider taking uploads) never
downloaded: pushUP's "no target"/"not allowed" error made the divider
skip pushDL, and with -cd/-yencout the articles weren't retired either,
so the run ended in the "impossible closeCase" safety exit.

- -dlmissing: download the articles missing on any provider into the
  cache (e.g. to re-upload them later)
- -dlall: download every article not yet cached (fill the cache, or get
  the files with -crc32 -yencout -yencmerge)
- both need -cd, not together, not with -checkonly
- pushUP only runs for items with an upload target; -dlall keeps done
  items until downloaded, then returns their mem slot
- without a flag such articles are retired and the run ends cleanly,
  with a hint naming the flags

Local servers, both NoUpload, 60 articles (15 missing on one): no flag
ends in 5 s with the hint; -dlmissing caches 15; -dlall with -yencmerge
caches 60 and merges 3 files matching their yEnc size and CRC32. Normal
re-upload runs are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@go-while @claude
Web login: limit failed logins per client address
655b0dc 
After 10 failed logins within 10 minutes an address gets 429 with
Retry-After for 10 minutes, before any bcrypt check. A success resets the
count. Logins already verified (cached) pass during a block, so other
users behind the same address (reverse proxy) keep working. The table of
addresses is capped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@go-while @claude
Idle timeout on provider connections; workers no longer exit while wo…
9f47dd7 
…rk can return

- every provider conn is an idleConn: each read and write gets a fresh
  60 s deadline, so a provider that stops answering mid-command fails
  that command instead of blocking the worker forever, while a slow but
  steady article is not cut off. Off during the handshake and the
  CAPABILITIES exchange, which have their own deadline
- download and upload routines exited after ~15 s without work; a retry
  that came back later (after another provider timed out) found no
  worker, and the run ended with that work undone. They now end when the
  session closes their channel or stops

Against a server that answers STAT but stalls every ARTICLE, with a
second download source: the old build was still blocked after 10 min;
now the run finishes in 71 s with every missing article copied.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@go-while @claude
-nzbdir: process the NZBs that appear in a directory
87b4ba6 
Every -dirrefresh seconds (default 60, minimum 15) the watcher takes the
.nzb/.nzb.gz files that are unchanged since the previous scan (not ones
still being copied in), oldest first, runs each in its own session and
removes the session afterwards. Then the file moves to done/, or failed/
if the run couldn't start; a name already there gets a timestamp.
SIGINT/SIGTERM stops the running NZB, which stays for the next start.

The old watchDirThread/refreshDir/processorThread stubs are gone (the
scan never ran: nothing set its interval). -nzbdir alone is allowed now;
with -nzb, -web or the proxy it is still refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@go-while @claude
Merge main into testing: keep build_and_publish_test.yml deleted
8d8ef22 
main changed the old workflow (PRs #75, #76: Go 1.25.3, fewer steps);
testing replaced it with ci.yml and release.yml (Go 1.27.2), which cover
those changes. Resolves the conflict in #62.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@go-while
go-while merged commit cc9fe68 into main Oct 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments

Assignees

@go-while go-while

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@go-while @github-advanced-security