testing.md

Testing

Rill's checks cover storage guarantees, native interactions, provider contracts, and packaged applications. Use isolated data directories. Public-feed checks need real subscriptions; fictional demo URLs cannot establish refresh, extraction, or original-link behavior.

Automated checks

Run scripts/check.sh for the complete workspace checks, or use its commands individually:

python3 -B -m unittest discover -s scripts/tests -v
python3 scripts/notices.py /tmp/rill-dependency-licenses.txt
cargo fmt --all -- --check
cargo test --locked --workspace
cargo clippy --locked --workspace --all-targets -- -D warnings

For a focused run, use cargo test --locked -p rill-core or cargo test --locked -p rill-desktop. Tests create temporary libraries and local fixtures. Provider tests use fake executables and local endpoints without live sign-in or paid generation.

Suite	Behavior covered
scripts/tests/test_notices.py	License discovery, missing-text failures, version and hash checks, path boundaries, and preserving an existing inventory on failure
scripts/tests/test_packaging.py	Linux archive contents and preserving the previous artifact when notices fail, using fixture binaries and Cargo metadata
Core src/newsletters.rs	IMAP greetings, read-only polling, oversized-message skipping, and persisted UID checkpoints
Core tests/library.rs	Reading-state persistence, undo, rules, search, cursor paging, unsubscribe recovery, preferences, OPML, backups, and migrations
Core tests/feeds.rs	HTML conversion, conditional refresh, offline retention, coalesced preparation, and cached full text
Core tests/integrations.rs	Digests, schedule deduplication, delivery retries, provider isolation, and newsletter import
Core tests/protocols.rs	Authentication, revocation, pagination, filter intersections, and idempotent state changes
Desktop palette/tests.rs	Native result geometry, group transitions, and keyboard activation
Desktop reading/tests.rs	Sidebar layout, pagination, preparation, stable Unread rows, pointer selection, and clipboard behavior
Desktop settings/tests.rs	Draft preservation, validation, save acknowledgments, optional fields, and rule editors
Desktop session/tests.rs	Close/reopen, stale or malformed state, scroll restoration, and cancellation

Core test paths are relative to crates/rill-core; desktop paths are relative to crates/rill-desktop/src. Add behavior tests through public storage or engine operations for persisted changes. A migration test must reopen an older schema with articles and reading state intact. Avoid assertions about source text or private call counts.

GPUI tests dispatch native component events and inspect layout under a test platform. They complement OS interaction checks; they do not establish hardware input, fullscreen transitions, display presentation, or screen-reader behavior.

Prepare a native test library

cargo build --locked --release --workspace
target/release/rill-cli --data-dir /tmp/rill-test-library add https://blog.rust-lang.org/feed.xml --folder Engineering
target/release/rill-cli --data-dir /tmp/rill-test-library add https://jvns.ca/atom.xml --folder Engineering
target/release/rill-cli --data-dir /tmp/rill-test-library add https://simonwillison.net/atom/everything/ --folder Reading
target/release/rill-cli --data-dir /tmp/rill-test-library refresh
target/release/rill --data-dir /tmp/rill-test-library --window-size 980x640

Choose a fresh directory and also check a separate empty library. Feed contents change; verify ingestion and readable content without relying on a fixed title or count. Keep screenshots, exported fixtures, and temporary libraries outside the repository.

Use a complete SQLite backup if a reproduction needs an existing library. Configure the copy before launching its desktop or daemon so saved schedules and providers cannot use personal accounts. Recovery explains what a backup contains.

Native interface

Inspect the rendered window and accessibility tree together. Cover 980 × 640, the default 1380 × 860, fullscreen, visible and hidden sidebars, and both themes. Include an empty reader, a populated list, a long article, and a utility page.

Layout and navigation

• Native window controls and the sidebar toggle align in a window. Fullscreen removes the native-control inset. The hidden-sidebar toggle and utility-page back action stay reachable.
• Sidebar rows have equal width, aligned source and age, and two title lines. Search, the picker, and footer remain fixed while the article list scrolls. No controls overflow at minimum width.
• The native picker filters views, folders, and feeds. Its trigger keeps the committed scope until Enter confirms a choice. Escape dismisses it; confirming the current scope preserves position.
• Search works over the reader, forms, and empty-library Settings. Check commands, articles, feeds, no results, clearing, and rapid queries. Rows stay separate even when earlier groups are empty. The panel shrinks for empty results while the input stays anchored.
• Arrows and Enter activate the expected result. Tab stays in the palette, background shortcuts remain inactive, and dismissal restores focus and unfinished form text.
• Page Up/Down and Home/End scroll the visible article or utility page. Space/Shift-Space apply while reading. Article changes reset reader scrolling; settings-section changes reset page scrolling; search preserves both.

Reading, loading, and selection

• Toggle read, star, and read-later; undo explicit changes; refresh and restart. Check stored flags and both read-on-open preference settings. The star icon and its next-action label agree in the list and toolbar.
• Click Unread rows above and below one another at the top, middle, and bottom, in both orders. Completed rows stay under the pointer while counts update. J/K revisits them. Mark view read clears the view; Undo restores unread rows. Returning to Unread excludes completed articles.
• Use a feed with more than 160 articles. Near each page end, another page appends without duplicates or a jump. A scope change discards an earlier pending page. New arrivals preserve the first visible article's pixel position.
• Check recent and old posts. Unread includes only the past 30 days; feed views and search retain older unread history. Marking Unread read leaves old history unchanged.
• Sweep over rows, pause, then open an article. Brief passes do not queue a preparation backlog, and preparation does not mark unopened articles read. Offline or failed extraction keeps cached content readable. Prepared full text remains available when the source is unreachable.
• Compare a cold article before and after loading at minimum width and in both themes. Title, metadata, row bounds, and body start stay fixed. A late response preserves manual reading-state changes and the selected article.
• Drag across body, title, and author text. Highlights update before mouse-up, glyphs remain visible, and copy contains exactly the selection. Check paragraphs, headings, lists, links, code, Unicode, long titles, and sanitized media fallbacks.
• Press a row, move away, and release. Cancellation leaves the article unchanged. A normal click opens on release.
• Copy an article URL with the button and C. Its acknowledgment stays within the same bounds for two seconds. Repeating copy extends the acknowledgment; an article without a URL leaves the clipboard unchanged.

Pointer auto-scroll

Click an article title, release, scroll the list away, and move the pointer into the reader. Stop input for several seconds. The list must remain still. Repeat near both window edges and after selecting title or author text.

Drag a selection toward the lower edge of a long article. Scrolling continues while the button is held and stops on release. Moving near the edge afterward must not restart scrolling, and the selected text remains copyable.

Keep both the released-pointer and active-drag GPUI regressions when replacing the dependency patches. The tests dispatch pointer and wheel events, then advance executor time without additional input.

Forms and utility pages

Area	Checks
Feed forms and management	Draft survives search; useful invalid-URL and refresh errors; folder retained on edit; bulk controls match selection; unsubscribe and resubscribe preserve cached history
Preferences	All nine sections, compact-window scrolling, unchanged save disabled, committed Saved state, invalid values retained, focus after removing an input, and drafts retained across section changes
Pending saves	Edit during a save and leave/reopen Settings during another. A late receipt cannot acknowledge or overwrite a newer draft. Rule saves preserve preference edits.
Optional settings	Enable, edit, save, disable, and re-enable schedules and client connections. Saved configuration and hidden drafts survive; validation applies when the feature is active.
Rules	Default save with no typing, inferred scope, optional filters, empty-filter validation, edit, cancel, and navigation during a pending save. A finishing save cannot close a newer editor.
Briefings	Empty state, no recent articles, local generation, readable excerpts, and article links
Other pages	Newsletter import, OPML transfer, backups, connection keys, keyboard help, and the separate news window. Verify exported files through their contents and reopen behavior.

Back and Escape return to the existing article and scroll position, including with the sidebar hidden. Utility pages exclude reading-only shortcuts. External mail and model checks use the separate procedure below.

Close and reopen

Omit --window-size when testing saved geometry. Use both Quit and the native main-window close button.

• Reopen an empty library and a populated library with no selection. Neither selects or marks an article read; Add feed and Search remain usable.
• Load several pages, select a long cached article, and scroll both the list and reader. Reopen with the same article, loaded history, visible-row anchor, reader offset, sidebar state, size, and fullscreen state.
• Mark the selected article unread before closing. Reopening preserves that flag and the retained Unread visit rows.
• Reopen a scrolled Settings section. Its section and offset return while unsaved inputs remain unapplied. Repeat with the sidebar hidden and a news window open.
• Check missing, malformed, oversized, unsupported-version, and stale article/feed session references. The app falls back without changing reading flags or making the library unusable.
• Navigate or scroll while native document parsing is pending. A delayed restore cannot override that input. Closing during startup cannot overwrite a valid session with the temporary loading view.
• Start with an explicit window size and confirm it overrides saved geometry.

External services and clients

Use dedicated test accounts and synthetic content. Record the actual provider, CLI, client, and OS versions with integration results. Configuration checks alone cannot establish sign-in, delivery, or client interoperability.

Integration	Checks
AI providers	First test arguments, stdin, output decoding, failures, and deadlines with a fake executable or local endpoint. Then verify sign-in and generation with the configured provider. Setup checks must not start a session or send content.
SMTP	Deliver to a controlled recipient, check TLS and authentication failures, retain failed outbox entries, retry, and verify delivery after restart. Account for possible duplicates after remote acceptance.
IMAP	Import from a dedicated folder without changing flags; repeat a poll; exercise UIDVALIDITY changes, duplicates, oversized or unreadable messages, and .eml recovery.
Sharing	Verify URL/title payloads, authorization, failure backoff, restart, and deduplication when an article is removed and saved again.
RSS clients	Initial sync, subsequent refresh, unread counts, stars and read state in both directions, folder changes, paging, and key revocation. Check the selected client's features against Compatibility.
Scheduler and listener	Two processes sharing a library, single scheduler ownership, takeover after exit, listener enable/disable, missed schedules, and retry limits. Use configured test services for any side effects.

Protocol checks must include malformed authentication, revoked credentials, browser Origin headers, cross-site requests, unexpected Host values, unsupported methods, and oversized request bodies. Parser and URL checks must preserve response limits and sanitization.

Platform and release checks

Build and package through Development. Validate the actual artifact on each target OS and architecture.

• macOS package checks: property-list validity, executable launch, strict ad hoc signature verification, and the architecture-specific archive used by the Homebrew cask. Native checks also cover window close/reopen, fullscreen, keyboard and pointer input, text copy, and VoiceOver.
• Linux: archive contents, desktop entry, bundled fonts and licenses, CLI operation, startup, close/reopen, keyboard and pointer input, and Orca. Exercise X11 and Wayland with real graphics drivers. Xvfb covers headless startup, not hardware input or compositor behavior.
• Both platforms: migration from an older library, restore from backup, empty startup, long reading sessions, large newsletters and briefing histories, and performance workloads from Performance.

Keep test logs, screenshots, measurements, and pass/fail results with the pull request or release artifacts. Include the build revision, OS, architecture, profile, fixture, exercised flows, and skipped checks. Reference guides describe behavior and reproducible procedures; they do not accumulate run histories.