Skip to content

The old "Backend ported to RUST" cliche#148

Open
jakeuribe wants to merge 21 commits into
mainfrom
dev
Open

The old "Backend ported to RUST" cliche#148
jakeuribe wants to merge 21 commits into
mainfrom
dev

Conversation

@jakeuribe

Copy link
Copy Markdown
Collaborator

Backend moved to rust. Feature dump.

  • Reading lists
  • P2P sync - most basic possible functionality, read-only copy of peer-project, no two-way sync, no credential rejection without deleting project, use at your own risk, but probably don't use yet, connects to Iroh continuously instead of lazily for now.
  • Minor bug fixes
  • Much, much faster CLI, MCP
  • Adding an RSS feed to homepage
  • New beautiful, relaxing theme added, called "Reading Room"
  • Updating your app in place!!
  • Annotating and commenting on pdfs tracked alongside notes in PaperDetails page
  • Minor, non-intrusive deduping functionality via DOI
  • Maybe more I forgot,
  • No db breaking but will need a working migration to adopt.

For all of these I basically just scoped the actual backend functionality and didn't make the gui pretty. Gui is prettier though! It uses whitespace a lot better and it's a lot easier to take notes while reading a pdf without opening the system viewer.

Backend moved to rust. Feature dump.
@jakeuribe jakeuribe requested a review from AndreFK July 11, 2026 00:10
jakeuribe and others added 17 commits July 13, 2026 13:04
IS_READING_LIST folded into PROJECT.sql's base def. PAPER_TO_READING
gets a composite FK -> PROJECT_TO_PAPER ON DELETE CASCADE, so removing
a paper from a project clears its reading status instead of it
resurrecting on re-add.

Fixing this exposed a deeper bug: the dedup needed for the new unique
index can't run from inside migration 6 once apply_tables creates the
composite-FK PAPER_TO_READING table - fixed but.

save_source_fks's blanket-delete-reinsert is documented + regression-
tested as a landmine
.slice(0, 50) ran after the filter, so a query matching >50 authors
still silently hid the rest. Cap now applies only in browse mode;
typing shows every match, with a hint when the cap is active.
dev_install_guard already took its value as a param, not via env -
no change needed there. registration_state couldn't distinguish
"not installed" from "installed, config unparseable"; now a 3-tuple
with config_error, threaded through MpcClientStatus and rendered as
a distinct "config unreadable" badge in Settings instead of silently
looking identical to never-installed.
cargo check --workspace fails on a fresh checkout since tauri-build
validates sidecar/resource paths at compile time. Documented
fetch_pdfium.sh + stage_rust_bins.sh in README; a build.rs placeholder
stub was rejected since it can't tell cargo check from a real bundle
build and would silently ship broken sidecars.
Setting existed but wasn't checked correctly on backend-side.
refactor sweep: cut ~2k dead/duplicated backend lines

remove all ponytail markers; ledger lives in docs repo

restore service::author/tag filter+link functions cut by code-reduction
sweep

cargo fmt
* fix: suffix-strip bug, shortcuts test imports

stripTrailingSuffixes stripped every trailing token instead of one,
and ran on the "Last, First" comma branch where it shouldn't.
shortcuts.test.ts failed under node's native TS runner: missing .ts
extensions on relative imports, and ApiError's parameter-property
ctor needs --experimental-transform-types (not just --strip-types).
Wires npm test into ci.yml; bumps CI node 20->22 (required for the
new flag). Also corrects a misleading comment on the stacked-suffix
("Jr. III") case pinned by an existing test - behavior unchanged.

* fix: reading-list FK cascade + boot crash

IS_READING_LIST folded into PROJECT.sql's base def. PAPER_TO_READING
gets a composite FK -> PROJECT_TO_PAPER ON DELETE CASCADE, so removing
a paper from a project clears its reading status instead of it
resurrecting on re-add.

Fixing this exposed a deeper bug: the dedup needed for the new unique
index can't run from inside migration 6 once apply_tables creates the
composite-FK PAPER_TO_READING table - any DML against the unindexed
parent fails immediately with "foreign key mismatch", including the
dedup DELETE itself. Restructured init_db to run a pre-schema dedup
pass before apply_tables; migration 6 now only creates the index.
Regression test boots a legacy DB with duplicate memberships through
init_db and confirms it dedups and starts cleanly.

save_source_fks's blanket-delete-reinsert is documented + regression-
tested as a landmine (currently inert - only caller is insert_project).

* fix: merge-picker cap hid matches past 50

.slice(0, 50) ran after the filter, so a query matching >50 authors
still silently hid the rest. Cap now applies only in browse mode;
typing shows every match, with a hint when the cap is active.

* fix: registration_state errors, surface in UI

dev_install_guard already took its value as a param, not via env -
no change needed there. registration_state couldn't distinguish
"not installed" from "installed, config unparseable"; now a 3-tuple
with config_error, threaded through MpcClientStatus and rendered as
a distinct "config unreadable" badge in Settings instead of silently
looking identical to never-installed.

* docs: document sidecar-binary bootstrap step

cargo check --workspace fails on a fresh checkout since tauri-build
validates sidecar/resource paths at compile time. Documented
fetch_pdfium.sh + stage_rust_bins.sh in README; a build.rs placeholder
stub was rejected since it can't tell cargo check from a real bundle
build and would silently ship broken sidecars.

* test: pdfium spike tests skip instead of panic

Replaced the panicking fixture loader with one that checks
bind_pdfium() and the fixture file first, skipping with a message.
Verified both paths: clean skip without libpdfium, real extraction
exercised when present.

* fix: version save + record now one transaction

Exposed write_paper_version_in_tx; version_monitor's save+record now
run in one transaction instead of two ordered-but-separate calls.
New test forces a mid-transaction FK violation and asserts rollback,
not just an ordering check.

* fix: saved notes render as markdown, not text

NoteBody rendered via MathText directly, so saved notes didn't match
the editor's Preview tab. Now delegates to NoteMarkdown, with a new
forceInline prop so NoteCard's clamped preview and NotePage's full
view keep their existing math behavior.

* fix: restore picker filter, backup date to UTC

Restore picker's "All files" filter defeated the .db-only intent -
removed. Backup filename switched from local-time to UTC date
accessors so it's consistent regardless of timezone.

* feat: enforce pdf_save_limit_mb as total storage

Setting existed but was never checked backend-side. Enforced as a
total-storage cap (matching the Settings UI's "Total PDF storage /
maximum combined disk space" description, not a per-file limit):
import_pdf and download_pdf sum the managed PDF directory and reject
a write that would push the total over the configured limit, checked
before any network fetch or PDF parse. DI'd rather than each module
reading config itself; reuses the existing PdfTooLarge error variant;
saturating_mul avoids overflow on a hand-edited settings value.

* better code

* ponytail sweep: cut ~2k dead/duplicated backend lines

drop ponytail comments promoted to docs TODO

remove all ponytail markers; ledger lives in docs repo

* restore service::author/tag filter+link functions cut by code-reduction sweep

* cargo fmt

* shared projects page: design-derived /shared view over Phase-0 share API

* vendor linxiv-p2p crate into workspace (crates/p2p, verbatim @ 54dfb88)

* swap share transport to vendored linxiv-p2p: persisted device key, published-only access check, feature passthrough

* wire linxiv-p2p W2+W3: stable uuids (share/note/annotation), evolving CRDT lineage, mirror import via service layer, per-share sync controls + interval sync, /shared page controls

* e2ee project sharing: keyhive membership + beelay sync + PDF blobs (W4+W5)

Vendored-crate: keyhive state persistence (archive + projects registry),
GroupId in invites, device-binding hellos with per-project accept gate,
disk-backed beelay KV + registry, FsStore blobs behind a membership-gated
blobs ALPN, single-endpoint bind_stack for sync+beelay+blobs.

Share crate: e2ee docs under share/e2ee[/received] (location is the mode),
publish_secure/invite/revoke/member-code/sync_e2ee/accept_invite wrapper,
blob store/read with caller-enforced quota.

App: e2ee legs in the 5-minute interval sync, invite/members/revoke/
member-code/publish-secure/shared-pdf routes, members sidecar with live
role truth-check, e2ee unpublish (revokes all members) and leave, PDF blob
population with post-grant re-key (#136 workaround), storage-cap-gated PDF
downloads. Frontend: E2EE publish option, members section with two-step
invite, revoke, per-share PDF download, honest sync/revoke copy.

auth-keyhive + sync-beelay are now enabled on the app's dep line; the
crates keep their feature gates.

* feat(p2p): read-role peers get serve-only sync

Read peers sync against a session-local scratch core (post-flush KV
clone, no disk mirror) so their uploads evaporate instead of reaching
the canonical store or other members. Relay/unknown refused. UploadBlob
frames hard-rejected for all roles (upstream todo!() panic DoS).

* feat(p2p): set_role with single-delegation invariant

Upgrade = revoke + re-grant (no eager rotation); downgrade = revoke +
force_pcs_update + re-grant. Doc-level query_access digest-orders
multiple delegations instead of taking the max, so every change path
keeps exactly one live delegation per member. Last-reader revoke maps
to a typed error.

* feat(p2p): scope blobs per project, 3-state host field

Blob-hash->doc map gates every fetch by the member's role on the
blob's own project; unmapped blobs and Relay denied; announced dials
pin the connection to one project. Registry host field distinguishes
Hosted / Member / MemberBadTicket so an unreadable host ticket surfaces
a typed rejoin error instead of silently acting as host. Old Option
registry layout decodes wire-compatibly (variant order load-bearing).

* feat(p2p): optional DEK encryption for key files

state.bin v2 and device.key gain XChaCha20-Poly1305 wrapping behind a
new encrypted-store feature (dek: Option<&[u8;32]>; None = plaintext
for tests/CI). One-time migrations re-persist v1/plaintext files
encrypted; wrong-DEK and locked-store are typed errors. App-side
keyring wiring lands separately.

* role management, viewer read-only, sealed keys (W6 app)

- POST /api/share/{id}/member/{mid}/role: hoster-gated role change
  (viewer/editor only), sidecar + blob re-key + republish; typed 4xx
  instead of substring matching
- accept_invite validates doc share_id before the mirror persists
- DEK from OS keychain (fallback: LINXIV_P2P_PASSPHRASE via Argon2id,
  else plaintext + warning) seals device.key, auth.key, state.bin
- viewer shares render read-only (affordances hidden) with fork to
  local library; members list gains role dropdown

* fix: read DEK back after first-run keychain mint

Two racing first launches could each mint a DEK, seal the key files
under the loser, and leave every share unrecoverable. Seal only under
the value the keychain actually stores.

* fix(p2p): close review races in role + blob persistence

- transition_lock serializes set_role/revoke against encrypt, and
  re-grant now precedes rotation, so a concurrent inbound-dial flush
  can no longer seal a commit in an epoch the downgraded member never
  gets (permanently partial mirror)
- blob->doc map rides inside kv.bin as one atomic tmp+rename payload
  (legacy two-file layout still loads); store_blob persists the
  mapping before the blob so a crash never strands an unmapped,
  unservable blob under unmapped=deny

* cargo fmt after w6-app merge

* cargo fmt: finish formatting sweep after w6-app merge

* p2p git module added

* ci: fetch git submodules for rust job

linxiv-p2p is a submodule of src-tauri/crates/p2p; checkout wasn't
fetching it so cargo failed to load the workspace manifest.
The uname case statement only matched Linux/Darwin, so a fresh Windows
checkout hit "unsupported host" and exit 1 — which blocks cargo check/build
entirely since tauri.conf.json validates the vendor/pdfium/lib resource path
at compile time. Adds a MINGW/MSYS/CYGWIN x86_64 case pulling the pinned
release's pdfium-win-x64.tgz (sha256 verified) and stages its bin/pdfium.dll
into lib/ to match the layout the other platforms ship and that
pdfium_lib_path() in pdf_metadata.rs expects.
…ch backend instead of just doing what works fastest
@jakeuribe jakeuribe marked this pull request as ready for review July 15, 2026 19:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants