Skip to content

Direct, account and hashtag timelines as two queries - #2174

Merged
karlitschek merged 2 commits into
masterfrom
perf/two-query-timelines
Sep 14, 2026
Merged

Direct, account and hashtag timelines as two queries#2174
karlitschek merged 2 commits into
masterfrom
perf/two-query-timelines

Conversation

@karlitschek

Copy link
Copy Markdown
Member

Item 10 of the fresh look. Performance.md's first "what to do next" said the timelines other than home and public still made the database sort or hash the whole post to deduplicate a page of twenty — and named notifications first. Reading the code: notifications, favourites, bookmarks and the list timeline had already moved. Direct, account and hashtag had not; this moves them.

What changes

Each becomes a *TimelineNids() method that decides the page over s.nid — with every filter the old query applied (type, media, recipient, hidden and silenced actors, the viewer's follows for the hashtag page) — and streamsByNids() reads exactly those rows, with the author, the viewer's actions and the boosted object joined:

  • direct — no DISTINCT: the recipient join fixes the viewer and the type dm, one row per post.
  • accountDISTINCT only when the account reads its own profile (the recipient join then names no recipient and a post to several accounts would repeat); the public page is not.
  • hashtagDISTINCT over one integer, where it used to be over content, source, details, cache, tags and to_array.

docs/Performance.md is corrected where it had gone stale: the two "still wants a transaction" items (StreamActionsRequest::save(), ModerationRequest::save()) were already insert-then-catch-then-update; every getTimeline() branch is now on the two-query path; what is left of the wide SELECT DISTINCT is named (searchContent(), the thread walk, getAnnouncesAndRepliesTo(), and the *_dep() methods behind the uncalled Custom Local API routes), and "what to do next" is rewritten accordingly.

Verification

  • tests/Db/TwoQueryTimelinesTest.php (7 cases) pins the shape for the three: the wide read is handed the page and nothing else, an empty page makes no wide read, an account timeline without an account asks nothing.
  • The SQL of each page is exercised by the integration suite in CI — TimelineSeedTest, StreamFilterTest, MediaTypeTimelineTest, OnlyMediaTimelineTest, DomainBlockTimelineTest, SilencedInstanceTimelineTest, BoostVisibilityTest all read these probes against a real database. That is the run to watch on this PR.
  • Full unit suite green (4624), psalm no errors, php-cs-fixer clean. Version 0.19.38.

🤖 Generated with Claude Code

Frank Karlitschek and others added 2 commits September 14, 2026 08:24
The last three timelines still asked the database for the page and the
rows in one statement: a SELECT DISTINCT over the whole post -- content,
source, details, cache, tags, to_array -- plus the joined author, sorted
or hashed in full to choose twenty rows. Home, public, favourites,
bookmarks, notifications and the list timeline had already moved to
deciding the page over the one indexed column and reading exactly those
rows afterwards; these follow, each as a *TimelineNids() method feeding
streamsByNids().

Direct needs no DISTINCT (the recipient join fixes the viewer and the
type, one row per post); the account page is DISTINCT only for the
account reading its own profile, where the recipient join names no
recipient; the hashtag page is DISTINCT over one integer where it used to
be over the post.

Performance.md is brought up to date: its two "still wants a transaction"
items were already done (StreamActionsRequest::save() and
ModerationRequest::save() both insert-then-catch-then-update), every
getTimeline() branch is on the two-query path now, and what is left of
the wide SELECT DISTINCT is named.

Signed-off-by: Frank Karlitschek <frank@nextcloud.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Frank Karlitschek <frank@nextcloud.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@karlitschek
karlitschek force-pushed the perf/two-query-timelines branch from 3404b94 to 6ca0e05 Compare September 14, 2026 06:24
@karlitschek
karlitschek merged commit 6fcc5a8 into master Sep 14, 2026
43 checks passed
@karlitschek
karlitschek deleted the perf/two-query-timelines branch September 14, 2026 06:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant