Traces the full journey of a feed item — from the first remote fetch through local merge, retention pruning, disk persistence, and dashboard display. Covers what is actually stored, when items are permanently deleted vs. visually filtered, and how retention setting changes affect stored data.
Remote RSS/Atom feed (XML)
│
▼
┌─────────────┐
│ FETCH │ HTTP request to the feed URL
└──────┬──────┘
│ raw XML text
▼
┌─────────────┐
│ PARSE │ Extract title, guid, pubDate, content, media type…
└──────┬──────┘
│ new FeedItem[]
▼
┌──────────────────────────────────────────────────┐
│ MERGE │
│ │
│ new items ──────────────────────────► merged │
│ list │
│ existing items not in new fetch │
│ (carry-forward) ──────────────────────► │
└──────────────────────┬───────────────────────────┘
│ combined FeedItem[]
▼
┌──────────────────────────────────────────────────┐
│ RETENTION PASS 1 (inline) │
│ │
│ Carry-forward candidates are pre-filtered: │
│ drop unprotected items older than cutoff │
│ before they enter the merged list │
└──────────────────────┬───────────────────────────┘
│
▼
┌──────────────────────────────────────────────────┐
│ RETENTION PASS 2 (applyFeedLimits) │
│ │
│ Time-based: drop read items older than cutoff │
│ Count-based: drop oldest non-protected items │
│ beyond the max-items limit │
│ │
│ Protected (starred / saved) → always kept │
└──────────────────────┬───────────────────────────┘
│ trimmed FeedItem[]
▼
┌─────────────┐
│ PERSIST │ saveSettings() → data.json
└──────┬──────┘
│
▼
┌─────────────┐
│ DISPLAY │ Dashboard reads from in-memory settings;
│ │ view filters (unread / starred / folder…)
│ │ are applied on top — no additional items
│ │ are deleted at display time
└─────────────┘
Key point: retention happens before the write to disk, not at display time. Items removed by retention are gone from data.json after the next save.
The plugin saves its entire settings object to .obsidian/plugins/obsidian-rss-dashboard/data.json. This includes:
- All feed configuration (URL, folder, per-feed retention settings, scan interval, keyword rules, etc.)
- All article items (
FeedItem[]) for every feed - Global plugin settings (display preferences, tag definitions, highlight words, sidebar state, icon ordering, etc.)
Not stored: raw RSS/XML documents. There is no hidden local XML archive.
- On startup: the entire settings object is loaded into memory.
- On every save:
saveSettings()overwritesdata.jsonwith the full current in-memory state. - Single-feed refresh: saves immediately after that feed's merge + retention step.
- Multi-feed batch refresh: one save at the end of the entire batch, not per-feed.
- While a multi-feed batch is running, completed feeds update the existing global progress label in place. The sidebar and article list are not rebuilt between feed completions, and image-cache completions produced by the batch do not schedule extra dashboard renders. The batch performs one final dashboard refresh after all feed attempts settle.
SQLite was trialled in 2.2.0-beta.2 and reverted in 2.2.0-beta.4 for cross-platform stability. JSON has been the sole persistence mechanism since then.
Each refresh cycle:
- Sends an HTTP request to the feed URL.
- Parses the raw XML into structured
FeedItemobjects (title, guid, pubDate, content, media type, cover image, etc.). - Passes the parsed items into the merge step.
Feeds with excludeFromRefresh: true are skipped by bulk and automatic refresh. They can still be refreshed directly from a feed view.
Automatic refresh uses one rearmable timeout. When at least one non-excluded
feed has scanInterval: 0 or no value, the global interval schedules one
all-eligible-feed refresh batch; its next run is anchored to
lastGlobalRefreshCompletedAt. scanInterval: -1 turns automatic scheduling
off for that feed, while a positive value uses that feed's own completion time
and interval. This means a feed with a positive custom value can still refresh
when the global interval is Off.
lastRefreshAttemptCompletedAt is persisted per feed after every completed
attempt, including parser-reported failure, thrown error, and timeout.
lastUpdated remains the timestamp of the most recent successful parse. The
next due time is derived in memory from the completion timestamp and effective
interval; it is never persisted. A changed feed URL starts a new refresh
identity and clears completion and error history.
lastGlobalRefreshCompletedAt is a separate persisted settings value. It
advances only after an explicit refresh of the complete eligible feed set has
settled, including batches with individual failures. It does not advance for
single-feed, folder, selection, due-subset, failed-only, empty, or
excluded-only refreshes. The legacy lastRefreshTimestamp remains readable for
older data but is not used or updated by refresh behavior or status UI.
When the user stops an automatic global batch, its completion timestamp remains unchanged. The in-memory scheduler defers the next automatic global attempt by one configured global interval, preventing an immediate retry of the stopped batch while preserving the recorded completion time.
The merge step combines two sources:
| Source | What it contains |
|---|---|
| Freshly fetched items | Whatever the remote feed returned this cycle |
| Carried-forward items | Existing locally-stored items that were absent from this fetch |
Carry-forward exists so local history is not wiped every time a feed server stops returning older items. The local list accumulates over time — up to the retention limits.
The merge actually has three distinct paths:
For each item guid seen in this refresh:
│
├── exists locally? ─────────────────────────────────────────────────┐
│ │
│ YES: build updatedItem as { ...existingItem, ...freshFields } │
│ Fresh content fields (title, description, content, │
│ pubDate, coverImage, etc.) are taken from the remote. │
│ Local state fields (read, starred, saved, tags, │
│ savedFilePath) are preserved from existingItem. │
│ → goes into updatedItems[] │
│ │
│ NO: brand-new item — read:false, starred:false, tags:[] │
│ → goes into newItems[] ◄───────┘
│
└── (after iterating all fetched items)
For each existing local item NOT seen in this refresh:
│
├── older than autoDeleteCutoff AND unprotected? ──► skip (pre-filter)
│
└── otherwise ──► carry forward unchanged into carriedForward[]
Final merged list = [...carriedForward, ...updatedItems, ...newItems]
Key implication: refreshing a feed does not reset read state, tags, or starred status. Those fields are always taken from the locally-stored version when the item matches by guid.
Retention runs in two passes during parseFeed, both before the write to disk.
Pass 1 (inline, during carry-forward assembly): Unprotected carry-forward candidates older than autoDeleteCutoffMs are dropped before they enter the merged list at all. This prevents old items from reappearing as unread when a duration is re-enabled after being disabled.
Pass 2 (applyFeedLimits): Runs applyFeedRetentionLimits() on the fully merged list, applying both time-based and count-based rules.
The two passes are separate functions with independent logic. Pass 1 only applies the time-based cutoff to carry-forward items. Pass 2 applies both rules to all items.
Removes old, unprotected articles so the feed does not grow indefinitely.
- When it runs: every feed refresh; also the "Apply feed limits to all feeds" command.
- What it removes: articles older than the cutoff that do not match an enabled retention protection.
- Disabled duration: an
autoDeleteDurationof0disables the time-based pass. - Protections: starred and saved articles are protected by default; tagged and unread articles are protected only when their respective global setting is enabled.
For each article in merged list:
│
├── matches an enabled retention protection? ► KEEP
│
├── pubDate within cutoff window? ──────────► KEEP
│
└── older than cutoff + unprotected ─────────► REMOVE
Enforces a hard cap on the number of non-protected articles per feed. This is to maintain optimal plugin performance by automatically removing unwanted feeds.
- Keeps the newest N non-protected items (N =
maxItemsLimit). - Protected items are excluded from the count and never removed.
- Any article state can be protected when its corresponding global protection setting is enabled.
Non-protected items sorted newest → oldest:
[1] [2] [3] … [N] ← kept
[N+1] [N+2] … ← removed
Protected items: always appended, never counted
Protection is determined solely by isProtectedItem() and the current global
protection settings:
function isProtectedItem(
item: FeedItem,
protections: FeedRetentionProtections,
): boolean {
return (
(protections.protectStarred && item.starred) ||
(protections.protectSaved && item.saved) ||
(protections.protectTagged && item.tags?.length > 0) ||
(protections.protectUnread && !item.read)
);
}The default policy protects starred and saved articles. Tagged and unread articles are unprotected by default, but users can enable either protection in General Settings. Matching any enabled protection shields an article from both retention passes.
| Article state | Default protection | When its protection is enabled |
|---|---|---|
| Starred | 🟢 Protected | 🟢 Protected |
| Saved | 🟢 Protected | 🟢 Protected |
| Tagged | 🔴 Unprotected | 🟢 Protected |
| Unread | 🔴 Unprotected | 🟢 Protected |
| Other | 🔴 Unprotected | 🔴 Unprotected |
An unprotected article is removed when it is past the active age cutoff or
falls beyond the newest unprotected maxItemsLimit items. An article matching
any enabled protection is retained by both passes.
The dashboard reads articles from the in-memory settings object (which mirrors data.json). View filters — unread, read, starred, saved, folder, tags — are applied at render time to decide which articles to show.
View filters are not retention. They do not remove articles from storage. An article hidden by the unread filter is still in data.json; switching the filter back reveals it.
data.json (all stored articles)
│
▼
in-memory settings
│
├── view filter: unread ──► show only !item.read
├── view filter: starred ──► show only item.starred
├── folder scope ──► show only feeds in folder
└── tag filters ──► show only matching tags
Articles not matching the filter are hidden, not deleted.
The answer depends on where you change the setting.
When you change autoDeleteDuration in the Edit Feed modal and click Save, the plugin:
- Writes the new value to the in-memory feed object and saves to
data.json. - Immediately triggers a single-feed refresh (
refreshSelectedFeed) — the full fetch + merge + retention pipeline runs right away.
The new cutoff is enforced before you close the modal. You do not need to wait for the next scheduled refresh.
Changing defaultAutoDeleteDuration in the plugin settings tab saves the new value to data.json. A longer duration or disabled auto-deletion applies immediately and affects future refreshes. Shortening an active duration, or changing from disabled to an active duration, opens a confirmation dialog.
For a global setting, users can choose Apply now, Apply on next refresh, or Cancel. Apply now saves the setting and applies retention to all existing feeds, including refreshing the active dashboard view. Apply on next refresh saves the setting without pruning cached articles immediately. Cancel restores the prior control value and does not save.
The setting change alone does not restore any articles. Articles that were already pruned from data.json are gone. On the next refresh, the plugin will carry forward whatever is still in local storage and will no longer remove items that are now within the wider window — but nothing is un-deleted.
Turning an enabled retention protection off follows the same confirmation flow as shortening the duration. Turning a protection on saves immediately. The protection choices apply globally to starred, saved, tagged, and unread article states, and to both age- and count-based retention.
Old articles only reappear if:
- The remote feed still publishes them and returns them on the next fetch, or
- They were still present in
data.jsonwhen the setting was changed (i.e. they had not yet been pruned).
| Concern | Location |
|---|---|
| Merge (all three paths) | parseFeed() — src/services/feed-parser.ts |
| Carry-forward pre-filter (pass 1) | inline inside parseFeed() — src/services/feed-parser.ts |
| Final dedup + concat | mergeFeedHistoryItems() — src/services/feed-parser.ts |
| Retention pass 2 | applyFeedRetentionLimits() via applyFeedLimits() — src/services/feed-parser.ts |
| Refresh orchestration (single feed) | refreshSingleFeed() — main.ts |
| Refresh orchestration (batch) | refreshFeedBatch() — main.ts |
| Exclude from refresh | getRefreshableFeeds() — main.ts |
| Persist to disk | saveSettings() → this.saveData(this.settings) — main.ts |
| Metadata storage (vault location) | loadMetadata(), ensureMetadataFolderExists() — main.ts |
| Metadata migration | migrateMetadataToVaultLocation(), revertMetadataToPluginDefault() — main.ts |
| View filter application | getFilteredArticles() / getUnfilteredArticles() — src/views/dashboard-view.ts |
In addition to feed item storage, the plugin can now store metadata (settings/data.json) in two locations:
- Plugin Default:
<vault>/.obsidian/plugins/rss-dashboard/data.json(isolated, no sync) - Vault Location: User-defined path, e.g.
<vault>/.rss-dashboard-data/data.json(synced with vault)
This allows users to sync metadata across devices while keeping it separate from feed shards.
On plugin load:
- Check
settings.metadataStorageMode - If "vault-location": Try to load from vault location first
- Fall back to plugin-default if vault location unavailable
- Merge with default settings using
Object.assign()
On settings save:
- Determine save target based on current
metadataStorageMode - If "vault-location": - Call
ensureMetadataFolderExists()to create folder if needed (idempotent) - Write to vault location usingvault.adapter.write() - If "plugin-default": - Use
Plugin.saveData()(Obsidian's default)
Migrate to Vault Location:
- Ensure target folder exists (create if needed)
- Load current metadata from plugin-default
- Write to vault location
- Update
settings.metadataStorageModeto "vault-location" - Save settings (which now writes to vault location)
- On error: revert mode to "plugin-default" (no partial state)
Revert to Plugin Default:
- Load current metadata from vault location
- Update
settings.metadataStorageModeto "plugin-default" - Save settings using Plugin.saveData()
- Optionally clean up vault location file
- On error: keep metadata where it is (no partial state)
The BackupService follows metadata storage location:
- Async backups check
metadataStorageModeand back up from the active location - Sync backups (desktop) resolve the absolute path and back up accordingly
- Portable data bundles include
metadataStorageModeandmetadataStorageFolderfields
When importing a portable data bundle:
- Extract
metadataStorageModeandmetadataStorageFolderfrom bundle - Back up current metadata before import
- Restore metadata according to bundle's configuration
- On import failure: restore backup (no partial state)
- File I/O: Uses
vault.adapterfor vault-location (mobile-compatible),Plugin.saveData()for plugin-default - Error Handling: All state changes are transactional — revert on any error
- Idempotency: Folder creation handles race conditions and existing folders
- Type Safety: Uses
instanceof TFolderchecks (notisFolder()which doesn't exist)