[Docs] Findablity & Accuracy: "Indexed Collection" and "Document Sources" pgs

[lisa-tarbo]

Summary: what and why

Findability: Split the overloaded "Indexed Collection" concepts page into three focused pages

Accuracy: The "Document Sources" content is out of date.

Context

The Collections / Indexed Collection page mixed three topics on one long page: the RAG overview, the local-vs-remote index comparison, and document-source syncing. The last two had no menu entry points, so they were hard to find.

The Document Sources documentation was missing information on managing sources: sync, the sync log, and snapshots.

Changes

Scope

Findability

• Split concepts/collections/indexed.md into three pages under concepts/collections/indexed_collections/:
  • index.md: overview and snapshots
  • local_and_remote_indexes.md: local vs remote index comparison
  • document_sources.md: what document sources are and how syncing works
• Moved tech-hub/local-index-optimization.md to tech-hub/collections/ and added tech-hub/collections/index.md.

Accuracy

• New tech-hub/collections/document_sources.md: configuration, authentication, sync behavior and limits, and troubleshooting for GitHub and Confluence. Config, authentication and troubleshooting were moved from the old how-to page. Sync behavior and limits are new.
• Rewrote and retitled how-to/document_sources.md as "Set Up and Manage Document Sources": adding a source, the management buttons, sync status, the sync log, and retrying failed files.
• Added a Snapshots section to indexed_collections/index.md. It describes the feature without steps, because snapshots apply to all indexed collections.
• Updated the concepts/index.md glossary entry and the authentication_providers.md page.
• Fixed supported file types list. And excluded CSV as file type supported

Menu structure

• Renamed media.md to media_collections.md for naming consistency.
• Grouped the indexed-collection pages in a collections/indexed_collections/ subfolder so the nav header is clickable, matching existing nav patterns.
• Added a Collections section to the Tech Hub nav, grouped the way Evaluations is.y.

Affected pages / sections

• Folder(s):
  • docs/tutorials/
  • docs/how-to/
  • docs/concepts/
  • docs/tech-hub/

Decisions and what was not addressed in this PR

• Existing content was moved and lightly edited for the split. New content covers sync management, snapshots, and the Document Sources reference.
• Document Sources accuracy: the weekly Auto Sync schedule was verified.
• docs/concepts/collections/media_collections.md has a title-case mismatch (title: Media collections vs # Media Collections). It predates this PR and was left as is.
• Accepted: docs/tech-hub/collections/document_sources.md is over the ~100-line target.

Validation

• This follows the relevant page-type contract.
• Examples and UI features/behavior were manually checked.
• Internal links and cross-references were manually reviewed for user value.
• Terminology matches current OCS naming and behavior.
• Validation commands run are listed below:
  • uv run zensical build --clean
  • uv run prek run markdownlint-cli2 --all-files
  • uv run prek run --all-files
  • uv run pytest scripts/tests — 1 pre-existing failure unrelated to this PR (test_update_confluence_release.py, missing RELEASE_URL env var; this branch touches no files under scripts/)

Risks / notes

• Renamed and moved files break external bookmarks and search-indexed links. No redirects are configured
• Changelog link retargeted: #indexing-options → #chunking-and-optimization. The entry is about PDF re-chunking, which the chunking section covers.

Co-Authored-By: Claude Sonnet 5 noreply@anthropic.com

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: e50192e8-2f6b-4407-8111-88f5d928cde0

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[claude]

Review: split looks faithful, two consistency items

The split itself is clean. I diffed the deleted concepts/collections/indexed.md against the three new pages — every paragraph, table row, admonition, and heading is accounted for, nothing dropped. Relative-path depth adjustments (../ to ../../) are correct throughout, anchors resolve on their targets, and a repo-wide search for collections/indexed.md and collections/media.md returns no matches, including reference-style definitions ([indexed-collections]: in concepts/experiment/index.md, [collections]: in how-to/assistants_migration.md). No mannered prose in the new text.

Two items left inline:

1. Nav label vs. landing-page title (mkdocs.yml:181) — an Indexed Collections section over an Indexed Collection (for RAG applications) page, where every sibling section matches its landing page's H1 exactly.
2. Changelog anchor retargeted (docs/changelog.md:150) — #indexing-options became #chunking-and-optimization, a semantic change the split didn't require. Just confirm it's deliberate.

One question

concepts/collections/indexed_collections/document_sources.md is 15 lines, and most of it points at how-to/document_sources.md. The one genuinely conceptual fact — that document-source syncs reach a published chatbot without a republish — justifies the page existing, but as its own sidebar entry it is thin, and the opening sentence restates the how-to page's own intro. Given the goal was to give this topic a menu entry point, is a one-paragraph page the entry point you wanted, or would it serve readers better as a section on the Indexed Collections landing page?

Validation not run

I could not run uv run zensical build --clean or markdownlint-cli2 in this environment (dependency install was unavailable). The link and anchor checks above were manual, so please confirm CI's strict build is green before merging — that is the authoritative check for broken refs.

🤖 Generated with Claude Code

[claude]

Review: structure is sound, a few things to fix

The three-way split holds up. I checked every link touched by the move and found no broken cross-references or dead anchors — #remote-index, #which-should-i-use, #chunking-and-optimization, #choosing-an-embedding-model, #read-the-sync-log, #retry-failed-files, #bearer-token, #basic-auth all resolve, and nothing in docs/ still points at collections/indexed.md, collections/media.md, or tech-hub/local-index-optimization.md. Every new file is in nav, nothing orphaned. The page-type split is right: concept material in concepts/, steps in how-to/, field tables and limits in tech-hub/. No mannered prose.

Four inline comments, in rough priority order:

1. tech-hub/collections/document_sources.md:25 — the chunking bullet contradicts local-index-optimization.md for remote indexes. The only substantive correctness issue here.
2. mkdocs.yml:75 — missing redirects for the other two renames.
3. docs/concepts/index.md:33 — glossary entry out of alphabetical order.
4. docs/changelog.md:155 — link lands on a stub section.

Build and lint not verified

uv isn't available in my environment, so I could not run uv run zensical build --clean or markdownlint-cli2. Taking the PR description's claim that they pass; worth a second pair of eyes given strict: true and the number of moved files.

Numbers that need a product-side check

These are new, specific, and not derivable from anything in this repo. None of them existed in the docs before, so if one is wrong it ships as fact and misleads exactly the person debugging a stuck sync:

• Stall threshold of two hours; 1000-file collection limit; 50 failed files per sync log (tech-hub/collections/document_sources.md, Shared behavior)
• GitHub: 50 MB skip threshold, default branch main, default pattern *.md, re-sync keyed on commit sha
• Confluence: Max Pages 1–10000 default 1000, re-sync keyed on last-modified time, and the claim that exceeding Max Pages produces no warning
• Chunk size 800 / overlap 400 for synced files (local-index-optimization.md:36)
• Auto Sync "once a week" — note this supersedes the old docs, which said "nightly" in concepts/collections/index.md and "on a schedule" in indexed.md. The PR says it was verified; flagging because it's a user-visible change of a previously documented number.

Smaller things, take or leave

• The weekly cadence is now stated in three files (concepts/.../document_sources.md:46, how-to/document_sources.md:24 and :32). If the schedule ever changes, that's three places to find. Consider stating it once and linking.
• local_and_remote_indexes.md has no ## See also, unlike both of its siblings in the same new folder.
• concepts/collections/index.md:17 links "document-source" to the how-to page, while the bullet two lines below links the same term to the concepts page.
• The PR description still has an editing placeholder: <or: "removed to match the no-redirects decision">. Worth clearing before merge, since the description becomes the squash commit body.
• Three files named document_sources.md across three folders is new for this repo, and every cross-reference between them uses a different ../ depth. Not wrong, and each title disambiguates, but it's a link-rot risk for future editors. document_sources_reference.md for the tech-hub page would remove it.

[lisa-tarbo]

@barry47products Noted:

One thing for #868: local_and_remote_indexes.md lists "pdf, txt, csv, docx" for local indexes, but regular upload to an indexed collection doesn't accept csv. Both index types use the same file_search list in settings.

[barry47products]

@lisa-tarbo I checked this against the code and pushed 3b3945d with the fixes. The numbers the bot asked about all match the code. The changes:

• Remote indexes use the chunk size and overlap set in OCS. The provider does the splitting, but with OCS's numbers. They also accept the same file types as local indexes.
• The GitHub sha is the file's content hash, not a commit hash.
• "A sync does not stop at this limit" now names the 1000-file limit.
• "Not yet synced" also covers a source whose first sync failed.

Once this merges I'll move the CSV section from #877 into your new layout.

[lisa-tarbo]

@snopoke Barry reviewed and made some corrections so should be quicker for you to review