Skip to content

Serve clean Markdown versions of every documentation page#446

Draft
Copilot wants to merge 3 commits intomainfrom
copilot/serve-markdown-docs
Draft

Serve clean Markdown versions of every documentation page#446
Copilot wants to merge 3 commits intomainfrom
copilot/serve-markdown-docs

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 29, 2026

Documentation pages are only served as HTML, which is hostile to AI ingestion. This PR exposes a clean .md version of every page, plus a manifest and a content-negotiation worker.

Markdown export plugin

  • _plugins/markdown_pages.rb writes a sibling .md file for every page during the Jekyll build (/foo/bar.html/foo/bar.md, //index.md).
  • Source is re-read from disk (Jekyll mutates page.content to HTML in-place by :site, :post_write), front matter stripped, then rendered through Liquid only — converters are skipped so output stays markdown.
  • Skips layout: redirect pages and any page with markdown_skip: true in front matter.
  • Emits /.well-known/markdown-index.txt listing every markdown URL for crawlers.

Clean output

  • _includes/code_tabs.html checks a markdown_format flag set by the plugin and emits plain fenced code blocks instead of the radio/label tab widget. HTML rendering is unchanged.
  • Layouts, nav and footer never appear because we render page.content directly, not through do_layout.
# GRAPH.QUERY

Executes the given query against a specified graph.

```python
graph.query("CREATE VECTOR INDEX FOR (p:Product) ON (p.description) ...")

### Build pipeline
- GitHub's built-in Pages builder runs Jekyll in safe mode and ignores `_plugins/`, so deployment moves to a GitHub Actions workflow (`.github/workflows/pages.yml`) using `actions/configure-pages` + `actions/deploy-pages`.
- Adds `Gemfile` (Jekyll 3.10, just-the-docs, kramdown-parser-gfm, the three existing plugins) and ignores `Gemfile.lock` / `vendor/` / `.jekyll-cache/`.

### `Accept: text/markdown` content negotiation
- Static Pages can't read request headers, so `markdown-content-negotiation.worker.js` is shipped as a reference Cloudflare Worker. It rewrites requests with `Accept: text/markdown` (or `text/x-markdown` / `text/plain`) to the corresponding `.md` URL, sets `Content-Type: text/markdown; charset=utf-8` and `Vary: Accept`, and falls back to the HTML response if the `.md` sibling 404s.

### Misc
- README documents the `.md` URL convention, the manifest path, and the worker.
- Adds `CDN`, `Cloudflare` to `.wordlist.txt` for spellcheck.

> [!WARNING]
>
> <details>
> <summary>Firewall rules blocked me from connecting to one or more addresses (expand for details)</summary>
>
> #### I tried to connect to the following addresses, but was blocked by firewall rules:
>
> - `docs.falkordb.com`
>   - Triggering command: `/usr/bin/curl curl -sI REDACTED` (dns block)
>   - Triggering command: `/usr/bin/curl curl -sI -L REDACTED` (dns block)
>   - Triggering command: `/usr/bin/curl curl -v REDACTED` (dns block)
>
> If you need me to access, download, or install something from one of these locations, you can either:
>
> - Configure [Actions setup steps](https://gh.io/copilot/actions-setup-steps) to set up my environment, which run before the firewall is enabled
> - Add the appropriate URLs or hosts to the custom allowlist in this repository's [Copilot coding agent settings](https://github.com/FalkorDB/docs/settings/copilot/coding_agent) (admins only)
>
> </details>

Copilot AI linked an issue Apr 29, 2026 that may be closed by this pull request
4 tasks
Copilot AI and others added 2 commits April 29, 2026 18:41
Agent-Logs-Url: https://github.com/FalkorDB/docs/sessions/a22cc9fa-6b51-4c1d-821b-103e4833d324

Co-authored-by: gkorland <753206+gkorland@users.noreply.github.com>
…edundant to_s

Agent-Logs-Url: https://github.com/FalkorDB/docs/sessions/a22cc9fa-6b51-4c1d-821b-103e4833d324

Co-authored-by: gkorland <753206+gkorland@users.noreply.github.com>
Copilot AI changed the title [WIP] Serve markdown versions of documentation pages Serve clean Markdown versions of every documentation page Apr 29, 2026
Copilot AI requested a review from gkorland April 29, 2026 18:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Documentation AI Readable - Serve .md files

2 participants