diff --git a/.github/workflows/gh-pages.yml b/.github/workflows/gh-pages.yml index 8d8249e..7a8fbc5 100644 --- a/.github/workflows/gh-pages.yml +++ b/.github/workflows/gh-pages.yml @@ -1,10 +1,14 @@ -name: Deploy docs +name: Redirect the old docs address + +# The docs moved to ducklocal.app/docs/ (source: JetSquirrel/ducklocal-site). +# This repository's GitHub Pages site, jetsquirrel.github.io/DuckLocal/, now +# only redirects each old page to its new address. on: push: branches: [main] paths: - - 'docs/**' + - 'scripts/pages-redirects.sh' - '.github/workflows/gh-pages.yml' workflow_dispatch: @@ -28,18 +32,8 @@ jobs: steps: - uses: actions/checkout@v4 - - name: Setup Node - uses: actions/setup-node@v4 - with: - node-version: 24 - cache: npm - cache-dependency-path: docs/package-lock.json - - - name: Install dependencies - run: npm --prefix docs ci - - - name: Build the docs site - run: npm --prefix docs run build + - name: Write the redirect pages + run: scripts/pages-redirects.sh target/pages-redirects - name: Setup Pages uses: actions/configure-pages@v4 @@ -47,7 +41,7 @@ jobs: - name: Upload artifact uses: actions/upload-pages-artifact@v3 with: - path: 'target/docs-site' + path: 'target/pages-redirects' - name: Deploy to GitHub Pages id: deployment diff --git a/.gitignore b/.gitignore index fdf8136..caf4366 100644 --- a/.gitignore +++ b/.gitignore @@ -1,10 +1,6 @@ # Rust build artifacts /target -# Docs site dependencies and VitePress build cache -docs/node_modules/ -docs/.vitepress/cache/ - # macOS packaging output (scripts/package-macos.sh writes these into the # repository root; CI uploads them as release assets) /DuckLocal.app/ diff --git a/README.md b/README.md index b257255..42eca69 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ DuckLocal logo -**[Docs](docs/index.md)** · **[中文文档](README.zh-CN.md)** +**[ducklocal.app](https://ducklocal.app/)** · **[Docs](https://ducklocal.app/docs/)** · **[中文文档](README.zh-CN.md)** A local-first workspace for querying and exploring your data, built natively on DuckDB. Point it at your files — there is no connection to configure, no schema to create, and nothing is uploaded anywhere. @@ -59,7 +59,7 @@ ducklocal check dashboard.dash # validate; JSON diagnostics, exit 2 o ducklocal lsp # language server over stdio, for editors ``` -See the [CLI guide](docs/cli.md) for conversion, stdin, output, app export, and safety details. +See the [CLI guide](https://ducklocal.app/docs/cli) for conversion, stdin, output, app export, and safety details. The [official agent skill](skills/ducklocal/SKILL.md) teaches schema-first exploration, SQL analysis, and verified file conversion. Copy it into your target project's supported skills directory; for example, from this checkout: @@ -72,9 +72,9 @@ Inspect existing destinations before overwriting. No global configuration is cha ## Documentation -Guides live in [`docs/`](docs/index.md): [getting started](docs/getting-started.md), a [10-minute tutorial](docs/tutorial.md) with sample data, [troubleshooting](docs/troubleshooting.md), [data sources](docs/data-sources.md), [S3 and httpfs](docs/s3.md), [SQL editor](docs/sql-editor.md), [schema browser and history](docs/schema-and-history.md), [results and charts](docs/results-and-charts.md), [settings and app data](docs/settings-and-data.md), and [development](docs/development.md). +The guides are at **[ducklocal.app/docs](https://ducklocal.app/docs/)**: [getting started](https://ducklocal.app/docs/getting-started), a [10-minute tutorial](https://ducklocal.app/docs/tutorial) with sample data, [troubleshooting](https://ducklocal.app/docs/troubleshooting), [data sources](https://ducklocal.app/docs/data-sources), [S3 and httpfs](https://ducklocal.app/docs/s3), [SQL editor](https://ducklocal.app/docs/sql-editor), [schema browser and history](https://ducklocal.app/docs/schema-and-history), [results and charts](https://ducklocal.app/docs/results-and-charts), [settings and app data](https://ducklocal.app/docs/settings-and-data), [dashboards](https://ducklocal.app/docs/dashboards), and [development](https://ducklocal.app/docs/development). -The same pages are published with VitePress. With Node.js 22 or later (24 LTS recommended), run `npm --prefix docs ci`, then `npm --prefix docs run build` to generate `target/docs-site`, or `npm --prefix docs run dev` for local development. CI deploys the site to GitHub Pages whenever `docs/` changes on `main`. See [development](docs/development.md#the-documentation-site) for preview commands and configuration. +The website — the product page at [ducklocal.app](https://ducklocal.app/) and the docs — lives in its own repository, [JetSquirrel/ducklocal-site](https://github.com/JetSquirrel/ducklocal-site). A change here that changes what the docs say needs a pull request there too. ## Run @@ -90,7 +90,7 @@ cargo run -- ./data/logs/ # or open something straight away # Produces target/release/DuckLocal.app ``` -The released disk image runs on macOS 12 or later, Apple silicon only. `bundle.sh` does not sign what it builds; `scripts/package-macos.sh` is the shipping path and signs and notarizes the dmg. See [development](docs/development.md). +The released disk image runs on macOS 12 or later, Apple silicon only. `bundle.sh` does not sign what it builds; `scripts/package-macos.sh` is the shipping path and signs and notarizes the dmg. See [development](https://ducklocal.app/docs/development). ## License diff --git a/README.zh-CN.md b/README.zh-CN.md index 1496497..b707360 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -2,7 +2,7 @@ DuckLocal logo -**[文档](docs/zh/index.md)** · **[English](README.md)** +**[ducklocal.app](https://ducklocal.app/zh/)** · **[文档](https://ducklocal.app/docs/zh/)** · **[English](README.md)** 本地优先的数据查询与分析工作台，原生基于 DuckDB。直接指向你的文件即可——不用配置连接、不用建 schema，也不会上传任何数据。 @@ -59,7 +59,7 @@ ducklocal check dashboard.dash # 校验；JSON 诊断，有错误时 ducklocal lsp # 面向编辑器的语言服务器（stdio） ``` -转换、stdin、输出编码、应用导出和安全说明见 [CLI 指南](docs/zh/cli.md)。 +转换、stdin、输出编码、应用导出和安全说明见 [CLI 指南](https://ducklocal.app/docs/zh/cli)。 [官方 agent skill](skills/ducklocal/SKILL.md) 教 AI 先查 schema，再进行 SQL 分析及验证格式转换。复制到目标项目支持的 skill 目录即可，例如在本仓库执行： @@ -72,9 +72,9 @@ cp -R skills/ducklocal /path/to/your-project/.claude/skills/ ## 文档 -完整指南在 [`docs/`](docs/zh/index.md)：[快速上手](docs/zh/getting-started.md)、带样例数据的 [10 分钟上手教程](docs/zh/tutorial.md)、[常见问题排查](docs/zh/troubleshooting.md)、[数据源](docs/zh/data-sources.md)、[S3 与 httpfs](docs/zh/s3.md)、[SQL 编辑器](docs/zh/sql-editor.md)、[Schema 浏览与历史](docs/zh/schema-and-history.md)、[结果与图表](docs/zh/results-and-charts.md)、[设置与应用数据](docs/zh/settings-and-data.md)、[开发](docs/zh/development.md)。 +完整指南在 **[ducklocal.app/docs](https://ducklocal.app/docs/zh/)**：[快速上手](https://ducklocal.app/docs/zh/getting-started)、带样例数据的 [10 分钟上手教程](https://ducklocal.app/docs/zh/tutorial)、[常见问题排查](https://ducklocal.app/docs/zh/troubleshooting)、[数据源](https://ducklocal.app/docs/zh/data-sources)、[S3 与 httpfs](https://ducklocal.app/docs/zh/s3)、[SQL 编辑器](https://ducklocal.app/docs/zh/sql-editor)、[Schema 浏览与历史](https://ducklocal.app/docs/zh/schema-and-history)、[结果与图表](https://ducklocal.app/docs/zh/results-and-charts)、[设置与应用数据](https://ducklocal.app/docs/zh/settings-and-data)、[Dashboard](https://ducklocal.app/docs/zh/dashboards)、[开发](https://ducklocal.app/docs/zh/development)。 -这些页面同时会以 VitePress 站点形式发布。需要 Node.js 22 及以上（推荐 24 LTS）：`npm --prefix docs ci` 安装依赖，`npm --prefix docs run build` 生成到 `target/docs-site`，`npm --prefix docs run dev` 本地开发。CI 在 `main` 分支的 `docs/` 有改动时自动部署到 GitHub Pages。详见[开发](docs/zh/development.md#文档站)。 +网站——[ducklocal.app](https://ducklocal.app/zh/) 产品首页与文档——放在独立的仓库 [JetSquirrel/ducklocal-site](https://github.com/JetSquirrel/ducklocal-site)。这里的改动如果改变了文档里的说法，也需要向那边提交 pull request。 ## 运行 @@ -90,7 +90,7 @@ cargo run -- ./data/logs/ # 或直接打开数据 # 生成 target/release/DuckLocal.app ``` -发布的安装包要求 macOS 12 及以上、Apple 芯片。`bundle.sh` 不做签名；正式发布走 `scripts/package-macos.sh`，它会签名并对 dmg 做公证。详见[开发](docs/zh/development.md)。 +发布的安装包要求 macOS 12 及以上、Apple 芯片。`bundle.sh` 不做签名；正式发布走 `scripts/package-macos.sh`，它会签名并对 dmg 做公证。详见[开发](https://ducklocal.app/docs/zh/development)。 ## 开源协议 diff --git a/docs/.vitepress/config.mts b/docs/.vitepress/config.mts deleted file mode 100644 index dc5e0a0..0000000 --- a/docs/.vitepress/config.mts +++ /dev/null @@ -1,166 +0,0 @@ -import { copyFile, writeFile } from 'node:fs/promises' -import { resolve } from 'node:path' -import { defineConfig, type DefaultTheme } from 'vitepress' - -const repository = 'https://github.com/JetSquirrel/DuckLocal' -const pages = [ - 'index', - 'getting-started', - 'data-sources', - 's3', - 'sql-editor', - 'schema-and-history', - 'results-and-charts', - 'settings-and-data', - 'analysis-app', - 'development', -] - -function sidebar(zh = false): DefaultTheme.SidebarItem[] { - const prefix = zh ? '/zh/' : '/' - const item = (page: string, english: string, chinese: string) => ({ - text: zh ? chinese : english, - link: `${prefix}${page}`, - }) - return [ - { - text: zh ? '入门' : 'Getting started', - items: [ - item('getting-started', 'Install and first query', '安装与第一条查询'), - item('tutorial', 'Your first 10 minutes', '10 分钟上手教程'), - item('troubleshooting', 'Troubleshooting', '常见问题排查'), - ], - }, - { - text: zh ? '使用指南' : 'Guides', - items: [ - item('data-sources', 'Data sources', '数据源'), - item('s3', 'S3 and httpfs', 'S3 与 httpfs'), - item('sql-editor', 'SQL editor', 'SQL 编辑器'), - item('schema-and-history', 'Schema and history', 'Schema 浏览与历史'), - item('results-and-charts', 'Results and charts', '结果与图表'), - item('settings-and-data', 'Settings and app data', '设置与应用数据'), - ], - }, - { - text: zh ? '自动化与扩展' : 'Automate and extend', - items: [ - item('cli', 'CLI and agent skill', 'CLI 与 agent skill'), - item('analysis-app', 'Apps and dashboards', '分析应用与 Dashboard'), - ], - }, - { - text: zh ? '参与开发' : 'Contributing', - items: [item('development', 'Development guide', '开发指南')], - }, - ] -} - -export default defineConfig({ - title: 'DuckLocal', - description: 'A local-first workspace for querying and exploring your data, built natively on DuckDB.', - base: '/DuckLocal/', - outDir: '../target/docs-site', - cleanUrls: false, - ignoreDeadLinks: false, - head: [['link', { rel: 'icon', type: 'image/png', href: '/DuckLocal/favicon.png' }]], - locales: { - root: { - label: 'English', - lang: 'en', - themeConfig: { - nav: [ - { text: 'Get started', link: '/getting-started' }, - { text: 'Tutorial', link: '/tutorial' }, - { text: 'CLI', link: '/cli' }, - { text: 'Download', link: `${repository}/releases/latest` }, - ], - sidebar: sidebar(), - }, - }, - zh: { - label: '简体中文', - lang: 'zh-CN', - description: '本地优先的数据查询与分析工作台，原生基于 DuckDB。', - themeConfig: { - nav: [ - { text: '快速上手', link: '/zh/getting-started' }, - { text: '教程', link: '/zh/tutorial' }, - { text: 'CLI', link: '/zh/cli' }, - { text: '下载', link: `${repository}/releases/latest` }, - ], - sidebar: sidebar(true), - outline: { label: '本页目录', level: [2, 3] }, - editLink: { pattern: `${repository}/edit/main/docs/:path`, text: '在 GitHub 上编辑此页' }, - docFooter: { prev: '上一页', next: '下一页' }, - langMenuLabel: '切换语言', - sidebarMenuLabel: '目录', - returnToTopLabel: '返回顶部', - skipToContentLabel: '跳转到内容', - darkModeSwitchLabel: '外观', - lightModeSwitchTitle: '切换到浅色主题', - darkModeSwitchTitle: '切换到深色主题', - notFound: { - title: '页面未找到', - quote: '这个页面不存在，请返回首页或使用搜索查找指南。', - linkLabel: '返回首页', - linkText: '返回首页', - }, - footer: { message: '基于 Apache-2.0 许可开源发布。' }, - }, - }, - }, - themeConfig: { - logo: { src: '/assets/logo.png', alt: 'DuckLocal' }, - outline: { level: [2, 3] }, - editLink: { pattern: `${repository}/edit/main/docs/:path` }, - socialLinks: [{ icon: 'github', link: repository }], - footer: { message: 'Released under the Apache-2.0 License.' }, - search: { - provider: 'local', - options: { - locales: { - zh: { - translations: { - button: { buttonText: '搜索', buttonAriaLabel: '搜索文档' }, - modal: { - displayDetails: '显示详细列表', - resetButtonTitle: '清除搜索', - backButtonTitle: '关闭搜索', - noResultsText: '没有找到相关结果', - footer: { - selectText: '选择', - selectKeyAriaLabel: '回车键', - navigateText: '切换', - navigateUpKeyAriaLabel: '向上箭头', - navigateDownKeyAriaLabel: '向下箭头', - closeText: '关闭', - closeKeyAriaLabel: 'Esc 键', - }, - }, - }, - }, - }, - }, - }, - }, - async buildEnd(config) { - await copyFile(resolve(config.srcDir, 'assets/logo.png'), resolve(config.outDir, 'assets/logo.png')) - await Promise.all(pages.map(async (page) => { - const target = `${config.site.base}zh/${page === 'index' ? '' : `${page}.html`}` - await writeFile(resolve(config.outDir, `${page}.zh-CN.html`), ` - - - - - -页面已迁移 · DuckLocal - - - -

文档已迁移。前往新页面

- -`) - })) - }, -}) diff --git a/docs/.vitepress/theme/custom.css b/docs/.vitepress/theme/custom.css deleted file mode 100644 index df0ee30..0000000 --- a/docs/.vitepress/theme/custom.css +++ /dev/null @@ -1,35 +0,0 @@ -:root { - --vp-c-brand-1: #806000; - --vp-c-brand-2: #967000; - --vp-c-brand-3: #a47b00; - --vp-c-brand-soft: rgba(242, 183, 5, 0.14); - --vp-button-brand-text: #242117; - --vp-button-brand-bg: #f2c438; - --vp-button-brand-hover-text: #242117; - --vp-button-brand-hover-bg: #f7d361; - --vp-button-brand-active-text: #242117; - --vp-button-brand-active-bg: #e6b824; -} - -.dark { - --vp-c-brand-1: #f2cb57; - --vp-c-brand-2: #e7bd3b; - --vp-c-brand-3: #d5a926; - --vp-c-brand-soft: rgba(242, 183, 5, 0.16); -} - -.VPHero .image-src { - max-width: 240px; - max-height: 240px; - border-radius: 32px; -} - -.VPNavBarTitle .logo { - border-radius: 6px; -} - -.VPHome .vp-doc img { - border: 1px solid var(--vp-c-divider); - border-radius: 12px; - margin: 24px auto; -} diff --git a/docs/.vitepress/theme/index.ts b/docs/.vitepress/theme/index.ts deleted file mode 100644 index 42fe9a9..0000000 --- a/docs/.vitepress/theme/index.ts +++ /dev/null @@ -1,4 +0,0 @@ -import DefaultTheme from 'vitepress/theme' -import './custom.css' - -export default DefaultTheme diff --git a/docs/analysis-app.md b/docs/analysis-app.md deleted file mode 100644 index b23bfdf..0000000 --- a/docs/analysis-app.md +++ /dev/null @@ -1,288 +0,0 @@ -# Analysis apps (JavaScript) - -**[中文](zh/analysis-app.md)** · [Docs](index.md) - -An analysis app is a workspace tab whose contents are a JavaScript application you -wrote. JavaScript is the declaration: the app's `render` function is the description of -the interface, and the range controls, chips, KPI cards, charts and tables are all yours -to compose. An analysis app reads the database the window already has open — the same -tables, views and attached files — through the host functions below. - -This is for building a view over your data: a dashboard, a purpose-built browser for one -table, a report you run every morning. An analysis app opens and closes like a query, and -sits beside your SQL tabs. - -## Open an app - -Any of these opens one, and each loads the folder immediately: - -- the **+** at the end of the tab strip, then **Open app…**; -- drag an app folder onto the window; -- name it on the command line: `ducklocal examples/analysis_app`. - -A folder that cannot be an app — a file, a path that is gone, or a folder with no -`main.js` — is refused **inside the tab**, naming the folder and what is missing. An app -that is already working is never thrown away by a folder you cancelled out of or named by -mistake. - -Apps close with the tab's close button and rename like a query tab. The app folders you -have open are remembered: they reopen at the next launch, and one that has been moved or -deleted is reported by name rather than silently forgotten. - -Apps run on their own DuckDB connection to the database the main window has open, so an -app and the SQL editor see the same tables, views and attached files without waiting on -each other: a dashboard refreshing half a dozen statements no longer freezes the editor. -What a second connection does not carry is connection-local state — a `TEMP` table or a -`SET` made in the SQL editor is not there for an app, and the other way round. All apps -share that one connection, so they still queue behind each other. - -## View definition - -**View definition** in the app's header shows the folder and the entry file's source, -read-only. An app is an ordinary directory of JavaScript files, so that is the whole -definition — there is no second, hidden form of the app to inspect. - -## Reload - -Save a `.js` or `.mjs` file in the app's folder and the app reloads. The host polls the -folder every 250 ms and waits 200 ms for writes to settle, so saving six files at once is -one reload. Dotfiles, `node_modules/` and `target/` are ignored. Only the folder currently -shown is watched. - -**Reload** in the app's header does the same thing on demand. - -A reload that fails **keeps the last app that loaded** and reports the error above it, -inside the tab. A load that fails on an app that has never loaded shows the error as the -tab's whole body instead. - -A reload restarts the app's JavaScript, not its connection. The app connection is one -long-lived connection shared by every app, so state an app made on it — an `ATTACH`, a -`SET`, a `TEMP` table — is still there after the reload. Write an app's setup so that -running it twice is safe (`ATTACH IF NOT EXISTS …`, `CREATE OR REPLACE TEMP TABLE …`) -rather than assuming a clean connection on every load. - -## Host functions - -Import them from the `ducklocal` module: - -```js -import { catalog, query, appDir, sqlLiteral, sqlIdentifier } from "ducklocal"; -``` - -They all run against the app connection — the same database the main window is on — and -they throw JavaScript `Error`s on failure. Everything but `appDir` may be called at any -time; `appDir` answers during the app's load. - -### `catalog()` - -```ts -catalog(): Promise - -interface CatalogEntry { - database: string; // e.g. "memory" - schema: string; // e.g. "main" - name: string; // e.g. "orders" - kind: "table" | "view"; - estimated_rows: number | null; - comment: string | null; - columns: { name: string; type: string }[]; // type is DuckDB's, e.g. "DECIMAL(38,10)" -} -``` - -Every table and view of every database on the connection — the one the window opened and -anything `ATTACH`ed, by the window or by the app itself — except DuckDB's own catalogs -(`information_schema`, `pg_catalog`, `system`). `entry.database` says which database an -entry belongs to; use it when you qualify a name. Tables come before views. DuckDB resolves -identifiers case-insensitively and keeps tables and views in one namespace, so qualify a -name when you build SQL from it, and quote it with `sqlIdentifier`. - -### `query(sql, limit?)` - -```ts -query(sql: string, limit?: number): Promise - -interface QueryResult { - columns: { name: string; type: string }[]; - rows: any[][]; // one array per row, one entry per column - row_count: number; // rows actually returned - truncated: boolean; // true when the row limit stopped the result early - elapsed_ms: number; -} -``` - -- `limit` defaults to **1000**; a value below 1 is refused and anything above 100000 is - clamped. A 2,000,000-cell budget applies - as well, so a very wide result returns fewer rows than `limit`. **Read `truncated`.** -- `type` is the Arrow type as rendered by DuckLocal, e.g. `Int64`, `Utf8`, - `Decimal(38, 10)`, `Timestamp(Microsecond, None)`. -- The statement is prepared and executed as given; `query()` does not split or validate - multi-statement SQL. - -### `appDir()` - -```ts -appDir(): string // the app's own folder, absolute -``` - -An app cannot read the filesystem, so this is how it addresses data that ships beside it — -`query("SELECT * FROM " + sqlLiteral(appDir() + "/orders.csv"))` works with nothing -attached and nothing open in the window. It answers while the app loads: call it in `init()` -and keep the result. - -`panelDir()` is the deprecated alias from before apps were called apps; it keeps working, -but new code should use `appDir()`. - -### `sqlLiteral(value)` - -```ts -sqlLiteral(value: string): string // e.g. "O'Brien" -> 'O''Brien' -``` - -Quotes and escapes a string as a SQL string literal. Use it for any value that comes from -your app's own state — a filter, a channel id, a date — instead of templating it into SQL -by hand. A `NUL` byte is refused, because a SQL literal cannot carry one. - -### `sqlIdentifier(name)` - -```ts -sqlIdentifier(name: string): string // e.g. "my table" -> "my table" -``` - -The same job for the other half of a statement: a table, view or column **name**, quoted so -it is read as one name and matched exactly as written. A name is not a string literal, and -the names an app builds SQL from are rarely its own — `catalog()` answers with whatever the -database holds, so `my table`, `Order`, and a name holding a `"` all turn up. An empty name -and a `NUL` byte are refused; neither names anything, and quoting them would only hide that. - -```js -const entry = (await catalog()).find((table) => table.name === "orders"); -const from = `${sqlIdentifier(entry.schema)}.${sqlIdentifier(entry.name)}`; -const rows = await query(`SELECT count(*) FROM ${from}`); -``` - -### Cell encodings - -A cell is plain JSON **when that loses nothing** — `null`, booleans, strings, and numbers -that fit an IEEE double exactly. Anything else arrives as an object with an `encoding` -field, the same encoding the `ducklocal query` CLI prints, so no digit is silently dropped: - -| Value | What arrives | -| --- | --- | -| `NULL` | `null` | -| `BOOLEAN` | `true` / `false` | -| integers within ±2^53−1 | number | -| larger integers (`HUGEINT`, `UBIGINT`, …) | `{ "encoding": "integer", "value": "170141183460469231731687303715884105727" }` | -| `DECIMAL` | `{ "encoding": "decimal", "value": "12345678901234567890.1234567890" }` | -| non-finite `FLOAT`/`DOUBLE` | `{ "encoding": "float", "value": "inf" }` | -| `DATE` | `{ "encoding": "date", "unit": "Day", "value": "19783" }` | -| `TIMESTAMP` | `{ "encoding": "timestamp", "unit": "Microsecond", "value": "1709296496789000" }` | -| `TIME` | `{ "encoding": "time", "unit": "Microsecond", "value": "45296789000" }` | -| `INTERVAL` | `{ "encoding": "interval", "months": 0, "days": 1, "nanos": "0" }` | -| `BLOB`, `GEOMETRY` | `{ "encoding": "hex", "value": "00ff" }` | -| `LIST` / `ARRAY` | array | -| `STRUCT` | `{ "encoding": "struct", "fields": [["name", ], …] }` | -| `MAP` | `{ "encoding": "map", "entries": [[, ], …] }` | -| `UNION` | `{ "encoding": "union-value", "value": }` | - -`DATE` value `19783` is days since 1970-01-01; `TIMESTAMP` and `TIME` values are counts of -`unit` since the epoch or midnight. These are the storage values, not formatted text: -format them yourself, or `CAST` the column to `VARCHAR` in SQL if you want the string form. - -## Components - -An app draws with the same component catalog the shell ships, imported from -`gpui-component`: `GroupBox` (a titled card), `Progress` (a bar), `Toggle`, `Badge`, `Tag`, -`Alert`, `Empty`, `Collapsible`, `DescriptionList`, `DataTable` with `DataTableState`, the -`Table` family, `Sidebar`, `Resizable`, `Scroll`, `DescriptionList`, and the charts — -`BarChart`, `LineChart`, `AreaChart`, `PieChart`, `RadarChart`. Layout primitives -(`div`, `h_flex`, `v_flex`, `Button`) come from `gpui-kit` and `gpui-base`. - -There is no `ToggleGroup`: compose a segmented control or a row of chips from several -`Toggle`s. The example app at -[`examples/analysis_app`](https://github.com/JetSquirrel/DuckLocal/tree/main/examples/analysis_app) -is a full dashboard built from exactly these, and it is the best thing to copy from. - -Every app folder ships a `jsconfig.json` and a generated `gpui-kit.d.ts`, so check the -app against the API the runtime will actually give it before you save: - -```sh -npx --yes -p typescript tsc -p /jsconfig.json --noImplicitAny false -``` - -That reports a property that does not exist on a catalog component — the kind of mistake -the runtime otherwise refuses only at render, inside the app's own error surface, which -is not written to the app log. Its blind spot is the `gpui-base` primitives (`div`, -`h_flex`, `v_flex`, `Button.new`): they are loosely typed, so a wrong method on one of -those is not caught here and still only shows up at render. - -## SQL is not sandboxed - -`query()` runs on DuckLocal's own database, **with DuckLocal's privileges and the user's**. -An app can read and write anything the SQL editor can, including `COPY` to files, `ATTACH` -of other databases, `INSTALL`/`LOAD` of extensions, and queries against S3 views whose -credentials are already configured. Treat an app's JavaScript as code you are choosing to -run, exactly as you would a shell script. - -So the choice is asked for. The first time a folder opens as an app — from the command line, -a drop, the picker, or a tab restored at launch — its tab says what the app's SQL can do and -waits: **View source** shows the entry file without running it, **Trust and run** runs it. -The answer is remembered per folder, so later launches and every reload after a save run -without asking again. `ducklocal export --html` runs the app you name on the command line -and does not ask. - -What an app does **not** get is anything else the process could do: - -- **No filesystem, network, process, or environment module.** `fs`, `net`, `process` and - friends are not available to a script unless the host grants them; DuckLocal grants none. -- **No S3 credentials.** The S3 browser's keys live in DuckLocal's own Rust state and are - used by DuckLocal's signing client; they are never written into the DuckDB connection, and - the host module cannot reach them. An app can still run SQL that uses httpfs if the user's - own connection is configured for it. -- **The host module is the whole surface.** Only the functions above cross into Rust. - -## Export an app as HTML - -`ducklocal export --html ` runs an app once and writes what it asked the database into one self-contained HTML file, for sending an app's numbers to someone who does not have DuckLocal. The command, its options and its exit codes are in [the CLI guide](cli.md#export-an-app-as-a-static-html-file). The capture ends when the app has gone quiet, or at `--timeout` seconds (default 15), whichever comes first; in the deadline case the JSON's `stop_reason` is `deadline` and the report carries a visible warning that it may be incomplete. - -The report is the app's data, not its interface. Only what `query()` returned is exported, in call order, one section per distinct statement — an identical statement run again collapses into its section, marked as run that many times — and every call is captured, including statements that are not `SELECT`, such as `ATTACH`. A `catalog()` call becomes an appendix of tables and columns rather than a numbered statement. A leading `-- title: …` comment on the statement's first line names its section ("Statement N — title"); the SQL itself shows verbatim. Each result shows as a table, with an inline bar chart above it when the result is a label and a number per row: exactly two columns, at most 25 rows, and a non-negative numeric second column (the first column labels the bars). Anything else is only a table. - -What the report does **not** hold is anything the app drew: KPI cards, charts, and tables built from JavaScript constants never reach the file, because the export records queries, not the render — an app draws native components and their contents are decided while they are laid out, so nothing describes a chart's bars well enough to reproduce them. Constants a report must show can be sent through SQL instead: `SELECT * FROM (VALUES ('Q1', 120), ('Q2', 95)) AS t(quarter, total)`. - -Because it is the loading state, a statement an app only runs on a click is not in the report, and neither is anything after a reload. The app's JavaScript runs with the same privileges as always: exporting is running the app, exactly as opening it in a tab is. - -## Dashboard specs (.dash) - -A `.dash` file is a dashboard declared as data — query and plot blocks, no JavaScript — for the common case of standard plots over saved queries. The file format and `ducklocal check` validation are in [the CLI guide](cli.md#check-a-dashboard-spec). - -Open one like an app: choose **Open dashboard…** from the tab strip's `+` menu, name it on the command line (`ducklocal dashboard.dash`), or drag the file onto the window, and it opens as a dashboard tab beside your queries and apps. The tab runs the spec's queries on the window's own connection — so a dashboard sees connection-local state such as `TEMP` tables, and queues with the editor's queries — and renders each plot as one panel of a vertical stack whose dividers drag to resize. Only read-only statements run: a query that could write (DDL, DML, `COPY`, `ATTACH`, …) is refused before it reaches the connection, so opening a `.dash` file someone sent you cannot change your data. A plot whose query fails shows the reason in its own panel; the rest of the dashboard still draws. The toolbar's reload re-reads the file and re-runs everything, and a reload that fails validation never replaces a working dashboard — the reason appears above it instead. Open dashboards are remembered between launches, exactly like apps. - -The tab is also an editor: its source view edits the file with syntax highlighting — the SQL in heredocs coloured as it is in the SQL editor — completion and diagnostics, and writes back with the save button or ⌘S — a save whose spec no longer validates keeps the last working dashboard up and says why. Outside the GUI, `ducklocal lsp` serves the same completion, diagnostics, hover and go-to-definition to any LSP-capable editor (see [the CLI guide](cli.md#edit-a-dashboard-spec-with-lsp)). - -## Known limits - -- **An app in a tab has no window-level overlay.** The shell's dialogs, sheets, toasts and - tooltip layer are found only when the shell's own root view is the window's first view. In - a tab DuckLocal's root holds that place — it has to, or DuckLocal's own dialogs would break — so - `window.open_dialog`, `window.open_sheet` and `window.push_toast` throw a `TypeError` - reading *needs a ShellRoot as the window's first view*, which surfaces the way any other - error from your code does. A tooltip is the exception, and the one silent one: it is - attached by a hover listener, which has nothing to throw into, so a tooltip in an app - simply never appears. Build app UI that stays inside its own bounds — an expanded region - rather than a modal. -- **Reload is implemented by DuckLocal, not by the shell's watcher.** gpui-shell's hot reload - hangs off `runtime.watch` / `runtime.refresh`, which only work for a `ShellRoot` the runtime - built itself; that path is not reachable from an embedding host, so DuckLocal watches the - folder itself and reloads through `load_application` / `mount_application`. The consequence: - reload watches `.js`/`.mjs` files only. Any other file an app reads is not watched. -- **A reload compiles on the UI thread.** The script runtime is not sendable, so mounting an - app happens on the main thread; a very large module graph will make the window stutter - while it loads. The first load is deferred until after the tab's first frame, so the tab - appears immediately either way. -- **One connection for all apps, and no cancellation.** Apps do not block the main - window, but they do block each other: they share one connection behind one lock, so a long - query in one app delays every other app's. One connection each is not available — a - host function is handed arguments, not a caller, so nothing at the moment `query()` runs - says which app asked. Cancelling a query in flight is not exposed either; keep an app's - statements bounded rather than counting on stopping one. -- **`row_count` is what came back, not what matched.** With `truncated: true`, the rows that - would have followed are not available; aggregate in SQL rather than counting in JavaScript. diff --git a/docs/assets/intro.jpg b/docs/assets/intro.jpg deleted file mode 100644 index f5bde47..0000000 Binary files a/docs/assets/intro.jpg and /dev/null differ diff --git a/docs/assets/logo.png b/docs/assets/logo.png deleted file mode 100644 index 485b984..0000000 Binary files a/docs/assets/logo.png and /dev/null differ diff --git a/docs/cli.md b/docs/cli.md deleted file mode 100644 index 37fc2f3..0000000 --- a/docs/cli.md +++ /dev/null @@ -1,260 +0,0 @@ -# AI CLI and official skill - -**[中文](zh/cli.md)** · [Docs](index.md) - -## Run without a window - -`ducklocal query` executes SQL without initializing the GUI, history, registered files, language settings, or the GUI's global connection. Each invocation owns a new connection. With no command (or file, directory, and glob arguments), the existing GUI still opens. To open a GUI path named `query`, `profile`, `export`, `check` or `dash`, use `./query`, `./profile`, `./export`, `./check`, `./dash`. - -`ducklocal export` is the one subcommand that starts the window platform, because an analysis app renders and rendering needs a window: it opens one hidden window, draws one frame, and exits when the app has stopped asking the database. Nothing appears on screen. - -The supported release target remains **macOS 12+, Apple silicon**. There is no separate DuckDB CLI dependency. A build containing this CLI can be invoked directly inside its app bundle: - -```bash -/Applications/DuckLocal.app/Contents/MacOS/ducklocal --help -/Applications/DuckLocal.app/Contents/MacOS/ducklocal --version -``` - -Alternatively, build this checkout with `cargo build --locked` and use `./target/debug/ducklocal`. The examples below assume that binary is available as `ducklocal` on your shell's PATH. Older releases may not have `query`; check help first. JSON and Parquet support are bundled, not downloaded at query time. - -## Query options - -```bash -ducklocal --help -ducklocal --version -ducklocal query --help -ducklocal query --sql "SELECT 1 AS n" -ducklocal query --sql "SELECT 1 AS n" --format md -ducklocal query --sql-file analysis.sql --limit 100 -ducklocal query --sql-file - < analysis.sql -ducklocal query --database warehouse.duckdb --sql "SHOW TABLES" -``` - -- Exactly one of `--sql SQL` and `--sql-file FILE` is required. Files must be UTF-8; `-` reads stdin. SQL must contain exactly one statement. DuckDB's parser validates this before opening the target database or executing anything; comments, quoted semicolons, and trailing semicolons work. Scripts are rejected, not partially executed. -- `--database PATH` opens an **existing file, read-only by default**. Omit it for a fresh in-memory database. `--read-write` requires `--database` and explicitly permits writes and creating a database (not parent directories). Database errors never fall back to memory. -- `--limit N` is a positive integer, defaults to 1000, and limits returned rows. A 2,000,000-cell budget also applies; an extra row is read to determine `truncated`. Neither limit constrains query computation or DuckDB's result buffering. Large individual cells can still be large. -- `--format json` (the default) is the contract below. `--format md` renders the same result as a Markdown table, for a document or an agent reading it. Nothing else changes: the same SQL, the same limits, the same errors. -- Repeated/unknown flags, missing values, empty SQL, and invalid limits are errors. Options use separate values, not `--flag=value`. Relative paths resolve against the process working directory, not the SQL file directory. Quote shell paths; inside SQL, double apostrophes (`'O''Brien.csv'`). SQL identifiers use double quotes. - -### Markdown output - -`--format md` is a rendering, not a second contract: the values are the ones the JSON holds, written the way the results grid shows them. - -| Value | Rendered as | -| --- | --- | -| `NULL` | `NULL` | -| integer, decimal, float | the digits the value carries, exactly — a `DECIMAL(38,10)` keeps its scale, `NaN`/`inf`/`-inf` are spelled out | -| `DATE` | `YYYY-MM-DD` | -| `TIMESTAMP` | `YYYY-MM-DD HH:MM:SS[.ffffff]`, with the fraction only when the value has one. No timezone is appended; `--format json` keeps the raw count | -| `TIME` | `HH:MM:SS[.ffffff]` | -| `INTERVAL` | `1 months 2 days 3000 ns` | -| `BLOB`, `GEOMETRY` | `0x00ff`, elided with `…` when long | -| `LIST`, `STRUCT`, `MAP` | `[1, 2]`, `{x: 1}`, elided with `…` past 120 characters | - -A `|` in a cell is escaped and a newline becomes `
`, so a value cannot break the table it is in. A result with no rows is followed by `_0 rows._`, and a result that was cut short by `--limit` or the cell budget by `_Truncated at N rows: …_` — a preview should not read like a complete answer. Errors are unchanged: JSON on stderr, empty stdout. - -**Read-only is not a filesystem or network sandbox.** SQL such as `COPY` can write files even when the database is read-only. Authorize output paths, overwrites, database mutations, extension installation, and external access before execution. DuckLocal does not automatically install extensions or retry operations. Explicit `INSTALL`/`LOAD` SQL remains possible with authorization; installed extensions may autoload. A failed command can already have performed side effects (for example before an output failure); inspect the destination before retrying. - -## Explore and convert files - -After confirming the actual file exists: - -```bash -ducklocal query --sql "DESCRIBE SELECT * FROM 'sales.csv'" -ducklocal query --sql "SELECT * FROM 'sales.csv'" --limit 20 -ducklocal query --sql "SELECT sum(amount) AS total FROM 'sales.csv'" -# Only after authorizing this output path: -ducklocal query --sql "COPY (SELECT * FROM 'sales.csv') TO 'sales.parquet' (FORMAT PARQUET)" -ducklocal query --sql "SELECT count(*), sum(amount) FROM 'sales.parquet'" -ducklocal query --sql "DESCRIBE SELECT * FROM 'events.json'" -``` - -Do not guess `amount` exists: inspect DESCRIBE first. Compute full aggregates in SQL rather than summing a truncated preview. CLI calls do not share the GUI's registered views or each other's in-memory tables. Persist explicitly when needed: - -```bash -# These are separate statements and separate invocations, with explicit write permission. -ducklocal query --database warehouse.duckdb --read-write --sql "CREATE TABLE sales AS SELECT * FROM 'sales.csv'" -ducklocal query --database warehouse.duckdb --sql "SELECT count(*) FROM sales" -``` - -A database lock conflict is an error. Close the other writer (including the GUI), or use an authorized copy; never delete lock files or silently create a different database. - -## JSONL and nested JSON - -`read_ndjson_objects` is the reader for newline-delimited JSON (one object per line, `.jsonl`/`.ndjson`); `read_json`/`read_json_auto` read JSON documents (an array or a single object). Name the reader explicitly for a JSONL file rather than relying on auto-detection. `json_extract_string(col, '$.a.b[0].c')` walks a nested path — object keys and array indexes — and answers VARCHAR: - -```bash -ducklocal query --sql "SELECT json_extract_string(event, '$.payload.items[0].sku') AS sku, count(*) AS n FROM read_ndjson_objects('events.jsonl') GROUP BY 1 ORDER BY n DESC" -``` - -Use `json_extract` instead when the leaf is not a string and you want it as JSON. - -## Profile a relation - -`ducklocal profile` answers the questions that decide a chart, a scale and a number format, before anything is built on them. `DESCRIBE` says a column is a `DOUBLE`; a profile says it uses two decimals, that its largest value is a thousand times its median, and that its dates skip eleven days in the middle. - -```bash -ducklocal profile sales.csv -ducklocal profile orders.parquet -ducklocal profile "my orders" --database warehouse.duckdb -``` - -TARGET is a data file — `csv`, `tsv`, `txt`, `parquet`, `json`, `ndjson`, `jsonl` — or a workbook (`xlsx`, `xls`, `xlsb`, `ods`; its first sheet is imported as a TEMP table and profiled), or, with `--database`, a table or view name. A dotted name is read as `schema.table` and each part is quoted, so a name with a space or a capital letter works as written. A target that names nothing is an argument error (exit 2), not a SQL one. - -One JSON object comes back: `target`, `relation` (the SQL the statistics ran against), `row_count`, `elapsed_ms`, and `columns` in the relation's own order. Per column: - -| Field | Meaning | -| --- | --- | -| `name`, `type` | as `DESCRIBE` reports them | -| `nulls` | rows where the column is `NULL` | -| `distinct` | exact, not estimated | -| `unique` | present and `true` when `distinct` equals `row_count` | -| `min`, `max` | as text, so a `DECIMAL` keeps its digits | -| `decimals` | digits after the point the column **actually uses**, which is not its declared scale | -| `median` | a value the column holds, not an interpolation between two of them | -| `max_over_median` | how many times the middle value the largest one is — the number that decides a linear axis from a logarithmic one | -| `covered_days`, `span_days`, `missing_days` | days the column names, days between its ends, and the difference: `0` is a continuous series, anything else is holes a line would draw over | - -`decimals`, `median` and `max_over_median` appear on numeric columns; the day fields on dates and timestamps. `LIST`, `STRUCT`, `MAP` and `UNION` columns report `nulls` only — `min` and `max` are not defined on them. - -Statistics are exact and read the whole relation, so a profile costs a full scan. `--limit` does not apply: a profile of a sample is not a profile. - -## Export an app as a static HTML file - -`ducklocal export` answers "what was that app showing?" for someone who does not have DuckLocal. It runs the app once — the same runtime, the same host module, the same database rules as `query` — and writes the statements its `query()` calls issued, with their results, into one self-contained HTML file. - -```bash -ducklocal export --html examples/analysis_app -ducklocal export --html --out report.html --database warehouse.duckdb apps/sales -``` - -- `--html` is required, and is the only format there is. APP is an app folder (one holding `main.js`) or its `main.js`, resolved the way the GUI resolves it; a path that names neither is an argument error (exit 2). -- `--out FILE` names the destination and defaults to `./.html` in the working directory. An existing file is refused unless `--force` is given — checked before the app runs, so a refusal costs nothing. -- `--database PATH` and `--read-write` mean exactly what they mean for `query`: an existing file, read-only unless asked otherwise, in memory when omitted. The app runs on that connection, so an app that writes needs `--read-write`. -- `--timeout SECONDS` is a positive integer, default 15: the capture stops when the app has stopped asking the database for a moment, or at the deadline, whichever comes first. -- The command starts the window platform, because an app renders and rendering needs a window. The window is hidden and never appears. - -One JSON object goes to stdout: - -```json -{"html":"/abs/report.html","app":"/abs/app","queries":2,"rows":212,"app_errors":0,"captured_ms":630,"stop_reason":"settled"} -``` - -`queries` counts the captured statements, `rows` the rows across those that succeeded, and `app_errors` what the app logged as an error while it ran. `stop_reason` is `settled` when the app went quiet on its own, `deadline` when `--timeout` cut the capture short — then the report may be missing statements, and the HTML shows a visible warning saying so. - -The report holds the app folder, the database it ran against, the export time, and one section per statement: the SQL, its columns and their Arrow types, the result as a table, and a bar chart when a result is a name and a number per row. A `catalog()` call becomes an appendix of tables, views and columns. Statements an app runs more than once appear once, with the last result. - -The report does **not** hold the app's own layout or its charts. An app draws native components, and a chart's bars are decided while it is laid out, not described anywhere that can be read out — so the export shows the data the app was built from, not the interface it built. Filters, toggles and later refreshes are not represented either, and there is no JavaScript in the file. It is a snapshot of the app's loading state, and it says so at the top. - -The app's JavaScript runs with the same privileges it always has: `query()` can `COPY`, `ATTACH` and write files. The export is not a sandbox. - -Exit codes: **0** with the file written; **2** for a bad command line, an app path that names nothing, or a destination that already exists; **1** when the app could not be loaded (nothing is written) or when it loaded and then failed (the report is written anyway, with the error in it, and the message names the file). An app failure reports the error kind `app`. - -## Check a dashboard spec - -A `.dash` file declares a dashboard the way Terraform declares infrastructure — queries and plots as blocks, references between them — rather than scripting one as an analysis app. The two formats coexist: the spec covers query + standard plot and is easy for a person or an agent to diff; an app stays for bespoke layout and interaction. - -```hcl -query "latency" { - sql = <