diff --git a/.gitignore b/.gitignore index 0e0ba36..084d910 100644 --- a/.gitignore +++ b/.gitignore @@ -1,5 +1,6 @@ node_modules/ dist/ +public/pdfjs/ playwright-report/ test-results/ src-tauri/target/ diff --git a/README.zh-CN.md b/README.zh-CN.md index 578a30c..aaee591 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -20,6 +20,7 @@ - **一本地书库：** PDF、EPUB、DOCX 不再需要分别使用不同阅读器。 - **本地优先：** 书库、阅读进度、笔记、索引和备份保存在你的电脑中。 - **AI 由你主动触发：** 普通阅读和本地搜索不需要 AI；只有你明确发起 AI 操作时才会发送相关内容。 +- **离线 AI 问答：** 已安装 Ollama 并下载模型后，可一键连接本地文字模型；支持识图的模型还可用于图片区域问答。LM Studio 文字通道暂为实验性，实机验证安排在 v0.2.1。 - **面向系统学习：** 本地搜索、批注、阅读历史、多语言界面、备份与恢复集中在一个桌面工作流中。 - **源码完整开放：** Rust + React + Tauri 全部源码采用 Apache-2.0 协议，开发者可以自由修改和改进。 diff --git a/docs/ai-services.md b/docs/ai-services.md index 62ee66d..2ba30ee 100644 --- a/docs/ai-services.md +++ b/docs/ai-services.md @@ -1,6 +1,6 @@ # AI services -TextbookLens supports user-configured official OpenAI, Gemini, Anthropic, DeepSeek, and Kimi services. Requests leave from the Rust desktop layer; the browser UI does not fetch providers or configure a production base URL. Adding or replacing a key validates it before a key/profile update is committed. Keys are in Windows Credential Manager, not profile lists, SQLite/frontend state, or backups. +TextbookLens supports user-configured official OpenAI, Gemini, Anthropic, DeepSeek, and Kimi services, plus [Ollama local text/image models and experimental LM Studio text models](local-models.md). Requests leave from the Rust desktop layer; the browser UI does not fetch providers or configure a production base URL. Adding or replacing a cloud key validates it before a key/profile update is committed. Cloud keys are in Windows Credential Manager, not profile lists, SQLite/frontend state, or backups. Local profiles use a discovered loopback port and require no cloud key. ## Configured capability registry diff --git a/docs/local-models.md b/docs/local-models.md new file mode 100644 index 0000000..07069c9 --- /dev/null +++ b/docs/local-models.md @@ -0,0 +1,32 @@ +# Offline local models + +In **Settings → AI services**, select **Auto-connect local AI**. The same button is available during onboarding. TextbookLens discovers an existing Ollama or LM Studio installation, starts its local service when needed, finds downloaded text models, and tests a model with a short synthetic question. It adds the discovered models and selects a tested model as the default for text answers. Repeating the action updates existing profiles instead of duplicating them. + +No API key is needed for the supported unauthenticated local services. The application does not install software, download models, use cloud models, or fall back to a cloud provider when a local request fails. A service that requires authentication is reported separately; TextbookLens does not change its authentication settings. + +## Supported workflow + +- **Ollama:** discover its executable from the normal Windows installation or PATH; use the local `OLLAMA_HOST` port when configured, otherwise 11434. Start `ollama serve` if the service is stopped. Use `/api/tags` and `/api/show` to find installed completion models. Reject cloud/remote models before each inference. Load the model on demand through `/api/chat`. +- **LM Studio (experimental; real-machine validation deferred to v0.2.1):** discover `lms.exe` from the LM Studio home or PATH and its saved local server port (default 1234). Start the server with the installed CLI. Read `lms ls --llm --json`, exclude models on other devices, and use `lms load` when needed. LM Link-aware versions are loaded with `--local`, and the resulting local instance is verified before sending a question. +- Models are stored as separate profiles, with their own model identifier, context budget, and numeric loopback port. Use **Use for learning** to switch models. Local profiles do not create or require Windows credentials. +- Local models support text selection questions, book questions, follow-up answers, and the teaching-instruction test. Ollama models whose installed metadata advertises vision can also be selected with **Use for vision** for image-region questions. Image input is checked again before every request; text-only and remote models never receive images. LM Studio vision, structured page indexing, and cloud file extraction remain unavailable for local profiles in this version. + +Vision capability is saved per local profile and survives restart. One request accepts one PNG/JPEG/WebP image, at most 2 MiB encoded, 2048 pixels on either side, and 1,048,576 decoded pixels. TextbookLens reserves additional context for that image. Discovered Ollama vision models use up to 16,384 context tokens, subject to the model's declared limit. Reconnecting retains an existing connected local text default; select a vision default independently in AI services. + +The software must be functional and its inference engine and model files must already be installed. Sufficient RAM/VRAM is still required. Initial model loading can take several minutes. A missing runtime, missing text model, authentication requirement, or failed answer test is shown in the connection result. + +All TextbookLens local HTTP requests use `127.0.0.1`, bypass proxies, and refuse redirects. Application-level validation checks model locality as well: an Ollama localhost endpoint can serve cloud models, and LM Studio can expose remote devices through LM Link. + +## References + +- [Ollama model list](https://docs.ollama.com/api/tags), [cloud/local model distinction](https://docs.ollama.com/cloud). +- [LM Studio CLI model list](https://lmstudio.ai/docs/cli/local-models/ls), [model loading](https://lmstudio.ai/docs/cli/local-models/load), [server startup](https://lmstudio.ai/docs/cli/serve/server-start). +- [LM Studio CLI implementation](https://github.com/lmstudio-ai/lms): `src/subcommands/list.ts` and `src/subcommands/load.ts` define local device metadata and local-only loading. + +## Verification + +Synthetic loopback tests cover local/remote filtering, credential-free persistence and deletion, repeated discovery, stream termination, and rejection of redirects. Learning preparation tests cover both selection and book questions without credential access. The database migration contract covers existing profile references and historical upgrades. + +An opt-in Rust test, `ai::local::tests::installed_local_runtime_offline_smoke`, exercises discovery, persistence, the normal provider runtime, and a real streamed answer using installed software and a temporary database. It never uses a user's books or cloud keys. Run it with `cargo test --manifest-path src-tauri/Cargo.toml --lib installed_local_runtime_offline_smoke -- --ignored`, using the isolated build directories described in CONTRIBUTING.md. + +Verified on 2026-09-07: 55 frontend tests, 347 Rust unit tests, and 37 binding, database, migration, and profile lifecycle tests passed. The installed Ollama 0.33.3 service with `deepseek-r1:8b` passed discovery, registration, and a complete streamed answer. A separate unused loopback port verified automatic startup and discovery of already downloaded models; that test-owned service was stopped afterward. LM Studio model filtering and stream handling have automated coverage; an actual LM Studio installation was not available for an end-to-end run on this machine. diff --git a/docs/releases/v0.2.0-validation.md b/docs/releases/v0.2.0-validation.md new file mode 100644 index 0000000..b46978e --- /dev/null +++ b/docs/releases/v0.2.0-validation.md @@ -0,0 +1,63 @@ +# v0.2.0 发布准备验证记录 + +2026-09-08。本记录对应 v0.2.0 公开发布候选。安装包未进行代码签名；公开文件及校验值以 [GitHub Release](https://github.com/gh615280-maker/TextbookLens/releases/tag/v0.2.0) 为准。 + +## 安装包 + +| 包 | 字节数 | 运行时处理 | +| ------------ | ----------: | ----------------------------- | +| 默认 EXE | 10,877,755 | 复用 WebView2，缺少时联网下载 | +| 默认 MSI | 13,963,264 | 同上 | +| 完整离线 EXE | 272,870,116 | 附带 WebView2 离线运行时 | +| 完整离线 MSI | 272,834,560 | 同上 | + +不包含 Ollama、LM Studio 或模型权重。微软 WebView2 原始安装文件为 258,614,480 字节；新增 PDF 字体和解码资源原始总量约 3 MB。安装文件经过压缩，原始文件大小不能直接相加。 + +构建目录：`D:\CodexBuild\textbooklens-v020-20260908`。最终包位于 `artifacts/standard` 和 `artifacts/offline`；校验值见 `evidence/package-checksums.json`。源码基于 `9a193045dcb3a10034393786118f88cedab144f8` 加本次受控修改，完整路径与哈希见 `evidence/source-manifest.json`。 + +## 已完成 + +- PDF 字体、CMap 和扫描页解码资源离线打包，导入/阅读/页面渲染共用配置；页面解码失败显示错误。 +- 修复页底提问表单越界、书库卡片/列表布局、目录跳转后工具栏不可见、调整大小手柄与阅读滚动条重叠。前后端版本统一为 0.2.0。 +- EPUB 的 xmldom 更新为 0.8.15，并加入 GHSA-6gmq-8vp8-gcm6 检查；生产 npm audit 为 0 项漏洞。 +- 前端和脚本 100 个文件、411 项测试通过；浏览器端到端 65 项通过；Rust Release 490 项通过、3 项默认忽略。 +- 另外执行两项真实环境测试：已安装模型的一键连接/文字回答通过；独立端口 11465 的 Ollama 自动启动通过。测试服务随后关闭，正常用户服务未停止。 +- TypeScript、ESLint、Prettier、Release Clippy、生成绑定字节一致性、依赖/许可/素材检查通过。最终分发内容扫描通过：285 个文件、580,170,456 字节。 + +## Windows 安装与升级 + +环境为关闭网络的 Windows Sandbox（Windows 11，构建 22621），数据与用户书库独立。 + +- 干净环境原先无 WebView2：完整离线 EXE 安装、首次启动和卸载通过；完整离线 MSI 安装和启动通过。 +- v0.1.0 EXE 覆盖升级：数据库 15→17，已导入 DOCX、2 条批注、2 条问答消息、3 个检索片段、设置、服务配置和保存凭据逐项保留，数据库完整性与外键检查正常。 +- 升级后实际打开 DOCX 并跳转目录，工具栏保持可见。 +- 已有 WebView2 时，默认小 EXE/MSI 的断网安装、启动和卸载通过；MSI 卸载后凭据保留，随后清理测试凭据。 +- 默认 MSI 中导入桌面《数学分析习题课讲义下》的副本并显示扫描目录页；浏览器另验证第 1、10 页非空且没有外部资源请求。 +- 原桌面 8 本教材的 SHA-256 全部与测试前一致。 + +中间曾有前一 Sandbox 客户端未退出；清理旧客户端后重新确认当前实例，并完成小包卸载、MSI 安装、扫描教材阅读和卸载。重复启动的中间结果不算额外独立样本。 + +## 本地视觉补测 + +使用已有 Ollama/Qwen3-VL 4B，经同一源码的 Release 应用库、真实 ProviderRuntime 和本地适配层测试，数据写入专用数据库，图片只发到本机。 + +| 样例 | 运行结果 | 内容核对 | +| ------------------------------ | -------------------------------- | ---------------------------------------------------------------- | +| 数学分析第 10 页，级数收敛条件 | 正常返回，约 4 秒 | 正确：通项趋于 0 是必要条件 | +| 公共经济学第 18 页，补贴图表 | 正常返回，约 5 秒 | 坐标轴和补贴对象正确，但把“明补效率损失更小”回答反了，质量不通过 | +| 1158×1700 图片 | 发送前拒绝超出本地像素限制的输入 | 当前提示较笼统 | +| 运行中取消 | 约 0.1 秒结束，无完成事件 | 适配层通过 | +| 取消后重试 | 正常返回完整正文 | 运行通过 | + +证据：`evidence/textbook-vision/adapter-results.json`。其中 `passed` 表示传输/状态断言，不能理解为事实正确。初始直接请求探针用了 768 token 上限，图表没有返回正文；按应用实际的 4096 token 上限复测后返回正文，但仍存在上述事实错误。没有为单个失败样例调整产品提示词。 + +## 验证边界与后续项 + +- 未进行真实显存/内存耗尽压力测试；未覆盖所有教材的每页/手写公式，也未完整复测原生 UI 的视觉取消/重试组合。适配层验证不能替代这些界面测试。 +- 未独立验证无 WebView2 时默认小包的联网下载路径；断网新装应选完整离线包。 +- 大尺寸扫描 PDF 默认缩放仍需手动调整，没有自动适应页宽；测试教材的原始标题元数据还含异常字符。这两项不在此次选定修复清单内。 +- 安装包未代码签名；安装验证不代表已获得 Windows 下载信誉。 +- LM Studio 文字实机验证按约定留到 v0.2.1；本版不承诺 LM Studio 视觉。 +- PDF 文字层、EPUB、DOCX 的文本提取不需要大模型。本地全书 OCR/视觉索引尚未实现；未来可用独立 OCR 或视觉模型，并不必然依赖本地大模型。当前全书扫描识别使用已配置且获确认的云端通道。 + +结论：安装包与主要阅读修复可作为候选版交付；本地视觉应明确为辅助能力，不能宣称复杂公式/图表可靠。本记录不能描述成全部场景均通过。 diff --git a/docs/releases/v0.2.0.md b/docs/releases/v0.2.0.md new file mode 100644 index 0000000..8adaa62 --- /dev/null +++ b/docs/releases/v0.2.0.md @@ -0,0 +1,24 @@ +# TextbookLens v0.2.0 + +## 本次范围 + +- Ollama 已安装模型的一键连接与离线文字问答。 +- 支持 vision 的 Ollama 模型可独立设为视觉服务，用于图片区域问答；原有文字默认模型保持独立。 +- EPUB 图片问答的位置保存和重新打开修复。 +- 扫描 PDF 的本地解码资源随应用提供；修复 JBIG2 页空白。 +- 修复页底提问表单越界、书库卡片和列表排版、目录跳转后工具栏不可见，以及调整大小手柄遮挡阅读滚动条。 + +## 使用边界 + +- 正式目标平台为 Windows 11 x64。模型运行软件和模型文件需事先安装；应用的一键连接不会自动下载模型。 +- 本地文字和视觉已在 Ollama 上实测。LM Studio 文字通道为实验性，实机验证留到 v0.2.1；本版不承诺 LM Studio 视觉输入。 +- 普通 PDF 的文字层、EPUB 和 DOCX 的本地文本读取不需要大模型。 +- 全书扫描识别／视觉索引仍使用已配置且支持该操作的云端服务，经用户确认后运行；本版没有独立本地 OCR 引擎或本地全书视觉索引。 +- 本地图片问答当前每次接受一张图片，限制见 `docs/local-models.md`。识图成功不保证复杂公式、手写内容和所有题目都正确。 +- 安装包、升级验证、签名状态和已知问题以[本次验证报告](v0.2.0-validation.md)为准。公开安装包请从 [GitHub Release](https://github.com/gh615280-maker/TextbookLens/releases/tag/v0.2.0) 下载。 + +## 安装包选择 + +- 默认安装包复用电脑已有的 WebView2；缺少运行时时需要联网下载安装。完成安装且本地模型已就绪后，阅读和本地问答可以断网使用。 +- 完整离线包附带微软 WebView2 离线运行时，可用于缺少该运行时的电脑断网安装。体积约 273 MB，主要来自约 259 MB 的运行时安装文件。 +- 两种安装包都不包含本地大模型文件。完整离线包使用 `npm run tauri -- build --config src-tauri/tauri.offline.conf.json` 构建；应放在单独的分发目录，避免与默认包混淆。 diff --git a/docs/superpowers/plans/2026-09-08-v020-release-preparation.md b/docs/superpowers/plans/2026-09-08-v020-release-preparation.md new file mode 100644 index 0000000..ec0040b --- /dev/null +++ b/docs/superpowers/plans/2026-09-08-v020-release-preparation.md @@ -0,0 +1,14 @@ +# v0.2.0 release preparation + +Scope approved in the conversation on 2026-09-08. Work in the current shared checkout; do not include unrelated experiments or user files. Build and test artifacts use a separate task directory. Do not tag, push, or publish as part of preparation. + +- [x] Fix PDF decoding in `src/features/reader/pdf/PdfReaderAdapter.ts`, `src/features/import/parsers/pdf-parser.ts`, and `src/features/indexing/pdf-page-renderer.ts` using shared local PDF.js resources. Bundle decoder/font resources from the pinned dependency, retain script confinement, and surface page-render failures. Verify JBIG2 pages from the desktop test corpus and a synthetic browser regression. +- [x] Fix `SelectionMenu.tsx` positioning when its form grows; retain cancellation and focus behavior. Verify bottom-edge question/note forms in normal and narrow windows. +- [x] Restore library card/table layout, persistent reader navigation, and distinct resize/scroll hit areas in the library components, `ReaderLayout.tsx`, and reader styles. Verify long book titles and actual desktop interactions. +- [x] Set application, lockfile, and package versions to 0.2.0. Document Ollama text/vision scope; defer LM Studio real-machine validation to 0.2.1. Existing cloud indexing remains available; do not claim local whole-book OCR/indexing. +- [x] Run affected tests, full application regression, formatting, lint, bindings, dependency/license/fixture checks, and release artifact scans. Produce Release EXE/MSI from a controlled source snapshot, preserving existing migration bytes. +- [x] Exercise fresh Windows installation and previous-version upgrade; verify library/history/settings/profile retention and uninstall behavior. Recheck actual textbook image questions, oversized input, cancellation, stopped-service recovery, and offline startup where the environment permits. Record unexecuted gates explicitly. + +Acceptance: the two observed P1 defects are fixed; the selected common UI defects are fixed; installable 0.2.0 artifacts and reproducible evidence are available. Remaining platform or model limitations are stated accurately. No automatic cloud fallback, model download, or whole-book local OCR is added. + +Validation and remaining coverage limits: [v0.2.0-validation.md](../../releases/v0.2.0-validation.md). Completed preparation does not mean every platform/model scenario passed. diff --git a/e2e/epub-local-reader.spec.ts b/e2e/epub-local-reader.spec.ts new file mode 100644 index 0000000..897bc04 --- /dev/null +++ b/e2e/epub-local-reader.spec.ts @@ -0,0 +1,290 @@ +import { expect, test, type Frame, type Page } from '@playwright/test'; +import { readFileSync } from 'node:fs'; +import JSZip from 'jszip'; + +const BOOK = '31000000-0000-4000-8000-000000000001'; +const SECTION = '31000000-0000-4000-8000-000000000002'; +const PROFILE = '31000000-0000-4000-8000-000000000003'; + +async function archive() { + const zip = new JSZip(); + zip.file('mimetype', 'application/epub+zip'); + zip.file( + 'META-INF/container.xml', + '', + ); + const items = Array.from( + { length: 6 }, + (_, i) => + ``, + ).join(''); + const spine = Array.from( + { length: 6 }, + (_, i) => ``, + ).join(''); + zip.file( + 'OPS/package.opf', + `synthetic-epubSynthetic EPUBen${items}${spine}`, + ); + zip.file( + 'OPS/Images/diagram.png', + readFileSync('fixtures/source/figure-energy.png'), + ); + zip.file( + 'OPS/Styles/book.css', + 'body { margin: 0; padding: 20px; } img { width: 64px; height: 40px; } #target { color: rgb(23, 45, 67); }', + ); + for (let i = 0; i < 6; i++) { + const body = + i === 0 + ? 'Synthetic cover' + : i === 5 + ? '

Synthetic chapter

First eigenvalue occurrence.

Second eigenvalue occurrence.

Synthetic figure' + + '

Local archived text only.

'.repeat(30) + : ''; + zip.file( + `OPS/Text/s${i}.xhtml`, + `Synthetic ${i}${body}`, + ); + } + return [...(await zip.generateAsync({ type: 'uint8array' }))]; +} + +async function installBackend(page: Page) { + await page.addInitScript( + ({ bytes, bookId, sectionId, profileId }) => { + const state = { + saved: [] as Array>, + unknown: [] as string[], + }; + const settings = { + onboardingCompleted: true, + activeProviderProfileId: profileId, + defaultLearningProfileId: profileId, + defaultVisionProfileId: null, + theme: 'system', + contextMode: 'standard', + uiLanguage: 'zh-CN', + uiLanguageInitialized: true, + firstReaderHintCompleted: true, + }; + const book = { + id: bookId, + title: 'Synthetic EPUB', + originalFilename: 'synthetic.epub', + author: null, + language: 'en', + format: 'epub', + importStatus: 'ready', + importErrorCode: null, + importErrorMessage: null, + importErrorStage: null, + readingProgress: 0, + fullTextQaReady: false, + indexAggregate: { + status: 'not_required', + totalPages: 0, + indexedPages: 0, + reviewPages: 0, + failedPages: 0, + }, + createdAt: '2026-09-08T00:00:00Z', + updatedAt: '2026-09-08T00:00:00Z', + lastOpenedAt: null, + }; + const testWindow = window as unknown as { + __epubReaderTest: typeof state; + __TAURI_INTERNALS__: { + invoke( + command: string, + payload?: Record, + ): Promise; + }; + }; + testWindow.__epubReaderTest = state; + testWindow.__TAURI_INTERNALS__ = { + async invoke(command, payload = {}) { + switch (command) { + case 'get_app_settings': + case 'initialize_ui_language': + case 'complete_first_reader_hint': + return settings; + case 'get_reader_settings': + return { + fontScale: 1, + lineHeight: 1.6, + readerWidth: 72, + pdfZoom: 1, + theme: 'system', + }; + case 'get_reader_bootstrap': + return { + book, + lastLocator: JSON.parse( + sessionStorage.getItem('epub-reader-progress') ?? 'null', + ), + }; + case 'read_book_source': + return new Uint8Array(bytes); + case 'list_reader_sections': + return [ + { + id: sectionId, + parentId: null, + ordinal: 0, + title: 'Synthetic chapter', + locator: { + format: 'epub', + cfi: 'epubcfi(/6/12!/4/2/1:0)', + sectionId, + }, + }, + ]; + case 'list_provider_profiles': + return payload.operation === 'vision_learning' + ? [] + : [ + { + id: profileId, + kind: 'ollama', + displayName: 'Synthetic local model', + modelId: 'synthetic-local', + contextWindowTokens: 8192, + isActive: true, + credentialStatus: 'not_required', + validatedAt: '2026-09-08T00:00:00Z', + }, + ]; + case 'list_annotation_markers': + case 'search_book': + case 'list_books': + return []; + case 'claim_index_render_batch': + return { claims: [] }; + case 'save_reading_progress': { + const locator = payload.locator as Record; + if (locator.sectionId !== sectionId) + throw { code: 'INVALID_INPUT' }; + state.saved.push(locator); + sessionStorage.setItem( + 'epub-reader-progress', + JSON.stringify(locator), + ); + return; + } + default: + state.unknown.push(command); + throw new Error('Unexpected test IPC: ' + command); + } + }, + }; + }, + { + bytes: await archive(), + bookId: BOOK, + sectionId: SECTION, + profileId: PROFILE, + }, + ); +} + +async function frameWith( + page: Page, + selector: string, +): Promise { + for (const frame of page.frames()) + if (frame !== page.mainFrame() && (await frame.locator(selector).count())) + return frame; + return undefined; +} + +test('real EPUB archive restores images, maps skipped spine entries, selects text, and restores a canonical reading location', async ({ + page, +}) => { + const external: string[] = []; + page.on('request', (request) => { + if ( + /^https?:/.test(request.url()) && + !request.url().startsWith('http://127.0.0.1:1420/') + ) + external.push(request.url()); + }); + await installBackend(page); + const csp = JSON.parse(readFileSync('src-tauri/tauri.conf.json', 'utf8')).app + .security.csp as string; + await page.route(`**/books/${BOOK}/read`, async (route) => { + const response = await route.fetch(); + await route.fulfill({ + response, + headers: { ...response.headers(), 'content-security-policy': csp }, + }); + }); + await page.goto(`/books/${BOOK}/read`); + await expect + .poll(async () => Boolean(await frameWith(page, '#cover'))) + .toBe(true); + const cover = (await frameWith(page, '#cover'))!; + await expect + .poll(() => + cover + .locator('#cover') + .evaluate( + (image: HTMLImageElement) => image.complete && image.naturalWidth > 0, + ), + ) + .toBe(true); + await expect(cover.locator('#cover')).toHaveCSS('width', '64px'); + await page.getByRole('button', { name: '目录', exact: true }).click(); + await page + .getByRole('button', { name: 'Synthetic chapter', exact: true }) + .click(); + await expect + .poll(async () => Boolean(await frameWith(page, '#target'))) + .toBe(true); + const chapter = (await frameWith(page, '#target'))!; + await expect(chapter.locator('#target')).toHaveCSS( + 'color', + 'rgb(23, 45, 67)', + ); + await expect + .poll(() => + chapter + .locator('#figure') + .evaluate( + (image: HTMLImageElement) => image.complete && image.naturalWidth > 0, + ), + ) + .toBe(true); + await chapter.locator('#target').dblclick({ position: { x: 84, y: 8 } }); + const menu = page.getByRole('menuitem', { name: '提问', exact: true }); + await expect(menu).toBeVisible(); + const wordBounds = (await chapter.locator('#target').boundingBox())!; + const menuBounds = (await menu.boundingBox())!; + expect(menuBounds.y).toBeGreaterThan(wordBounds.y); + expect(menuBounds.y - wordBounds.y).toBeLessThan(180); + await expect + .poll(() => + page.evaluate( + () => + JSON.parse(sessionStorage.getItem('epub-reader-progress') ?? 'null') + ?.sectionId, + ), + ) + .toBe(SECTION); + expect(external).toEqual([]); + await page.reload(); + await expect + .poll(async () => Boolean(await frameWith(page, '#target'))) + .toBe(true); + await expect( + (await frameWith(page, '#target'))!.locator('#target'), + ).toBeVisible(); + expect( + await page.evaluate( + () => + (window as unknown as { __epubReaderTest: { unknown: string[] } }) + .__epubReaderTest.unknown, + ), + ).toEqual([]); + expect(external).toEqual([]); +}); diff --git a/e2e/floating-learning-panels.spec.ts b/e2e/floating-learning-panels.spec.ts index 14ac44e..9b5bd04 100644 --- a/e2e/floating-learning-panels.spec.ts +++ b/e2e/floating-learning-panels.spec.ts @@ -646,6 +646,90 @@ learningTest( }, ); +for (const surface of [ + 'marker row', + 'marker button', + 'resize handle', +] as const) { + learningTest( + `H: reopened answer controls remain clickable over the reader ${surface}`, + async ({ page, backend }) => { + await openReader(page); + const requestId = await startSelection(page, backend, BLOCK_ALPHA, ALPHA); + await emit(page, backend, requestId, { + type: 'text_delta', + text: 'Durable overlapping answer.', + }); + await emit(page, backend, requestId, { type: 'completed' }); + await panelForAnswer(page, 'Durable overlapping answer.') + .getByRole('button', { name: 'Hide' }) + .click(); + + const readerMarker = page.locator( + '.docx-reader-markers .reader-marker-button', + ); + await readerMarker.click(); + const reopened = page.locator('[data-panel-id]'); + await expect(reopened).toContainText('Durable overlapping answer.'); + const panelId = await reopened.getAttribute('data-panel-id'); + const bottomHandle = reopened.getByRole('button', { + name: 'Resize learning panel bottom edge', + }); + for (let step = 0; step < 18; step += 1) { + await bottomHandle.press('ArrowUp'); + } + const target = + surface === 'marker row' + ? page.locator('.docx-reader-markers') + : surface === 'marker button' + ? readerMarker + : page.locator('[data-reader-resize-handle="e"]'); + const targetBox = await box(target); + const point = { + x: + targetBox.x + + targetBox.width * (surface === 'marker row' ? 0.25 : 0.5), + y: targetBox.y + targetBox.height / 2, + }; + const hide = reopened.getByRole('button', { name: 'Hide' }); + const original = await box(hide); + const mover = reopened.getByRole('button', { + name: 'Move learning panel', + }); + // Use the product's keyboard movement, keeping the reader overlay beneath + // a real button. Do not force clicks or change CSS/z-index in the test. + for (const [delta, negative, positive] of [ + [ + point.x - (original.x + original.width / 2), + 'ArrowLeft', + 'ArrowRight', + ], + [point.y - (original.y + original.height / 2), 'ArrowUp', 'ArrowDown'], + ] as const) { + const steps = Math.round(delta / 16); + for (let step = 0; step < Math.abs(steps); step += 1) { + await mover.press(steps < 0 ? negative : positive); + } + } + const moved = await box(hide); + expect(point.x).toBeGreaterThan(moved.x); + expect(point.x).toBeLessThan(moved.x + moved.width); + expect(point.y).toBeGreaterThan(moved.y); + expect(point.y).toBeLessThan(moved.y + moved.height); + await hide.click({ + position: { x: point.x - moved.x, y: point.y - moved.y }, + timeout: 3_000, + }); + await expect(reopened).toHaveCount(0); + // The reader's real marker remains usable when the floating panel hides. + await readerMarker.click(); + await expect(reopened).toHaveCount(1); + expect(await reopened.getAttribute('data-panel-id')).toBe(panelId); + await expect(reopened).toContainText('Durable overlapping answer.'); + }, + ); +} + learningTest( 'I: restart restores only durable history, captures current followup profile, and deletes atomically', async ({ page, backend }) => { diff --git a/e2e/selection-regions.spec.ts b/e2e/selection-regions.spec.ts index 39cec32..42d0681 100644 --- a/e2e/selection-regions.spec.ts +++ b/e2e/selection-regions.spec.ts @@ -239,7 +239,7 @@ class SyntheticSelectionBackend { : format === 'epub' ? { format: 'epub', - cfi: 'epubcfi(/6/2)', + cfi: 'epubcfi(/6/2!/4/2)', sectionId: SECTIONS.epub, } : { @@ -414,7 +414,7 @@ const selectionTest = test.extend<{ backend: SyntheticSelectionBackend }>({ /* eslint-enable react-hooks/rules-of-hooks */ selectionTest( - 'PDF scroll viewport stays attached to every resized frame edge', + 'PDF scroll viewport fills the content area while reserving resize gutters', async ({ page, backend }) => { backend.pdfZoom = 3; await page.addInitScript(() => { @@ -461,8 +461,8 @@ selectionTest( expect(geometry.documentRegion).toEqual({ left: geometry.frame.left + 1, top: geometry.frame.top + 1, - right: geometry.frame.right - 1, - bottom: geometry.frame.bottom - 1, + right: geometry.frame.right - 13, + bottom: geometry.frame.bottom - 13, }); expect(geometry.viewport).toEqual(geometry.documentRegion); expect(geometry.documentZoom).toBe('1'); diff --git a/e2e/v020-reader.spec.ts b/e2e/v020-reader.spec.ts new file mode 100644 index 0000000..0ae0c19 --- /dev/null +++ b/e2e/v020-reader.spec.ts @@ -0,0 +1,247 @@ +import { expect, test, type Page } from '@playwright/test'; +import { readFileSync } from 'node:fs'; + +test.use({ screenshot: 'off', trace: 'off', video: 'off' }); + +const BOOK = '42000000-0000-4000-8000-000000000001'; +async function installBackend(page: Page, bytes: number[], pages: number[]) { + await page.addInitScript( + ({ bytes, pages, bookId }) => { + const settings = { + onboardingCompleted: true, + activeProviderProfileId: null, + defaultLearningProfileId: null, + defaultVisionProfileId: null, + theme: 'system', + contextMode: 'standard', + uiLanguage: 'zh-CN', + uiLanguageInitialized: true, + firstReaderHintCompleted: true, + }; + const book = { + id: bookId, + title: '阅读验证教材', + originalFilename: 'reader-check.pdf', + author: null, + language: 'zh', + format: 'pdf', + importStatus: 'ready', + importErrorCode: null, + importErrorMessage: null, + importErrorStage: null, + readingProgress: 0, + fullTextQaReady: false, + indexAggregate: { + status: 'not_required', + totalPages: 0, + indexedPages: 0, + reviewPages: 0, + failedPages: 0, + }, + createdAt: '2026-09-08T00:00:00Z', + updatedAt: '2026-09-08T00:00:00Z', + lastOpenedAt: null, + }; + ( + window as unknown as { __TAURI_INTERNALS__: unknown } + ).__TAURI_INTERNALS__ = { + async invoke(command: string) { + switch (command) { + case 'get_onboarding_state': + return { canSkipOnboarding: true }; + case 'get_app_settings': + case 'initialize_ui_language': + case 'complete_first_reader_hint': + return settings; + case 'get_reader_settings': + return { + fontScale: 1, + lineHeight: 1.6, + readerWidth: 72, + pdfZoom: 1, + theme: 'system', + }; + case 'get_reader_bootstrap': + return { book, lastLocator: null }; + case 'read_book_source': + return new Uint8Array(bytes); + case 'list_reader_sections': + return pages.map((p, i) => ({ + id: `42000000-0000-4000-8000-${String(i + 2).padStart(12, '0')}`, + parentId: null, + ordinal: i, + title: `第 ${p} 页`, + locator: { + format: 'pdf', + startPage: p, + endPage: p, + rectsByPage: null, + }, + })); + case 'list_provider_profiles': + case 'list_annotation_markers': + case 'search_book': + return []; + case 'list_books': + return [ + book, + { + ...book, + id: '42000000-0000-4000-8000-000000000005', + title: + '一本用于检验较长中文书名、格式标识和导入状态能否清晰排列的测试教材', + format: 'epub', + }, + { + ...book, + id: '42000000-0000-4000-8000-000000000006', + title: + 'A long synthetic textbook title for testing the library layout', + format: 'docx', + }, + ]; + case 'claim_index_render_batch': + return { claims: [] }; + case 'save_reading_progress': + return; + default: + throw new Error('Unexpected test IPC: ' + command); + } + }, + }; + }, + { bytes, pages, bookId: BOOK }, + ); +} + +async function darkPixels(page: Page, number: number) { + const canvas = page + .locator(`.page[data-page-number="${number}"] canvas`) + .first(); + if (!(await canvas.count())) return 0; + return canvas.evaluate((canvas: HTMLCanvasElement) => { + const context = canvas.getContext('2d'); + if (!context || !canvas.width || !canvas.height) return 0; + const data = context.getImageData(0, 0, canvas.width, canvas.height).data; + let count = 0; + for (let i = 0; i < data.length; i += 64) + if ( + data[i] < 150 && + data[i + 1] < 150 && + data[i + 2] < 150 && + data[i + 3] > 0 + ) + count++; + return count; + }); +} + +test('library cards separate titles and status, and detail columns align', async ({ + page, +}) => { + await page.setViewportSize({ width: 850, height: 700 }); + await installBackend( + page, + [...readFileSync('fixtures/textbook.pdf')], + [1, 2], + ); + await page.goto('/library'); + await expect(page.locator('.library-card')).toHaveCount(3); + await expect(page.locator('.library-grid')).toHaveCSS( + 'list-style-type', + 'none', + ); + const card = page.locator('.library-card').nth(1); + const title = await card.locator('.library-item__title').boundingBox(); + const status = await card + .locator('.library-item__import-status') + .boundingBox(); + expect(title!.y + title!.height).toBeLessThanOrEqual(status!.y); + await page.getByRole('button', { name: '详细列表', exact: true }).click(); + const headers = page.locator( + '.library-details__header [role="columnheader"]', + ); + const cells = page + .locator('.library-details__row') + .first() + .locator('[role="gridcell"]'); + for (let i = 0; i < 6; i++) + expect( + Math.abs( + (await headers.nth(i).boundingBox())!.x - + (await cells.nth(i).boundingBox())!.x, + ), + ).toBeLessThan(1); +}); + +test('PDF navigation preserves the toolbar and resize handles do not cover its scrollbar', async ({ + page, +}) => { + await installBackend( + page, + [...readFileSync('fixtures/textbook.pdf')], + [1, 2], + ); + await page.goto(`/books/${BOOK}/read`); + await expect.poll(() => darkPixels(page, 1)).toBeGreaterThan(50); + await page.getByRole('button', { name: '目录', exact: true }).click(); + await page.getByRole('button', { name: '第 2 页', exact: true }).click(); + await expect.poll(() => darkPixels(page, 2)).toBeGreaterThan(50); + const toolbar = await page + .getByRole('toolbar', { name: '阅读工具栏' }) + .boundingBox(); + expect(toolbar!.y).toBeGreaterThanOrEqual(0); + const content = await page.locator('.pdf-viewer-container').boundingBox(); + const handle = await page + .getByRole('button', { name: '调整教材区域宽度', exact: true }) + .boundingBox(); + expect(content!.x + content!.width).toBeLessThanOrEqual(handle!.x); +}); + +test.describe('opt-in local scan decoder', () => { + test('JBIG2 pages render with packaged decoders under the production CSP', async ({ + page, + }) => { + test.skip( + !process.env.TEXTBOOKLENS_SCAN_PDF, + 'Requires a user-designated local scan PDF', + ); + const external: string[] = []; + page.on('request', (request) => { + if ( + /^https?:/.test(request.url()) && + !request.url().startsWith('http://127.0.0.1:1420/') + ) + external.push(request.url()); + }); + await installBackend( + page, + [...readFileSync(process.env.TEXTBOOKLENS_SCAN_PDF!)], + [1, 10], + ); + const csp = JSON.parse(readFileSync('src-tauri/tauri.conf.json', 'utf8')) + .app.security.csp as string; + await page.route(`**/books/${BOOK}/read`, async (route) => { + const response = await route.fetch(); + await route.fulfill({ + response, + headers: { ...response.headers(), 'content-security-policy': csp }, + }); + }); + const decoder = page.waitForResponse( + (response) => + response.url().endsWith('/pdfjs/wasm/jbig2.wasm') && response.ok(), + ); + await page.goto(`/books/${BOOK}/read`); + await decoder; + await expect + .poll(() => darkPixels(page, 1), { timeout: 30000 }) + .toBeGreaterThan(100); + await page.getByRole('button', { name: '目录', exact: true }).click(); + await page.getByRole('button', { name: '第 10 页', exact: true }).click(); + await expect + .poll(() => darkPixels(page, 10), { timeout: 30000 }) + .toBeGreaterThan(100); + expect(external).toEqual([]); + }); +}); diff --git a/eslint.config.js b/eslint.config.js index 69a9467..66ecc3f 100644 --- a/eslint.config.js +++ b/eslint.config.js @@ -12,6 +12,7 @@ export default tseslint.config( 'playwright-report', 'src-tauri/target', 'test-results', + 'public/pdfjs', ], }, js.configs.recommended, diff --git a/package-lock.json b/package-lock.json index 71215fe..609f8bd 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "textbooklens", - "version": "0.1.0", + "version": "0.2.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "textbooklens", - "version": "0.1.0", + "version": "0.2.0", "license": "Apache-2.0", "dependencies": { "@tauri-apps/api": "2.11.1", @@ -2954,9 +2954,9 @@ } }, "node_modules/@xmldom/xmldom": { - "version": "0.8.13", - "resolved": "https://registry.npmjs.org/@xmldom/xmldom/-/xmldom-0.8.13.tgz", - "integrity": "sha512-KRYzxepc14G/CEpEGc3Yn+JKaAeT63smlDr+vjB8jRfgTBBI9wRj/nkQEO+ucV8p8I9bfKLWp37uHgFrbntPvw==", + "version": "0.8.15", + "resolved": "https://registry.npmjs.org/@xmldom/xmldom/-/xmldom-0.8.15.tgz", + "integrity": "sha512-/5NV/vDALVFDXgLmfsy9TRCBlKwO2LNBFzpzvb9iIj+jR+eSc6DLYYvVOdivT/jm7MtU6TebYuRmzEOI7w40UA==", "license": "MIT", "engines": { "node": ">=10.0.0" diff --git a/package.json b/package.json index 490f2eb..d9be810 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "textbooklens", "private": true, - "version": "0.1.0", + "version": "0.2.0", "license": "Apache-2.0", "type": "module", "engines": { @@ -9,6 +9,8 @@ "npm": "=11.16.0" }, "scripts": { + "predev": "node scripts/sync-pdf-resources.mjs", + "prebuild": "node scripts/sync-pdf-resources.mjs", "dev": "vite", "preview": "vite preview", "build": "tsc -b && vite build", @@ -49,7 +51,7 @@ }, "overrides": { "epubjs@0.3.93": { - "@xmldom/xmldom": "0.8.13" + "@xmldom/xmldom": "0.8.15" } }, "devDependencies": { diff --git a/scripts/audit-release-artifacts.mjs b/scripts/audit-release-artifacts.mjs index be1c10d..dc61a8b 100644 --- a/scripts/audit-release-artifacts.mjs +++ b/scripts/audit-release-artifacts.mjs @@ -13,6 +13,8 @@ const projectRoot = path.resolve(path.dirname(scriptPath), '..'); export const DEFAULT_AUDIT_LIMITS = Object.freeze({ ...DEFAULT_SCAN_LIMITS, + // Installers may contain the official offline WebView2 runtime. + maxFileBytes: 512 * 1024 * 1024, maxFiles: 12_000, maxDepth: 24, maxTotalBytes: 2 * 1024 * 1024 * 1024, diff --git a/scripts/check-production-dependencies.mjs b/scripts/check-production-dependencies.mjs index d6cf4ef..05cdcf1 100644 --- a/scripts/check-production-dependencies.mjs +++ b/scripts/check-production-dependencies.mjs @@ -8,6 +8,7 @@ const moduleFile = import.meta.url.startsWith('file:') const projectRoot = path.resolve(path.dirname(moduleFile), '..'); export const XMLDOM_ADVISORIES = Object.freeze([ + 'GHSA-6gmq-8vp8-gcm6', 'GHSA-wh4c-j3r5-mjhp', 'GHSA-j759-j44w-7fr8', 'GHSA-x6wf-f3px-wcqx', @@ -232,8 +233,8 @@ export function isAffectedXmldomVersion(version) { const patch = Number(match[3]); if (major > 0) return false; if (minor < 8) return true; - if (minor === 8) return patch < 13; - if (minor === 9) return patch < 10; + if (minor === 8) return patch < 15; + if (minor === 9) return patch < 12; return false; } diff --git a/scripts/check-production-dependencies.test.mjs b/scripts/check-production-dependencies.test.mjs index 5bba44d..71ea7f6 100644 --- a/scripts/check-production-dependencies.test.mjs +++ b/scripts/check-production-dependencies.test.mjs @@ -14,7 +14,7 @@ function fixture() { manifest: { dependencies: { dompurify: '3.4.13', epubjs: '0.3.93' }, overrides: { - 'epubjs@0.3.93': { '@xmldom/xmldom': '0.8.13' }, + 'epubjs@0.3.93': { '@xmldom/xmldom': '0.8.15' }, }, }, lockfile: { @@ -28,7 +28,7 @@ function fixture() { version: '0.3.93', dependencies: { '@xmldom/xmldom': '^0.7.5' }, }, - 'node_modules/@xmldom/xmldom': { version: '0.8.13' }, + 'node_modules/@xmldom/xmldom': { version: '0.8.15' }, 'node_modules/postcss': { version: '8.5.26', dev: true, @@ -56,7 +56,7 @@ describe('supply-chain dependency security policy', () => { production: [], development: ['3.3.17', '5.1.16'], }); - expect(productionXmldomVersions(lockfile)).toEqual(['0.8.13']); + expect(productionXmldomVersions(lockfile)).toEqual(['0.8.15']); }); it('rejects affected DOMPurify releases and requires an exact direct patch', () => { @@ -111,16 +111,18 @@ describe('supply-chain dependency security policy', () => { it('rejects every reviewed vulnerable @xmldom/xmldom release line', () => { expect(isAffectedXmldomVersion('0.7.13')).toBe(true); expect(isAffectedXmldomVersion('0.8.12')).toBe(true); - expect(isAffectedXmldomVersion('0.8.13')).toBe(false); + expect(isAffectedXmldomVersion('0.8.14')).toBe(true); + expect(isAffectedXmldomVersion('0.9.11')).toBe(true); + expect(isAffectedXmldomVersion('0.8.15')).toBe(false); expect(isAffectedXmldomVersion('0.9.9')).toBe(true); - expect(isAffectedXmldomVersion('0.9.10')).toBe(false); + expect(isAffectedXmldomVersion('0.9.12')).toBe(false); const { manifest, lockfile } = fixture(); lockfile.packages['node_modules/@xmldom/xmldom'].version = '0.7.13'; expect(auditDependencyPolicy(manifest, lockfile)).toEqual( expect.arrayContaining([ expect.stringContaining('@xmldom/xmldom 0.7.13 is affected'), - expect.stringContaining('does not contain the epubjs override 0.8.13'), + expect.stringContaining('does not contain the epubjs override 0.8.15'), ]), ); }); @@ -141,7 +143,7 @@ describe('supply-chain dependency security policy', () => { it('requires a scoped exact safe override when the epubjs range is unsafe', () => { const { manifest, lockfile } = fixture(); manifest.overrides = { - epubjs: { '@xmldom/xmldom': '^0.8.13' }, + epubjs: { '@xmldom/xmldom': '^0.8.15' }, }; expect(auditDependencyPolicy(manifest, lockfile)).toEqual( diff --git a/scripts/sync-pdf-resources.mjs b/scripts/sync-pdf-resources.mjs new file mode 100644 index 0000000..58c8fd4 --- /dev/null +++ b/scripts/sync-pdf-resources.mjs @@ -0,0 +1,20 @@ +import { cp, mkdir, readdir } from 'node:fs/promises'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; + +const root = fileURLToPath(new URL('..', import.meta.url)); +const source = path.join(root, 'node_modules', 'pdfjs-dist'); +const target = path.join(root, 'public', 'pdfjs'); +await mkdir(target, { recursive: true }); +for (const directory of ['cmaps', 'standard_fonts']) { + await cp(path.join(source, directory), path.join(target, directory), { + recursive: true, + }); +} +await mkdir(path.join(target, 'wasm'), { recursive: true }); +for (const file of await readdir(path.join(source, 'wasm'))) { + if (file.startsWith('quickjs')) continue; + await cp(path.join(source, 'wasm', file), path.join(target, 'wasm', file)); +} +await cp(path.join(source, 'LICENSE'), path.join(target, 'LICENSE')); +console.log('Local PDF decoder, CMap, font and license resources are ready.'); diff --git a/src-tauri/Cargo.lock b/src-tauri/Cargo.lock index 258a249..58fe616 100644 --- a/src-tauri/Cargo.lock +++ b/src-tauri/Cargo.lock @@ -4873,7 +4873,7 @@ dependencies = [ [[package]] name = "textbooklens" -version = "0.1.0" +version = "0.2.0" dependencies = [ "async-trait", "chrono", @@ -5021,6 +5021,7 @@ dependencies = [ "libc", "mio", "pin-project-lite", + "signal-hook-registry", "socket2", "tokio-macros", "windows-sys 0.61.2", diff --git a/src-tauri/Cargo.toml b/src-tauri/Cargo.toml index 1678ee0..a11373f 100644 --- a/src-tauri/Cargo.toml +++ b/src-tauri/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "textbooklens" -version = "0.1.0" +version = "0.2.0" description = "Local-first AI textbook reader" authors = ["TextbookLens contributors"] license = "Apache-2.0" @@ -30,7 +30,7 @@ sqlx = { version = "=0.9.0", default-features = false, features = ["runtime-toki tauri = { version = "=2.11.5", features = [] } tauri-plugin-dialog = "=2.7.2" thiserror = "=2.0.19" -tokio = { version = "=1.53.1", features = ["macros", "rt-multi-thread", "sync", "time"] } +tokio = { version = "=1.53.1", features = ["macros", "rt-multi-thread", "sync", "time", "process", "io-util"] } tokio-util = "=0.7.19" tracing = "=0.1.44" tracing-appender = "=0.2.5" diff --git a/src-tauri/migrations/0016_local_model_profiles.sql b/src-tauri/migrations/0016_local_model_profiles.sql new file mode 100644 index 0000000..de39147 --- /dev/null +++ b/src-tauri/migrations/0016_local_model_profiles.sql @@ -0,0 +1,42 @@ +-- no-transaction +PRAGMA foreign_keys = OFF; +BEGIN IMMEDIATE; + +CREATE TABLE provider_profiles_v16 ( + id TEXT PRIMARY KEY NOT NULL, + provider_kind TEXT NOT NULL CHECK (provider_kind IN ('openai', 'gemini', 'anthropic', 'deepseek', 'kimi', 'ollama', 'lm_studio')), + display_name TEXT NOT NULL, + model_id TEXT NOT NULL, + context_window_tokens INTEGER NOT NULL CHECK (context_window_tokens > 0), + is_active INTEGER NOT NULL DEFAULT 0 CHECK (is_active IN (0, 1)), + created_at TEXT NOT NULL, + updated_at TEXT NOT NULL, + validated_at TEXT, + kimi_api_region TEXT CHECK (kimi_api_region IS NULL OR kimi_api_region IN ('cn', 'international')), + local_port INTEGER CHECK (local_port BETWEEN 1 AND 65535), + CHECK ((provider_kind IN ('ollama', 'lm_studio')) = (local_port IS NOT NULL)) +); + +INSERT INTO provider_profiles_v16 ( + id, provider_kind, display_name, model_id, context_window_tokens, + is_active, created_at, updated_at, validated_at, kimi_api_region +) +SELECT id, provider_kind, display_name, model_id, context_window_tokens, + is_active, created_at, updated_at, validated_at, kimi_api_region +FROM provider_profiles; + +DROP TABLE provider_profiles; +ALTER TABLE provider_profiles_v16 RENAME TO provider_profiles; +CREATE UNIQUE INDEX provider_profiles_at_most_one_active +ON provider_profiles (is_active) WHERE is_active = 1; +CREATE UNIQUE INDEX provider_profiles_local_identity +ON provider_profiles (provider_kind, local_port, model_id) WHERE local_port IS NOT NULL; + +CREATE TABLE local_profiles_foreign_key_guard ( + invalid_count INTEGER NOT NULL CHECK (invalid_count = 0) +); +INSERT INTO local_profiles_foreign_key_guard SELECT COUNT(*) FROM pragma_foreign_key_check; +DROP TABLE local_profiles_foreign_key_guard; + +COMMIT; +PRAGMA foreign_keys = ON; diff --git a/src-tauri/migrations/0017_local_vision.sql b/src-tauri/migrations/0017_local_vision.sql new file mode 100644 index 0000000..eeb5089 --- /dev/null +++ b/src-tauri/migrations/0017_local_vision.sql @@ -0,0 +1,2 @@ +ALTER TABLE provider_profiles ADD COLUMN local_vision INTEGER NOT NULL DEFAULT 0 +CHECK (local_vision IN (0, 1) AND (local_vision = 0 OR provider_kind = 'ollama')); diff --git a/src-tauri/src/ai/local/discovery.rs b/src-tauri/src/ai/local/discovery.rs new file mode 100644 index 0000000..3075488 --- /dev/null +++ b/src-tauri/src/ai/local/discovery.rs @@ -0,0 +1,294 @@ +use std::{ + path::{Path, PathBuf}, + process::Stdio, + time::Duration, +}; + +use serde_json::Value; +use sha2::{Digest, Sha256}; +use tokio::process::Command; +use tokio_util::sync::CancellationToken; + +use super::http::LocalHttp; +use crate::{ai::error::AiError, domain::ProviderKind}; + +const CLI_TIMEOUT: Duration = Duration::from_secs(30); + +#[derive(Clone)] +pub(super) struct Installation { + pub kind: ProviderKind, + pub port: u16, + pub cli: Option, +} + +impl Installation { + pub fn detect(kind: ProviderKind, saved_port: Option) -> Self { + let mut candidates = Vec::new(); + let name = if kind == ProviderKind::Ollama { + "ollama.exe" + } else { + "lms.exe" + }; + if let Some(local) = std::env::var_os("LOCALAPPDATA") + && kind == ProviderKind::Ollama + { + candidates.push(PathBuf::from(local).join("Programs/Ollama/ollama.exe")); + } + if kind == ProviderKind::LmStudio { + for home in lmstudio_homes() { + candidates.push(home.join("bin/lms.exe")); + } + } + if let Some(paths) = std::env::var_os("PATH") { + candidates.extend( + std::env::split_paths(&paths) + .filter(|path| path.is_absolute()) + .map(|path| path.join(name)), + ); + } + let cli = candidates.into_iter().find(|path| path.is_file()); + let port = saved_port.unwrap_or_else(|| { + if kind == ProviderKind::Ollama { + std::env::var("OLLAMA_HOST") + .ok() + .and_then(|host| loopback_port(&host)) + .unwrap_or(11434) + } else { + lmstudio_homes() + .iter() + .find_map(|home| { + read_small_json(&home.join(".internal/http-server-config.json")) + .and_then(|value| value["port"].as_u64()) + .and_then(|port| u16::try_from(port).ok()) + .filter(|port| *port > 0) + }) + .unwrap_or(1234) + } + }); + Self { kind, port, cli } + } + + pub async fn ensure_running(&self, cancel: &CancellationToken) -> Result<(), AiError> { + let http = LocalHttp::new(self.port)?; + let path = if self.kind == ProviderKind::Ollama { + "/api/tags" + } else { + "/lmstudio-greeting" + }; + if http + .json(path, None, cancel, Duration::from_secs(2)) + .await + .is_ok() + { + return Ok(()); + } + let cli = self + .cli + .as_ref() + .ok_or_else(AiError::provider_unavailable)?; + if self.kind == ProviderKind::Ollama { + let mut command = hidden_command(cli); + command + .arg("serve") + .env("OLLAMA_HOST", format!("127.0.0.1:{}", self.port)) + .env("OLLAMA_NO_CLOUD", "1") + .stdin(Stdio::null()) + .stdout(Stdio::null()) + .stderr(Stdio::null()); + // This is a reusable local server, intentionally allowed to outlive this request. + let mut child = command + .spawn() + .map_err(|_| AiError::provider_unavailable())?; + tokio::spawn(async move { + let _ = child.wait().await; + }); + } else { + run_cli( + cli, + &[ + "server", + "start", + "--port", + &self.port.to_string(), + "--bind", + "127.0.0.1", + ], + cancel, + CLI_TIMEOUT, + ) + .await?; + } + for _ in 0..30 { + if http + .json(path, None, cancel, Duration::from_secs(1)) + .await + .is_ok() + { + return Ok(()); + } + tokio::select! { + _ = cancel.cancelled() => return Err(AiError::cancelled()), + _ = tokio::time::sleep(Duration::from_millis(500)) => {}, + } + } + Err(AiError::provider_unavailable()) + } + + pub async fn lmstudio_models(&self, cancel: &CancellationToken) -> Result { + let cli = self + .cli + .as_ref() + .ok_or_else(AiError::provider_unavailable)?; + cli_json(cli, &["ls", "--llm", "--json"], cancel).await + } + + pub async fn load_lmstudio( + &self, + key: &str, + context: u32, + cancel: &CancellationToken, + ) -> Result { + let cli = self + .cli + .as_ref() + .ok_or_else(AiError::provider_unavailable)?; + let models = self.lmstudio_models(cancel).await?; + let model = models + .as_array() + .and_then(|models| { + models + .iter() + .find(|model| local_lm_model(model) && model_key(model) == Some(key)) + }) + .ok_or_else(AiError::model_not_found)?; + let supports_local = model.get("deviceIdentifier").is_some(); + let loaded = cli_json(cli, &["ps", "--json"], cancel).await?; + if let Some(model) = loaded.as_array().and_then(|models| { + models.iter().find(|model| { + local_lm_model(model) + && model_key(model) == Some(key) + && model["contextLength"] + .as_u64() + .is_some_and(|tokens| tokens >= u64::from(context)) + }) + }) && let Some(id) = model["identifier"].as_str() + { + return Ok(id.to_owned()); + } + let digest: String = Sha256::digest(key.as_bytes()) + .iter() + .map(|byte| format!("{byte:02x}")) + .collect(); + let identifier = format!("textbooklens-{digest}"); + let context = context.to_string(); + let mut args = vec![ + "load", + key, + "--identifier", + &identifier, + "--context-length", + &context, + "--yes", + ]; + // Modern LM Link-aware CLIs provide --local. Older CLIs only know local models. + if supports_local { + args.push("--local"); + } + run_cli(cli, &args, cancel, super::http::INFERENCE_TIMEOUT).await?; + let loaded = cli_json(cli, &["ps", "--json"], cancel).await?; + if !loaded.as_array().is_some_and(|models| { + models.iter().any(|model| { + local_lm_model(model) + && model["identifier"] == identifier + && model_key(model) == Some(key) + }) + }) { + return Err(AiError::provider_unavailable()); + } + Ok(identifier) + } +} + +pub(super) fn model_key(value: &Value) -> Option<&str> { + value["modelKey"] + .as_str() + .or_else(|| value["path"].as_str()) +} + +pub(super) fn local_lm_model(value: &Value) -> bool { + value["type"] == "llm" && value.get("deviceIdentifier").is_none_or(Value::is_null) +} + +fn lmstudio_homes() -> Vec { + let mut homes = Vec::new(); + if let Some(home) = std::env::var_os("LMSTUDIO_HOME") { + homes.push(PathBuf::from(home)); + } + if let Some(home) = std::env::var_os("USERPROFILE") { + let home = PathBuf::from(home); + homes.push(home.join(".lmstudio")); + homes.push(home.join(".cache/lm-studio")); + } + homes +} + +fn read_small_json(path: &Path) -> Option { + if std::fs::metadata(path).ok()?.len() > 64 * 1024 { + return None; + } + serde_json::from_slice(&std::fs::read(path).ok()?).ok() +} + +fn loopback_port(host: &str) -> Option { + let host = if host.contains("://") { + host.to_owned() + } else { + format!("http://{host}") + }; + let url = reqwest::Url::parse(&host).ok()?; + if !matches!(url.host_str(), Some("127.0.0.1" | "localhost" | "0.0.0.0")) { + return None; + } + url.port().filter(|port| *port > 0) +} + +fn hidden_command(path: &Path) -> Command { + let mut command = Command::new(path); + #[cfg(windows)] + command.creation_flags(0x08000000); // CREATE_NO_WINDOW + command.stdin(Stdio::null()); + command +} + +async fn cli_json( + path: &Path, + args: &[&str], + cancel: &CancellationToken, +) -> Result { + let output = run_cli(path, args, cancel, CLI_TIMEOUT).await?; + serde_json::from_slice(&output).map_err(|_| AiError::provider_unavailable()) +} + +async fn run_cli( + path: &Path, + args: &[&str], + cancel: &CancellationToken, + timeout: Duration, +) -> Result, AiError> { + let mut command = hidden_command(path); + command + .args(args) + .kill_on_drop(true) + .stdout(Stdio::piped()) + .stderr(Stdio::null()); + // No shell, installation commands, model downloads, or remote-host arguments. + let output = tokio::select! { + biased; + _ = cancel.cancelled() => return Err(AiError::cancelled()), + result = tokio::time::timeout(timeout, command.output()) => result.map_err(|_| AiError::provider_unavailable())?.map_err(|_| AiError::provider_unavailable())?, + }; + if !output.status.success() || output.stdout.len() > 4 * 1024 * 1024 { + return Err(AiError::provider_unavailable()); + } + Ok(output.stdout) +} diff --git a/src-tauri/src/ai/local/http.rs b/src-tauri/src/ai/local/http.rs new file mode 100644 index 0000000..2fc1cc0 --- /dev/null +++ b/src-tauri/src/ai/local/http.rs @@ -0,0 +1,218 @@ +use std::{collections::VecDeque, time::Duration}; + +use futures_util::{StreamExt, stream}; +use reqwest::{Client, Method, Response}; +use serde_json::Value; +use tokio_util::sync::CancellationToken; +use zeroize::Zeroizing; + +use crate::{ + ai::{ + error::AiError, + provider::ProviderStream, + stream::{SseEvent, SseEventMapper, decode_sse}, + }, + domain::UnifiedStreamEvent, +}; + +pub(super) const INFERENCE_TIMEOUT: Duration = Duration::from_secs(300); +const MAX_BODY: usize = 4 * 1024 * 1024; + +/// Only numeric IPv4 loopback addresses are constructible. Never uses proxies, +/// redirects, remote credentials, retries, or a caller-provided origin. +#[derive(Clone)] +pub(super) struct LocalHttp { + client: Client, + port: u16, +} + +impl LocalHttp { + pub fn new(port: u16) -> Result { + if port == 0 { + return Err(AiError::invalid_input()); + } + Ok(Self { + client: Client::builder() + .no_proxy() + .redirect(reqwest::redirect::Policy::none()) + .retry(reqwest::retry::never()) + .connect_timeout(Duration::from_secs(1)) + .build() + .map_err(|_| AiError::provider_unavailable())?, + port, + }) + } + + pub async fn request( + &self, + path: &'static str, + body: Option, + cancel: &CancellationToken, + timeout: Duration, + ) -> Result { + let mut request = self.client.request( + if body.is_some() { + Method::POST + } else { + Method::GET + }, + format!("http://127.0.0.1:{}{path}", self.port), + ); + if let Some(body) = body { + request = request.json(&body); + } + let response = tokio::select! { + biased; + _ = cancel.cancelled() => return Err(AiError::cancelled()), + response = tokio::time::timeout(timeout, request.send()) => response.map_err(|_| AiError::provider_unavailable())?.map_err(|e| AiError::from_reqwest(&e))?, + }; + match response.status().as_u16() { + 200..=299 => Ok(response), + 401 | 403 => Err(AiError::invalid_api_key()), + 404 => Err(AiError::model_not_found()), + _ => Err(AiError::provider_unavailable()), + } + } + + pub async fn json( + &self, + path: &'static str, + body: Option, + cancel: &CancellationToken, + timeout: Duration, + ) -> Result { + let response = self.request(path, body, cancel, timeout).await?; + let read = async { + let mut bytes = Zeroizing::new(Vec::new()); + let mut source = response.bytes_stream(); + while let Some(chunk) = source.next().await { + let chunk = chunk.map_err(|_| AiError::provider_unavailable())?; + if bytes.len().saturating_add(chunk.len()) > MAX_BODY { + return Err(AiError::provider_unavailable()); + } + bytes.extend_from_slice(&chunk); + } + serde_json::from_slice(&bytes).map_err(|_| AiError::provider_unavailable()) + }; + tokio::select! { + biased; + _ = cancel.cancelled() => Err(AiError::cancelled()), + result = tokio::time::timeout(timeout, read) => result.map_err(|_| AiError::provider_unavailable())?, + } + } +} + +pub(super) fn lmstudio_stream(response: Response, cancel: CancellationToken) -> ProviderStream { + decode_sse( + response.bytes_stream(), + ChatMapper, + cancel, + INFERENCE_TIMEOUT, + ) +} + +struct ChatMapper; +impl SseEventMapper for ChatMapper { + fn map_event(&mut self, event: &SseEvent) -> Result, AiError> { + if event.data() == "[DONE]" { + return Ok(Vec::new()); + } + let value: Value = + serde_json::from_str(event.data()).map_err(|_| AiError::malformed_event())?; + if value.get("error").is_some() { + return Err(AiError::provider_unavailable()); + } + let choices = value["choices"] + .as_array() + .ok_or_else(AiError::malformed_event)?; + let mut events = Vec::new(); + for choice in choices { + if choice["index"].as_u64().unwrap_or(0) != 0 { + continue; + } + if let Some(text) = choice["delta"]["content"] + .as_str() + .filter(|v| !v.is_empty()) + { + events.push(UnifiedStreamEvent::TextDelta { + text: text.to_owned(), + }); + } + match choice["finish_reason"].as_str() { + Some("stop") => events.push(UnifiedStreamEvent::Completed), + Some("length") => return Err(AiError::context_too_large()), + Some(_) => return Err(AiError::refused()), + None => {} + } + } + Ok(events) + } +} + +pub(super) fn ollama_stream(response: Response, cancel: CancellationToken) -> ProviderStream { + let source = Box::pin(response.bytes_stream()); + let initial = ( + source, + Zeroizing::new(Vec::::new()), + VecDeque::new(), + false, + cancel, + ); + Box::pin(stream::unfold( + initial, + |(mut source, mut buffer, mut pending, mut terminal, cancel)| async move { + loop { + if cancel.is_cancelled() && (!terminal || !pending.is_empty()) { + pending.clear(); + return Some(( + Err(AiError::cancelled()), + (source, buffer, pending, true, cancel), + )); + } + if let Some(event) = pending.pop_front() { + return Some((Ok(event), (source, buffer, pending, terminal, cancel))); + } + if terminal { + return None; + } + let outcome: Result<(), AiError> = async { + if cancel.is_cancelled() { return Err(AiError::cancelled()); } + if let Some(end) = buffer.iter().position(|byte| *byte == b'\n') { + let line: Vec = buffer.drain(..=end).collect(); + if line.iter().all(u8::is_ascii_whitespace) { return Ok(()); } + let value: Value = serde_json::from_slice(&line).map_err(|_| AiError::malformed_event())?; + if value.get("error").is_some() { return Err(AiError::provider_unavailable()); } + if let Some(text) = value["message"]["content"].as_str().filter(|v| !v.is_empty()) { + pending.push_back(UnifiedStreamEvent::TextDelta { text: text.to_owned() }); + } + if value["done"] == true { + if value["done_reason"] == "length" { return Err(AiError::context_too_large()); } + pending.push_back(UnifiedStreamEvent::Completed); + terminal = true; + } + return Ok(()); + } + let chunk = tokio::select! { + biased; + _ = cancel.cancelled() => return Err(AiError::cancelled()), + chunk = tokio::time::timeout(INFERENCE_TIMEOUT, source.next()) => chunk.map_err(|_| AiError::provider_unavailable())?, + }; + match chunk { + Some(Ok(chunk)) => { + if buffer.len().saturating_add(chunk.len()) > MAX_BODY { return Err(AiError::malformed_event()); } + buffer.extend_from_slice(&chunk); + }, + Some(Err(_)) => return Err(AiError::provider_unavailable()), + None if !buffer.is_empty() => buffer.push(b'\n'), + None => return Err(AiError::unexpected_eof()), + } + Ok(()) + }.await; + if let Err(error) = outcome { + pending.clear(); + return Some((Err(error), (source, buffer, pending, true, cancel))); + } + } + }, + )) +} diff --git a/src-tauri/src/ai/local/mod.rs b/src-tauri/src/ai/local/mod.rs new file mode 100644 index 0000000..8342ea2 --- /dev/null +++ b/src-tauri/src/ai/local/mod.rs @@ -0,0 +1,521 @@ +mod discovery; +mod http; + +use std::time::Duration; + +use async_trait::async_trait; +use secrecy::SecretString; +use serde_json::{Value, json}; +use sqlx::SqlitePool; +use tokio_util::sync::CancellationToken; + +use crate::{ + ai::{ + error::{AiError, AiErrorKind}, + provider::{AiProvider, ProviderStream, validate_model_id}, + }, + db::providers, + domain::{ + LocalModelConnectResult, LocalServiceReport, LocalServiceStatus, ProviderKind, + UnifiedChatRequest, UnifiedRole, ValidationResult, + }, + errors::AppResult, +}; +use discovery::{Installation, local_lm_model, model_key}; +use http::{INFERENCE_TIMEOUT, LocalHttp}; + +const METADATA_TIMEOUT: Duration = Duration::from_secs(5); +const DEFAULT_CONTEXT: u32 = 8_192; +const VISION_CONTEXT: u32 = 16_384; +pub(crate) const LOCAL_IMAGE_LIMITS: crate::domain::ImageLimits = crate::domain::ImageLimits { + max_images: 1, + max_encoded_bytes_each: 2 * 1024 * 1024, + max_total_encoded_bytes: 2 * 1024 * 1024, + max_dimension_px: 2048, + max_decoded_pixels_each: 1_048_576, +}; + +#[derive(Clone)] +pub(crate) struct LocalModel { + pub id: String, + pub context: u32, + pub vision: bool, + size: u64, + loaded: bool, +} + +pub(crate) struct LocalProvider { + installation: Installation, + http: LocalHttp, + context: u32, +} + +impl LocalProvider { + pub fn new(kind: ProviderKind, port: u16, context: u32) -> Result { + if !kind.is_local() || context == 0 { + return Err(AiError::invalid_input()); + } + Ok(Self { + installation: Installation::detect(kind, Some(port)), + http: LocalHttp::new(port)?, + context, + }) + } + + async fn models(&self, cancel: &CancellationToken) -> Result, AiError> { + self.installation.ensure_running(cancel).await?; + let mut models = if self.installation.kind == ProviderKind::Ollama { + let list = self + .http + .json("/api/tags", None, cancel, METADATA_TIMEOUT) + .await?; + let entries = list["models"] + .as_array() + .ok_or_else(AiError::malformed_event)?; + let loaded = self + .http + .json("/api/ps", None, cancel, METADATA_TIMEOUT) + .await + .unwrap_or(Value::Null); + let mut models = Vec::new(); + for entry in entries.iter().take(256) { + let Some(id) = entry["name"].as_str().filter(|id| valid_local_id(id)) else { + continue; + }; + let info = match self + .http + .json( + "/api/show", + Some(json!({"model": id})), + cancel, + METADATA_TIMEOUT, + ) + .await + { + Ok(info) => info, + Err(error) if error.kind() == AiErrorKind::Cancelled => return Err(error), + Err(_) => continue, + }; + if !local_ollama_info(&info) { + continue; + } + let vision = ollama_has_vision(&info); + let context = info["model_info"] + .as_object() + .and_then(|fields| { + fields + .iter() + .find(|(key, _)| key.ends_with(".context_length")) + }) + .and_then(|(_, value)| value.as_u64()) + .and_then(|value| u32::try_from(value).ok()) + .unwrap_or(DEFAULT_CONTEXT) + .min(if vision { + VISION_CONTEXT + } else { + DEFAULT_CONTEXT + }); + if context < 2048 { + continue; + } + models.push(LocalModel { + id: id.to_owned(), + context, + vision, + size: entry["size"].as_u64().unwrap_or(u64::MAX), + loaded: loaded["models"].as_array().is_some_and(|models| { + models + .iter() + .any(|model| model["name"] == id || model["model"] == id) + }), + }); + } + models + } else { + let list = self.installation.lmstudio_models(cancel).await?; + parse_lmstudio_models(&list)? + }; + models.sort_by(|a, b| { + b.loaded + .cmp(&a.loaded) + .then(a.size.cmp(&b.size)) + .then(a.id.cmp(&b.id)) + }); + models.dedup_by(|a, b| a.id == b.id); + Ok(models) + } + + async fn prepare_model(&self, id: &str, cancel: &CancellationToken) -> Result { + self.prepare_model_for_input(id, cancel, false).await + } + + async fn prepare_model_for_input( + &self, + id: &str, + cancel: &CancellationToken, + vision: bool, + ) -> Result { + validate_model_id(id)?; + if !valid_local_id(id) { + return Err(AiError::model_not_found()); + } + self.installation.ensure_running(cancel).await?; + if self.installation.kind == ProviderKind::Ollama { + // Recheck the installed model at each use. localhost alone does not imply local inference. + let info = self + .http + .json( + "/api/show", + Some(json!({"model": id})), + cancel, + METADATA_TIMEOUT, + ) + .await?; + if !local_ollama_info(&info) { + return Err(AiError::model_not_found()); + } + if vision && !ollama_has_vision(&info) { + return Err(AiError::model_not_found()); + } + Ok(id.to_owned()) + } else { + self.installation + .load_lmstudio(id, self.context, cancel) + .await + } + } + + fn body(&self, id: &str, request: &UnifiedChatRequest, stream: bool) -> Value { + let mut messages = vec![json!({"role": "system", "content": request.system})]; + messages.extend(request.messages.iter().map(|message| { + json!({ + "role": if message.role == UnifiedRole::User { "user" } else { "assistant" }, + "content": message.content, + }) + })); + if self.installation.kind == ProviderKind::Ollama { + json!({"model": id, "messages": messages, "stream": stream, "think": false, + "options": {"num_ctx": self.context, "num_predict": request.max_output_tokens}}) + } else { + json!({"model": id, "messages": messages, "stream": stream, "max_tokens": request.max_output_tokens}) + } + } + + fn chat_path(&self) -> &'static str { + if self.installation.kind == ProviderKind::Ollama { + "/api/chat" + } else { + "/v1/chat/completions" + } + } + + async fn smoke_test(&self, id: &str, cancel: &CancellationToken) -> Result<(), AiError> { + let wire_id = self.prepare_model(id, cancel).await?; + let request = UnifiedChatRequest { + model: id.to_owned(), + system: "Reply briefly.".to_owned(), + messages: vec![crate::domain::UnifiedMessage { + role: UnifiedRole::User, + content: "Reply with OK.".to_owned(), + }], + max_output_tokens: super::registry::local_output_tokens(self.context), + expected_language: None, + }; + let reply = self + .http + .json( + self.chat_path(), + Some(self.body(&wire_id, &request, false)), + cancel, + INFERENCE_TIMEOUT, + ) + .await?; + let message = if self.installation.kind == ProviderKind::Ollama { + if reply["done"] != true || reply["done_reason"] == "length" { + return Err(AiError::provider_unavailable()); + } + &reply["message"] + } else { + if reply["choices"][0]["finish_reason"] != "stop" { + return Err(AiError::provider_unavailable()); + } + &reply["choices"][0]["message"] + }; + if !message["content"] + .as_str() + .is_some_and(|text| !text.trim().is_empty()) + { + return Err(AiError::provider_unavailable()); + } + Ok(()) + } +} + +#[async_trait] +impl AiProvider for LocalProvider { + fn kind(&self) -> ProviderKind { + self.installation.kind.clone() + } + async fn validate( + &self, + _credential: &SecretString, + model: &str, + ) -> Result { + let cancel = CancellationToken::new(); + self.prepare_model(model, &cancel).await?; + Ok(ValidationResult { + model: model.to_owned(), + context_window_tokens: self.context, + }) + } + async fn stream_chat( + &self, + _credential: &SecretString, + request: UnifiedChatRequest, + cancel: CancellationToken, + ) -> Result { + if request.messages.is_empty() || request.max_output_tokens == 0 { + return Err(AiError::invalid_input()); + } + let wire_id = self.prepare_model(&request.model, &cancel).await?; + let response = self + .http + .request( + self.chat_path(), + Some(self.body(&wire_id, &request, true)), + &cancel, + INFERENCE_TIMEOUT, + ) + .await?; + Ok(if self.installation.kind == ProviderKind::Ollama { + http::ollama_stream(response, cancel) + } else { + http::lmstudio_stream(response, cancel) + }) + } + + async fn stream_vision( + &self, + _credential: &SecretString, + request: crate::domain::UnifiedVisionRequest, + cancel: CancellationToken, + ) -> AppResult { + if self.installation.kind != ProviderKind::Ollama { + return Err(crate::errors::AppError::unsupported_provider_capability()); + } + let book_id = request + .images + .first() + .map(|image| image.meta.book_id) + .ok_or_else(|| AiError::invalid_input().into_app_error())?; + crate::ai::multimodal::validate_vision_request(book_id, &request, LOCAL_IMAGE_LIMITS)?; + if request + .text + .messages + .last() + .is_none_or(|message| message.role != UnifiedRole::User) + || request.text.max_output_tokens == 0 + { + return Err(AiError::invalid_input().into_app_error()); + } + let wire_id = self + .prepare_model_for_input(&request.text.model, &cancel, true) + .await + .map_err(AiError::into_app_error)?; + let mut body = self.body(&wire_id, &request.text, true); + let messages = body["messages"] + .as_array_mut() + .ok_or_else(|| AiError::invalid_input().into_app_error())?; + messages + .last_mut() + .ok_or_else(|| AiError::invalid_input().into_app_error())?["images"] = json!( + request + .images + .iter() + .map(|image| encode_image(image.bytes())) + .collect::>() + ); + let response = self + .http + .request(self.chat_path(), Some(body), &cancel, INFERENCE_TIMEOUT) + .await + .map_err(AiError::into_app_error)?; + Ok(http::ollama_stream(response, cancel)) + } +} + +pub async fn connect_local_models(pool: &SqlitePool) -> AppResult { + let guard = providers::try_provider_mutation(pool).await?; + let previous_default: Option = + sqlx::query_scalar("SELECT default_learning_profile_id FROM app_settings WHERE id = 1") + .fetch_one(pool) + .await?; + let cancel = CancellationToken::new(); + let mut reports = Vec::new(); + let mut connected = Vec::new(); + let mut default_id = None; + for kind in [ProviderKind::Ollama, ProviderKind::LmStudio] { + let installation = Installation::detect(kind.clone(), None); + let port = installation.port; + let provider = LocalProvider::new(kind.clone(), port, DEFAULT_CONTEXT) + .map_err(AiError::into_app_error)?; + let models = match provider.models(&cancel).await { + Ok(models) => models, + Err(error) => { + reports.push(LocalServiceReport { + kind, + status: if error.kind() == AiErrorKind::InvalidApiKey { + LocalServiceStatus::AuthenticationRequired + } else if installation.cli.is_none() { + LocalServiceStatus::NotInstalled + } else { + LocalServiceStatus::Unavailable + }, + model_count: 0, + }); + continue; + } + }; + let mut working_model = None; + let mut authentication_required = false; + for model in &models { + let provider = LocalProvider::new(kind.clone(), port, model.context) + .map_err(AiError::into_app_error)?; + match provider.smoke_test(&model.id, &cancel).await { + Ok(()) => { + working_model = Some(model.id.clone()); + break; + } + Err(error) if error.kind() == AiErrorKind::InvalidApiKey => { + authentication_required = true; + break; + } + Err(_) => {} + } + } + let Some(working_model) = working_model else { + reports.push(LocalServiceReport { + kind, + status: if authentication_required { + LocalServiceStatus::AuthenticationRequired + } else if models.is_empty() { + LocalServiceStatus::NoModels + } else { + LocalServiceStatus::NoUsableModels + }, + model_count: 0, + }); + continue; + }; + let profiles = + providers::save_local_models(pool, &guard, kind.clone(), port, &models).await?; + if default_id.is_none() { + default_id = profiles + .iter() + .find(|profile| profile.model_id == working_model) + .map(|profile| profile.id); + } + reports.push(LocalServiceReport { + kind, + status: LocalServiceStatus::Connected, + model_count: profiles.len() as u32, + }); + connected.extend(profiles); + } + if let Some(profile) = connected + .iter() + .find(|profile| previous_default.as_deref() == Some(profile.id.to_string().as_str())) + { + default_id = Some(profile.id); + } + if let Some(id) = default_id { + providers::select_local_default(pool, &guard, id).await?; + } + for profile in &mut connected { + profile.is_active = Some(profile.id) == default_id; + } + Ok(LocalModelConnectResult { + profiles: connected, + default_profile_id: default_id, + services: reports, + }) +} + +fn valid_local_id(id: &str) -> bool { + validate_model_id(id).is_ok() + && !id.to_ascii_lowercase().contains("cloud") + && !id.starts_with('-') +} + +fn local_ollama_info(value: &Value) -> bool { + value + .get("remote_host") + .is_none_or(|v| v.is_null() || v == "") + && value + .get("remote_model") + .is_none_or(|v| v.is_null() || v == "") + && value["capabilities"] + .as_array() + .is_some_and(|caps| caps.iter().any(|cap| cap == "completion")) + && value["model_info"].is_object() +} + +fn ollama_has_vision(value: &Value) -> bool { + local_ollama_info(value) + && value["capabilities"] + .as_array() + .is_some_and(|caps| caps.iter().any(|cap| cap == "vision")) +} + +fn encode_image(bytes: &[u8]) -> String { + const TABLE: &[u8; 64] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"; + let mut output = String::with_capacity(bytes.len().div_ceil(3) * 4); + for chunk in bytes.chunks(3) { + let first = chunk[0]; + let second = chunk.get(1).copied().unwrap_or(0); + let third = chunk.get(2).copied().unwrap_or(0); + output.push(char::from(TABLE[usize::from(first >> 2)])); + output.push(char::from( + TABLE[usize::from(((first & 3) << 4) | (second >> 4))], + )); + output.push(if chunk.len() > 1 { + char::from(TABLE[usize::from(((second & 15) << 2) | (third >> 6))]) + } else { + '=' + }); + output.push(if chunk.len() > 2 { + char::from(TABLE[usize::from(third & 63)]) + } else { + '=' + }); + } + output +} + +fn parse_lmstudio_models(list: &Value) -> Result, AiError> { + Ok(list + .as_array() + .ok_or_else(AiError::malformed_event)? + .iter() + .take(256) + .filter(|model| local_lm_model(model)) + .filter_map(|model| { + let id = model_key(model).filter(|id| valid_local_id(id))?; + let context = model["maxContextLength"] + .as_u64() + .and_then(|v| u32::try_from(v).ok()) + .unwrap_or(DEFAULT_CONTEXT) + .min(DEFAULT_CONTEXT); + (context >= 2048).then(|| LocalModel { + id: id.to_owned(), + context, + vision: false, + size: model["sizeBytes"].as_u64().unwrap_or(u64::MAX), + loaded: false, + }) + }) + .collect()) +} + +#[cfg(test)] +mod tests; diff --git a/src-tauri/src/ai/local/tests.rs b/src-tauri/src/ai/local/tests.rs new file mode 100644 index 0000000..6a2300f --- /dev/null +++ b/src-tauri/src/ai/local/tests.rs @@ -0,0 +1,627 @@ +use super::*; +use crate::{ + ai::{registry::ProviderCapabilityRegistry, runtime::ProviderRuntime}, + credentials::MemoryCredentialStore, + db::{Database, providers}, + domain::{AiOperation, CredentialStatus, UnifiedMessage, UnifiedStreamEvent}, +}; +use futures_util::StreamExt; +use serde_json::json; +use std::sync::Arc; +use tempfile::TempDir; +use wiremock::{ + Mock, MockServer, ResponseTemplate, + matchers::{body_partial_json, method, path}, +}; + +fn info() -> Value { + json!({"capabilities":["completion"],"model_info":{"synthetic.context_length":32768}}) +} +fn request() -> UnifiedChatRequest { + UnifiedChatRequest { + model: "synthetic:small".to_owned(), + system: "Synthetic teaching instruction".to_owned(), + messages: vec![UnifiedMessage { + role: UnifiedRole::User, + content: "What is 2 + 2?".to_owned(), + }], + max_output_tokens: 512, + expected_language: None, + } +} + +#[tokio::test] +async fn local_vision_transmits_an_image_to_a_verified_local_vision_model() { + use crate::domain::{ImageLimits, ImageMime, UnifiedVisionRequest}; + let server = MockServer::start().await; + Mock::given(path("/api/tags")) + .respond_with(ResponseTemplate::new(200).set_body_json(json!({"models":[]}))) + .mount(&server) + .await; + Mock::given(path("/api/show")) + .respond_with(ResponseTemplate::new(200).set_body_json(json!({ + "capabilities":["completion","vision"],"model_info":{} + }))) + .mount(&server) + .await; + Mock::given(path("/api/chat")) + .respond_with(ResponseTemplate::new(200).set_body_string( + "{\"message\":{\"content\":\"blue\"},\"done\":true,\"done_reason\":\"stop\"}\n", + )) + .expect(1) + .mount(&server) + .await; + let bytes = include_bytes!(concat!( + env!("CARGO_MANIFEST_DIR"), + "/../fixtures/source/vision/tiny-blue.png" + )) + .to_vec(); + let asset = crate::ai::multimodal::stage_vision_asset( + uuid::Uuid::new_v4(), + uuid::Uuid::new_v4(), + ImageMime::Png, + u32::from_be_bytes(bytes[16..20].try_into().unwrap()), + u32::from_be_bytes(bytes[20..24].try_into().unwrap()), + bytes, + ImageLimits { + max_images: 1, + max_encoded_bytes_each: 1_000_000, + max_total_encoded_bytes: 1_000_000, + max_dimension_px: 2048, + max_decoded_pixels_each: 4_194_304, + }, + ) + .unwrap(); + let provider = LocalProvider::new(ProviderKind::Ollama, server.address().port(), 8192).unwrap(); + let stream = provider + .stream_vision( + &SecretString::default(), + UnifiedVisionRequest { + text: request(), + images: vec![asset], + }, + CancellationToken::new(), + ) + .await; + assert!( + stream.is_ok(), + "installed local vision model should accept an image" + ); + let events = stream.unwrap().collect::>().await; + assert!(matches!( + events.last(), + Some(Ok(UnifiedStreamEvent::Completed)) + )); + let requests = server.received_requests().await.unwrap(); + let sent = requests + .iter() + .find(|r| r.url.path() == "/api/chat") + .unwrap(); + let body: Value = serde_json::from_slice(&sent.body).unwrap(); + assert_eq!( + body["messages"][1]["images"][0], + "iVBORw0KGgoAAAANSUhEUgAAAAIAAAACCAIAAAD91JpzAAAACXBIWXMAAAABAAAAAQBPJcTWAAAAEElEQVR4nGMwTDkJRAwQCgAj9gV5wT7pWAAAAABJRU5ErkJggg==" + ); + assert_eq!(body["messages"][1]["content"], "What is 2 + 2?"); + assert!(!sent.headers.contains_key("authorization")); +} + +#[tokio::test] +async fn local_vision_rechecks_capabilities_and_never_sends_images_to_text_or_remote_models() { + for metadata in [ + info(), + json!({"capabilities":["completion","vision"],"model_info":{},"remote_host":"https://example.invalid"}), + ] { + let server = MockServer::start().await; + Mock::given(path("/api/tags")) + .respond_with(ResponseTemplate::new(200).set_body_json(json!({"models":[]}))) + .mount(&server) + .await; + Mock::given(path("/api/show")) + .respond_with(ResponseTemplate::new(200).set_body_json(metadata)) + .mount(&server) + .await; + Mock::given(path("/api/chat")) + .respond_with(ResponseTemplate::new(200)) + .expect(0) + .mount(&server) + .await; + let bytes = include_bytes!(concat!( + env!("CARGO_MANIFEST_DIR"), + "/../fixtures/source/vision/tiny-blue.png" + )) + .to_vec(); + let image = crate::ai::multimodal::stage_vision_asset( + uuid::Uuid::new_v4(), + uuid::Uuid::new_v4(), + crate::domain::ImageMime::Png, + 2, + 2, + bytes, + LOCAL_IMAGE_LIMITS, + ) + .unwrap(); + let provider = + LocalProvider::new(ProviderKind::Ollama, server.address().port(), 8192).unwrap(); + assert!( + provider + .stream_vision( + &SecretString::default(), + crate::domain::UnifiedVisionRequest { + text: request(), + images: vec![image] + }, + CancellationToken::new() + ) + .await + .is_err() + ); + } +} + +#[test] +fn local_vision_capability_and_separate_defaults_survive_restart_and_downgrade() { + let temp = TempDir::new().unwrap(); + let path = temp.path().join("vision.sqlite3"); + let database = Database::open(&path).unwrap(); + let registry = ProviderCapabilityRegistry::load_embedded().unwrap(); + let (vision_id, text_id) = tauri::async_runtime::block_on(async { + let guard = providers::try_provider_mutation(database.pool()) + .await + .unwrap(); + let models = [ + LocalModel { + id: "synthetic:vision".into(), + context: 16384, + vision: true, + size: 1, + loaded: false, + }, + LocalModel { + id: "synthetic:text".into(), + context: 8192, + vision: false, + size: 1, + loaded: false, + }, + ]; + let profiles = + providers::save_local_models(database.pool(), &guard, ProviderKind::Ollama, 9, &models) + .await + .unwrap(); + providers::select_local_default(database.pool(), &guard, profiles[1].id) + .await + .unwrap(); + drop(guard); + providers::set_default_provider_profile( + database.pool(), + ®istry, + AiOperation::VisionLearning, + profiles[0].id, + ) + .await + .unwrap(); + assert!( + providers::set_default_provider_profile( + database.pool(), + ®istry, + AiOperation::VisionLearning, + profiles[1].id + ) + .await + .is_err() + ); + let runtime = + ProviderRuntime::new(Arc::new(MemoryCredentialStore::default()), registry.clone()); + assert!( + runtime + .load(database.pool(), profiles[0].id, AiOperation::VisionLearning) + .await + .is_ok() + ); + assert!( + runtime + .load( + database.pool(), + profiles[0].id, + AiOperation::StructuredPageAnalysis + ) + .await + .is_err() + ); + database.pool().close().await; + (profiles[0].id, profiles[1].id) + }); + let reopened = Database::open(&path).unwrap(); + tauri::async_runtime::block_on(async { + let settings = crate::db::settings::get_app_settings(reopened.pool()) + .await + .unwrap(); + assert_eq!(settings.default_learning_profile_id, Some(text_id)); + assert_eq!(settings.default_vision_profile_id, Some(vision_id)); + let public = crate::db::local_capabilities::public_registry(reopened.pool(), ®istry) + .await + .unwrap(); + let provider = public + .providers + .iter() + .find(|p| p.kind == ProviderKind::Ollama) + .unwrap(); + assert_eq!( + provider + .models + .iter() + .find(|m| m.id == "synthetic:vision") + .unwrap() + .image_input, + crate::domain::CapabilitySupport::Supported + ); + let guard = providers::try_provider_mutation(reopened.pool()) + .await + .unwrap(); + providers::save_local_models( + reopened.pool(), + &guard, + ProviderKind::Ollama, + 9, + &[LocalModel { + id: "synthetic:vision".into(), + context: 8192, + vision: false, + size: 1, + loaded: false, + }], + ) + .await + .unwrap(); + drop(guard); + assert!( + providers::set_default_provider_profile( + reopened.pool(), + ®istry, + AiOperation::VisionLearning, + vision_id + ) + .await + .is_err() + ); + reopened.pool().close().await; + }); +} + +#[test] +fn local_inventory_excludes_cloud_remote_and_embedding_models() { + assert!(local_ollama_info(&info())); + let mut remote = info(); + remote["remote_host"] = json!("https://ollama.com"); + assert!(!local_ollama_info(&remote)); + assert!(!local_ollama_info( + &json!({"capabilities":["embedding"],"model_info":{}}) + )); + let models = parse_lmstudio_models(&json!([ + {"type":"llm","modelKey":"synthetic-local","deviceIdentifier":null,"sizeBytes":100,"maxContextLength":4096}, + {"type":"llm","modelKey":"remote","deviceIdentifier":"remote-device"}, + {"type":"embedding","modelKey":"embedding","deviceIdentifier":null} + ])).unwrap(); + assert_eq!(models.len(), 1); + assert_eq!(models[0].id, "synthetic-local"); + assert_eq!(models[0].context, 4096); +} + +#[tokio::test] +async fn ollama_preserves_the_model_prompt_and_emits_only_the_final_answer() { + let server = MockServer::start().await; + Mock::given(path("/api/tags")) + .respond_with(ResponseTemplate::new(200).set_body_json(json!({"models":[]}))) + .mount(&server) + .await; + let mut metadata = info(); + metadata["model_info"]["general.architecture"] = json!("qwen3"); + metadata["template"] = json!("old template"); + Mock::given(path("/api/show")) + .respond_with(ResponseTemplate::new(200).set_body_json(metadata)) + .mount(&server) + .await; + Mock::given(path("/api/chat")).and(body_partial_json(json!({"messages":[{"role":"system","content":"Synthetic teaching instruction"},{"role":"user","content":"What is 2 + 2?"}]}))) + .respond_with(ResponseTemplate::new(200).set_body_string("{\"message\":{\"thinking\":\"hidden synthetic reasoning\"},\"done\":false}\n{\"message\":{\"content\":\"4\"},\"done\":true,\"done_reason\":\"stop\"}\n")).mount(&server).await; + let provider = LocalProvider::new(ProviderKind::Ollama, server.address().port(), 8192).unwrap(); + let events = provider + .stream_chat( + &SecretString::default(), + request(), + CancellationToken::new(), + ) + .await + .unwrap() + .collect::>() + .await; + assert!(matches!( + events.last(), + Some(Ok(UnifiedStreamEvent::Completed)) + )); + assert_eq!(events.len(), 2); + assert_eq!( + events[0].as_ref().unwrap(), + &UnifiedStreamEvent::TextDelta { + text: "4".to_owned() + } + ); +} + +#[tokio::test] +async fn ollama_cancel_discards_queued_completion() { + let server = MockServer::start().await; + Mock::given(path("/stream")) + .respond_with(ResponseTemplate::new(200).set_body_string( + "{\"message\":{\"content\":\"4\"},\"done\":true,\"done_reason\":\"stop\"}\n", + )) + .mount(&server) + .await; + let cancel = CancellationToken::new(); + let response = LocalHttp::new(server.address().port()) + .unwrap() + .request("/stream", None, &cancel, METADATA_TIMEOUT) + .await + .unwrap(); + let mut stream = http::ollama_stream(response, cancel.clone()); + assert!(matches!( + stream.next().await, + Some(Ok(UnifiedStreamEvent::TextDelta { .. })) + )); + cancel.cancel(); + assert!(matches!(stream.next().await,Some(Err(error)) if error.kind()==AiErrorKind::Cancelled)); + assert!(stream.next().await.is_none()); +} + +#[test] +fn local_profiles_survive_restart_deduplicate_and_answer_without_credentials() { + let temp = TempDir::new().unwrap(); + let db_path = temp.path().join("local.sqlite3"); + let database = Database::open(&db_path).unwrap(); + tauri::async_runtime::block_on(async { + let server = MockServer::start().await; + Mock::given(method("GET")) + .and(path("/api/tags")) + .respond_with(ResponseTemplate::new(200).set_body_json(json!({"models":[]}))) + .mount(&server) + .await; + Mock::given(method("POST")) + .and(path("/api/show")) + .respond_with(ResponseTemplate::new(200).set_body_json(info())) + .mount(&server) + .await; + Mock::given(method("POST")).and(path("/api/chat")).and(body_partial_json(json!({"think":false,"options":{"num_ctx":8192},"messages":[{"role":"system","content":"Synthetic teaching instruction"},{"role":"user","content":"What is 2 + 2?"}]}))) + .respond_with(ResponseTemplate::new(200).set_body_string("{\"message\":{\"content\":\"4\"},\"done\":false}\n{\"message\":{\"content\":\"\"},\"done\":true,\"done_reason\":\"stop\"}\n")).expect(1).mount(&server).await; + let guard = providers::try_provider_mutation(database.pool()) + .await + .unwrap(); + let model = LocalModel { + id: "synthetic:small".to_owned(), + context: 8192, + size: 100, + loaded: true, + vision: false, + }; + let profiles = providers::save_local_models( + database.pool(), + &guard, + ProviderKind::Ollama, + server.address().port(), + std::slice::from_ref(&model), + ) + .await + .unwrap(); + let again = providers::save_local_models( + database.pool(), + &guard, + ProviderKind::Ollama, + server.address().port(), + &[model], + ) + .await + .unwrap(); + assert_eq!(profiles[0].id, again[0].id); + providers::select_local_default(database.pool(), &guard, profiles[0].id) + .await + .unwrap(); + drop(guard); + let store = Arc::new(MemoryCredentialStore::default()); + let list = providers::list_provider_profiles(database.pool(), store.as_ref()) + .await + .unwrap(); + assert_eq!(list.len(), 1); + assert_eq!(list[0].credential_status, CredentialStatus::NotRequired); + assert!(list[0].is_active); + let runtime = ProviderRuntime::new( + store.clone(), + ProviderCapabilityRegistry::load_embedded().unwrap(), + ); + let provider = runtime + .load(database.pool(), profiles[0].id, AiOperation::TextLearning) + .await + .unwrap(); + assert!( + runtime + .load(database.pool(), profiles[0].id, AiOperation::VisionLearning) + .await + .is_err() + ); + let events = provider + .stream_text_learning(request(), CancellationToken::new()) + .await + .unwrap() + .collect::>() + .await; + assert_eq!(events.len(), 2); + assert_eq!( + events[0].as_ref().unwrap(), + &UnifiedStreamEvent::TextDelta { + text: "4".to_owned() + } + ); + assert_eq!(events[1].as_ref().unwrap(), &UnifiedStreamEvent::Completed); + database.pool().close().await; + }); + let database = Database::open(&db_path).unwrap(); + tauri::async_runtime::block_on(async { + let store = Arc::new(MemoryCredentialStore::default()); + let list = providers::list_provider_profiles(database.pool(), store.as_ref()) + .await + .unwrap(); + assert_eq!(list.len(), 1); + assert!(list[0].is_active); + providers::delete_provider_profile(database.pool(), store.clone(), list[0].id) + .await + .unwrap(); + assert!( + providers::list_provider_profiles(database.pool(), store.as_ref()) + .await + .unwrap() + .is_empty() + ); + database.pool().close().await; + }); +} + +#[tokio::test] +async fn local_http_never_follows_redirects_and_remote_models_never_receive_prompts() { + let remote = MockServer::start().await; + Mock::given(method("GET")) + .respond_with(ResponseTemplate::new(200)) + .expect(0) + .mount(&remote) + .await; + let server = MockServer::start().await; + Mock::given(path("/redirect")) + .respond_with(ResponseTemplate::new(302).insert_header("Location", remote.uri())) + .mount(&server) + .await; + let client = LocalHttp::new(server.address().port()).unwrap(); + assert!( + client + .json( + "/redirect", + None, + &CancellationToken::new(), + METADATA_TIMEOUT + ) + .await + .is_err() + ); + Mock::given(path("/api/tags")) + .respond_with(ResponseTemplate::new(200).set_body_json(json!({"models":[]}))) + .mount(&server) + .await; + let mut remote_info = info(); + remote_info["remote_model"] = json!("remote-model"); + Mock::given(path("/api/show")) + .respond_with(ResponseTemplate::new(200).set_body_json(remote_info)) + .mount(&server) + .await; + Mock::given(path("/api/chat")) + .respond_with(ResponseTemplate::new(200)) + .expect(0) + .mount(&server) + .await; + let provider = LocalProvider::new(ProviderKind::Ollama, server.address().port(), 8192).unwrap(); + assert!( + provider + .stream_chat( + &SecretString::default(), + request(), + CancellationToken::new() + ) + .await + .is_err() + ); +} + +#[tokio::test] +async fn lmstudio_stream_requires_a_successful_terminal_event() { + let server = MockServer::start().await; + Mock::given(path("/stream")).respond_with(ResponseTemplate::new(200).set_body_string("data: {\"choices\":[{\"index\":0,\"delta\":{\"content\":\"4\"},\"finish_reason\":null}]}\n\n")).mount(&server).await; + let http = LocalHttp::new(server.address().port()).unwrap(); + let cancel = CancellationToken::new(); + let response = http + .request("/stream", None, &cancel, METADATA_TIMEOUT) + .await + .unwrap(); + let events = http::lmstudio_stream(response, cancel) + .collect::>() + .await; + assert!(events[0].is_ok()); + assert!(events.last().unwrap().is_err()); +} + +#[test] +#[ignore = "Requires an installed local runtime and already downloaded model; uses only a temporary database and synthetic question"] +fn installed_local_runtime_offline_smoke() { + let temp = TempDir::new().unwrap(); + let database = Database::open(temp.path().join("smoke.sqlite3")).unwrap(); + tauri::async_runtime::block_on(async { + let result = connect_local_models(database.pool()).await.unwrap(); + let id = result + .default_profile_id + .expect("a local model must load and answer"); + let provider = ProviderRuntime::new( + Arc::new(MemoryCredentialStore::default()), + ProviderCapabilityRegistry::load_embedded().unwrap(), + ) + .load(database.pool(), id, AiOperation::TextLearning) + .await + .unwrap(); + let mut request = request(); + request.model = provider.profile().model_id.clone(); + request.max_output_tokens = + crate::ai::registry::local_output_tokens(provider.profile().context_window_tokens); + let events = provider + .stream_text_learning(request, CancellationToken::new()) + .await + .unwrap() + .collect::>() + .await; + assert!( + events.iter().all(Result::is_ok), + "stream errors: {:?}", + events + .iter() + .filter_map(|event| event.as_ref().err().map(|error| error.kind())) + .collect::>() + ); + assert!(events.iter().any(|event| matches!(event,Ok(UnifiedStreamEvent::TextDelta{text}) if !text.trim().is_empty()))); + assert!(matches!( + events.last(), + Some(Ok(UnifiedStreamEvent::Completed)) + )); + database.pool().close().await; + }); +} + +#[test] +#[ignore = "Requires an unused test-specific OLLAMA_HOST port; the test runner must clean up the server it starts"] +fn installed_ollama_service_starts_offline() { + tauri::async_runtime::block_on(async { + let installation = Installation::detect(ProviderKind::Ollama, None); + assert_ne!( + installation.port, 11434, + "use an isolated port, never the user's normal server" + ); + let http = LocalHttp::new(installation.port).unwrap(); + let cancel = CancellationToken::new(); + assert!( + http.json("/api/tags", None, &cancel, METADATA_TIMEOUT) + .await + .is_err() + ); + installation.ensure_running(&cancel).await.unwrap(); + let list = http + .json("/api/tags", None, &cancel, METADATA_TIMEOUT) + .await + .unwrap(); + assert!( + list["models"] + .as_array() + .is_some_and(|models| !models.is_empty()) + ); + }); +} diff --git a/src-tauri/src/ai/mod.rs b/src-tauri/src/ai/mod.rs index b2fe4ee..3986e19 100644 --- a/src-tauri/src/ai/mod.rs +++ b/src-tauri/src/ai/mod.rs @@ -1,5 +1,6 @@ pub mod error; pub(crate) mod kimi_region; +pub mod local; pub mod multimodal; pub mod provider; pub mod providers; diff --git a/src-tauri/src/ai/registry.rs b/src-tauri/src/ai/registry.rs index e1b8dca..aa89799 100644 --- a/src-tauri/src/ai/registry.rs +++ b/src-tauri/src/ai/registry.rs @@ -18,6 +18,16 @@ const MAX_REGISTRY_IMAGE_DIMENSION: u32 = 8_192; const MAX_REGISTRY_DECODED_PIXELS: u64 = 40_000_000; pub const UNKNOWN_MODEL_CONTEXT_WINDOW_TOKENS: u32 = 32_000; +pub fn local_output_tokens(context: u32) -> u32 { + // Reasoning and visible text share the generation allowance. The normal + // 8192-token local context already reserves 4096 tokens for generation. + if context >= 8_192 { + 4_096 + } else { + (context / 4).clamp(256, 2_048) + } +} + #[derive(Clone, Debug)] pub struct ProviderCapabilityRegistry { schema_version: u16, @@ -172,6 +182,13 @@ impl ProviderCapabilityRegistry { model_id: &str, operation: AiOperation, ) -> CapabilitySupport { + if kind.is_local() { + return if operation == AiOperation::TextLearning { + CapabilitySupport::Supported + } else { + CapabilitySupport::Unsupported + }; + } let Some(model) = self.model(kind, model_id) else { return CapabilitySupport::Unknown; }; @@ -234,6 +251,31 @@ impl ProviderCapabilityRegistry { .find(|capability| &capability.kind == kind) .and_then(|capability| capability.models.iter().find(|model| model.id == model_id)) } + + /// Local capabilities come from validated installed models, never the cloud model catalog. + pub fn model_for_profile( + &self, + kind: &ProviderKind, + id: &str, + context: u32, + ) -> Option { + if kind.is_local() { + return Some(ProviderModelCapability { + id: id.to_owned(), + display_name: id.to_owned(), + context_window_tokens: context, + default_max_output_tokens: local_output_tokens(context), + text_chat: CapabilitySupport::Supported, + image_input: CapabilitySupport::Unsupported, + native_pdf_input: CapabilitySupport::Unsupported, + pdf_input: CapabilitySupport::Unsupported, + strict_structured_output: CapabilitySupport::Unsupported, + image_limits: None, + last_verified: String::new(), + }); + } + self.model(kind, id).cloned() + } } fn validate(document: &RegistryDocument) -> Result<(), RegistryError> { diff --git a/src-tauri/src/ai/runtime.rs b/src-tauri/src/ai/runtime.rs index 113b519..1dc13ed 100644 --- a/src-tauri/src/ai/runtime.rs +++ b/src-tauri/src/ai/runtime.rs @@ -158,11 +158,35 @@ impl ProviderRuntime { region_override: Option, ) -> AppResult { let mut profile = providers::load_provider_profile_metadata(pool, profile_id).await?; - self.ensure_model_support(&profile.kind, &profile.model_id, operation)?; + if profile.kind.is_local() { + if !crate::db::local_capabilities::supports(pool, &self.registry, &profile, operation) + .await? + { + return Err(AppError::unsupported_provider_capability()); + } + } else { + self.ensure_model_support(&profile.kind, &profile.model_id, operation)?; + } if profile.kind != ProviderKind::Kimi && region_override.is_some() { return Err(AppError::new(AppErrorCode::InvalidInput)); } let region = region_override.or(profile.kimi_api_region); + if profile.kind.is_local() { + let port = providers::load_local_port(pool, profile_id).await?; + let adapter = super::local::LocalProvider::new( + profile.kind.clone(), + port, + profile.context_window_tokens, + ) + .map_err(AiError::into_app_error)?; + return Ok(LoadedProvider { + profile, + operation, + adapter: Box::new(adapter), + // Legacy trait argument is unused by LocalProvider; no credential is read or sent. + credential: SecretString::default(), + }); + } let adapter = self.adapter(&profile.kind, region)?; if adapter.kind() != profile.kind { return Err(AppError::new(AppErrorCode::ProviderUnavailable)); @@ -187,6 +211,9 @@ impl ProviderRuntime { credential: SecretString, operation: AiOperation, ) -> AppResult { + if kind.is_local() { + return Err(AppError::new(AppErrorCode::InvalidInput)); + } validate_credential(&credential).map_err(|error| error.into_app_error())?; let model_id = self.resolve_model(&kind, requested_model)?; self.ensure_model_support(&kind, &model_id, operation)?; @@ -265,6 +292,14 @@ impl ProviderRuntime { model_id: &str, operation: AiOperation, ) -> AppResult<()> { + if kind.is_local() { + validate_model_id(model_id).map_err(AiError::into_app_error)?; + return if operation == AiOperation::TextLearning { + Ok(()) + } else { + Err(AppError::unsupported_provider_capability()) + }; + } let owns_model = self .registry .capabilities() @@ -290,6 +325,7 @@ impl ProviderRuntime { #[cfg(test)] if let Some(origin) = self.loopback_origin.as_deref() { return match kind { + ProviderKind::Ollama | ProviderKind::LmStudio => Err(AiError::invalid_input()), ProviderKind::OpenAi => OpenAiProvider::new_for_test(origin) .map(|provider| Box::new(provider) as Box), ProviderKind::Gemini => GeminiProvider::new_for_test(origin) @@ -320,6 +356,7 @@ impl ProviderRuntime { } match kind { + ProviderKind::Ollama | ProviderKind::LmStudio => Err(AiError::invalid_input()), ProviderKind::OpenAi => { OpenAiProvider::new().map(|provider| Box::new(provider) as Box) } diff --git a/src-tauri/src/ai/transport.rs b/src-tauri/src/ai/transport.rs index e88e28e..0d0d777 100644 --- a/src-tauri/src/ai/transport.rs +++ b/src-tauri/src/ai/transport.rs @@ -438,6 +438,7 @@ fn production_origin(kind: &ProviderKind) -> Option<&'static str> { ProviderKind::Anthropic => Some("https://api.anthropic.com/"), ProviderKind::DeepSeek => Some("https://api.deepseek.com/"), ProviderKind::Kimi => None, + ProviderKind::Ollama | ProviderKind::LmStudio => None, } } diff --git a/src-tauri/src/commands/onboarding.rs b/src-tauri/src/commands/onboarding.rs index 80d7ac1..8812afe 100644 --- a/src-tauri/src/commands/onboarding.rs +++ b/src-tauri/src/commands/onboarding.rs @@ -40,20 +40,25 @@ pub async fn get_onboarding_state( .any(|book| book.import_status == ImportStatus::Ready); let connected = |id: Option| { id.and_then(|profile_id| profiles.iter().find(|profile| profile.id == profile_id)) - .is_some_and(|profile| profile.credential_status == CredentialStatus::Available) + .is_some_and(|profile| profile.credential_status != CredentialStatus::Missing) }; let learning_profile_connected = connected(settings.default_learning_profile_id); - let vision_profile_connected = settings + let vision_profile_connected = if let Some(profile) = settings .default_vision_profile_id - .and_then(|profile_id| profiles.iter().find(|profile| profile.id == profile_id)) - .is_some_and(|profile| { - profile.credential_status == CredentialStatus::Available - && state.provider_capabilities.operation_support( - &profile.kind, - &profile.model_id, - crate::domain::AiOperation::VisionLearning, - ) == crate::domain::CapabilitySupport::Supported - }); + .and_then(|id| profiles.iter().find(|profile| profile.id == id)) + { + profile.credential_status != CredentialStatus::Missing + && crate::db::local_capabilities::supports( + state.db.pool(), + &state.provider_capabilities, + profile, + crate::domain::AiOperation::VisionLearning, + ) + .await + .map_err(AppErrorDto::from)? + } else { + false + }; let local_text_quality = match selected_book.as_ref() { Some(book) if book.import_status == ImportStatus::Ready => LocalTextQuality::Ready, Some(book) diff --git a/src-tauri/src/commands/providers.rs b/src-tauri/src/commands/providers.rs index 5858e37..e024e57 100644 --- a/src-tauri/src/commands/providers.rs +++ b/src-tauri/src/commands/providers.rs @@ -14,8 +14,13 @@ use crate::{ }; #[tauri::command] -pub fn list_provider_capabilities(state: State<'_, AppState>) -> ProviderCapabilityRegistryDto { - state.provider_capabilities.public_registry() +pub async fn list_provider_capabilities( + state: State<'_, AppState>, +) -> Result { + let _permit = maintenance_permit(&state)?; + crate::db::local_capabilities::public_registry(state.db.pool(), &state.provider_capabilities) + .await + .map_err(AppErrorDto::from) } #[tauri::command] @@ -28,12 +33,25 @@ pub async fn list_provider_profiles( providers::list_provider_profiles(state.db.pool(), state.credential_store.as_ref()) .await .map_err(AppErrorDto::from)?; - Ok(match operation { - Some(operation) => state - .provider_capabilities - .resolve_operation(operation, &profiles), - None => profiles, - }) + if let Some(operation) = operation { + let mut compatible = Vec::new(); + for profile in profiles { + if crate::db::local_capabilities::supports( + state.db.pool(), + &state.provider_capabilities, + &profile, + operation, + ) + .await + .map_err(AppErrorDto::from)? + { + compatible.push(profile); + } + } + Ok(compatible) + } else { + Ok(profiles) + } } #[tauri::command] @@ -140,3 +158,13 @@ fn maintenance_permit( .map_err(crate::errors::AppError::from) .map_err(AppErrorDto::from) } + +#[tauri::command] +pub async fn connect_local_models( + state: State<'_, AppState>, +) -> Result { + let _permit = maintenance_permit(&state)?; + crate::ai::local::connect_local_models(state.db.pool()) + .await + .map_err(AppErrorDto::from) +} diff --git a/src-tauri/src/db/annotations.rs b/src-tauri/src/db/annotations.rs index 8406bd4..e7f1d34 100644 --- a/src-tauri/src/db/annotations.rs +++ b/src-tauri/src/db/annotations.rs @@ -374,6 +374,7 @@ impl MarkerRelocationContext { locator_section == §ion_id && stored_section == §ion_id && (cfi == stored_cfi + || same_epub_spine(cfi, stored_cfi) || self.blocks.iter().any(|block| { block.section_id == section_id && matches!(&block.locator, Some(DocumentLocator::Epub { section_id: block_section, cfi: block_cfi }) if block_section == §ion_id && block_cfi == cfi) @@ -676,6 +677,7 @@ async fn resolve_region_anchor( locator_section == §ion_id && stored_section == §ion_id && (cfi == stored_cfi + || same_epub_spine(cfi, stored_cfi) || epub_block_locator_exists(pool, book_id, section_id, cfi).await?) } (RegionLocator::Docx { block_id }, "docx", _) => { @@ -718,6 +720,55 @@ async fn epub_block_locator_exists( .await?) } +// An image element has its own CFI, often absent from the text-only block index. +// The database verifies section/spine ownership; the reader verifies the exact +// element and its content hash when placing a marker. +fn same_epub_spine(cfi: &str, section_cfi: &str) -> bool { + fn package(cfi: &str) -> Option<&str> { + if cfi.len() > 8192 || cfi.chars().any(char::is_control) { + return None; + } + let inner = cfi.strip_prefix("epubcfi(")?.strip_suffix(')')?; + let (package, local) = inner.split_once('!')?; + if !local.starts_with('/') + || !local.as_bytes().get(1).is_some_and(u8::is_ascii_digit) + || local.contains('!') + { + return None; + } + let mut assertion = false; + let mut escaped = false; + for c in inner.chars() { + if escaped { + escaped = false; + continue; + } + match c { + '^' => escaped = true, + '[' if !assertion => assertion = true, + ']' if assertion => assertion = false, + '[' | ']' | '(' | ')' => return None, + _ => {} + } + } + if assertion || escaped { + return None; + } + let steps = package.strip_prefix('/')?.split('/').collect::>(); + if steps.len() != 2 { + return None; + } + for step in steps { + let number = step.split('[').next()?.parse::().ok()?; + if number == 0 || !number.is_multiple_of(2) { + return None; + } + } + Some(package) + } + matches!((package(cfi),package(section_cfi)),(Some(left),Some(right)) if left==right) +} + async fn primary_hash_semantics_hold( pool: &SqlitePool, book_id: Uuid, diff --git a/src-tauri/src/db/local_capabilities.rs b/src-tauri/src/db/local_capabilities.rs new file mode 100644 index 0000000..7fa6c3a --- /dev/null +++ b/src-tauri/src/db/local_capabilities.rs @@ -0,0 +1,107 @@ +use sqlx::{Row, SqlitePool}; + +use crate::{ + ai::registry::ProviderCapabilityRegistry, + domain::{ + AiOperation, CapabilitySupport, ProviderCapability, ProviderCapabilityRegistryDto, + ProviderFileCapabilities, ProviderKind, ProviderModelCapability, ProviderProfileSummary, + }, + errors::{AppError, AppResult}, +}; + +/// Persisted discovery enables the UI; the adapter independently rechecks locality and vision +/// capability before sending any image to the runtime. +pub(crate) async fn model( + pool: &SqlitePool, + registry: &ProviderCapabilityRegistry, + profile: &ProviderProfileSummary, +) -> AppResult { + let mut model = registry + .model_for_profile( + &profile.kind, + &profile.model_id, + profile.context_window_tokens, + ) + .ok_or_else(AppError::unsupported_provider_capability)?; + if profile.kind == ProviderKind::Ollama && profile.validated_at.is_some() { + let vision: bool = + sqlx::query_scalar("SELECT local_vision FROM provider_profiles WHERE id = ?") + .bind(profile.id.to_string()) + .fetch_one(pool) + .await?; + if vision { + model.image_input = CapabilitySupport::Supported; + model.image_limits = Some(crate::ai::local::LOCAL_IMAGE_LIMITS); + } + } + Ok(model) +} + +pub(crate) async fn supports( + pool: &SqlitePool, + registry: &ProviderCapabilityRegistry, + profile: &ProviderProfileSummary, + operation: AiOperation, +) -> AppResult { + if profile.kind.is_local() { + return Ok(match operation { + AiOperation::TextLearning => true, + AiOperation::VisionLearning => { + model(pool, registry, profile).await?.image_input == CapabilitySupport::Supported + } + AiOperation::StructuredPageAnalysis => false, + }); + } + let support = registry.operation_support(&profile.kind, &profile.model_id, operation); + Ok(support == CapabilitySupport::Supported + || (support == CapabilitySupport::Unknown + && operation == AiOperation::TextLearning + && profile.validated_at.is_some())) +} + +pub(crate) async fn public_registry( + pool: &SqlitePool, + registry: &ProviderCapabilityRegistry, +) -> AppResult { + let mut result = registry.public_registry(); + let rows = sqlx::query("SELECT id FROM provider_profiles WHERE provider_kind IN ('ollama','lm_studio') ORDER BY created_at,id") + .fetch_all(pool).await?; + for row in rows { + let id: String = row.try_get("id")?; + let profile = super::providers::load_provider_profile_metadata( + pool, + id.parse() + .map_err(|_| AppError::unsupported_provider_capability())?, + ) + .await?; + let model = model(pool, registry, &profile).await?; + if let Some(provider) = result.providers.iter_mut().find(|p| p.kind == profile.kind) { + if let Some(existing) = provider.models.iter_mut().find(|m| m.id == model.id) { + // Registry DTOs are model keyed, while identities include the local port. + // Advertise the conservative intersection when different services reuse a name. + if model.image_input != CapabilitySupport::Supported { + existing.image_input = CapabilitySupport::Unsupported; + existing.image_limits = None; + } + existing.context_window_tokens = existing + .context_window_tokens + .min(model.context_window_tokens); + } else { + provider.models.push(model); + } + } else { + result.providers.push(ProviderCapability { + kind: profile.kind.clone(), + display_name: profile.kind.display_name().to_owned(), + default_model: profile.model_id.clone(), + models: vec![model], + file_capabilities: ProviderFileCapabilities { + file_extraction: false, + file_ocr: false, + max_file_bytes: None, + }, + }); + } + } + Ok(result) +} diff --git a/src-tauri/src/db/mod.rs b/src-tauri/src/db/mod.rs index e8fceef..978cee8 100644 --- a/src-tauri/src/db/mod.rs +++ b/src-tauri/src/db/mod.rs @@ -2,6 +2,7 @@ pub mod annotations; pub mod conversations; pub mod corrections; pub mod indexing; +pub(crate) mod local_capabilities; pub mod messages; pub mod notes; pub mod overview; @@ -275,7 +276,7 @@ fn atomic_migration_sql(migration: &sqlx::migrate::Migration) -> MigrationResult if !migration.no_tx { return Ok(migration.sql.clone()); } - if !matches!(migration.version, 2 | 11) { + if !matches!(migration.version, 2 | 11 | 16) { return Err(()); } diff --git a/src-tauri/src/db/notes_test.rs b/src-tauri/src/db/notes_test.rs index d3753eb..3434f77 100644 --- a/src-tauri/src/db/notes_test.rs +++ b/src-tauri/src/db/notes_test.rs @@ -20,6 +20,73 @@ const NOW: &str = "2026-08-05T00:00:00.000Z"; const BODY_SENTINEL: &str = "NOTE_BODY_PRIVATE_SENTINEL"; const HASH_SENTINEL: &str = "abcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdefabcd"; +#[test] +fn epub_image_element_cfi_can_be_saved_and_reopened_only_in_its_own_spine() { + let temporary = tempfile::tempdir().unwrap(); + let path = temporary.path().join("image-anchor.sqlite3"); + let database = Database::open(&path).unwrap(); + let book_id = Uuid::new_v4(); + let section_id = Uuid::new_v4(); + tauri::async_runtime::block_on(async { + seed_book( + database.pool(), + book_id, + section_id, + "epub", + DocumentLocator::epub("epubcfi(/6/2!/4/2[chapter-1]/2)".into(), section_id).unwrap(), + None, + ) + .await; + create_note( + database.pool(), + CreateNote { + book_id, + section_id, + anchor: region_anchor( + RegionLocator::epub(section_id, "epubcfi(/6/2!/4/12/2)".into()).unwrap(), + ), + selected_text: None, + note_text: "Synthetic image note".into(), + }, + ) + .await + .unwrap(); + for cfi in [ + "epubcfi(/6/4!/4/12/2)", + "epubcfi(/6/2!/)", + "epubcfi(/6/2[bad!/4/12/2)", + ] { + let error = create_note( + database.pool(), + CreateNote { + book_id, + section_id, + anchor: region_anchor(RegionLocator::epub(section_id, cfi.into()).unwrap()), + selected_text: None, + note_text: "Must reject".into(), + }, + ) + .await + .unwrap_err(); + assert_eq!(error.code, AppErrorCode::AnchorNotFound); + } + database.pool().close().await; + }); + drop(database); + let reopened = Database::open(&path).unwrap(); + tauri::async_runtime::block_on(async { + let markers = list_annotation_markers(reopened.pool(), book_id) + .await + .unwrap(); + assert_eq!(markers.len(), 1); + assert_eq!( + markers[0].relocation_status, + MarkerRelocationStatus::Primary + ); + reopened.pool().close().await; + }); +} + struct Fixture { _temporary: tempfile::TempDir, database: Database, diff --git a/src-tauri/src/db/providers.rs b/src-tauri/src/db/providers.rs index 61cc4f9..e7a936a 100644 --- a/src-tauri/src/db/providers.rs +++ b/src-tauri/src/db/providers.rs @@ -60,6 +60,60 @@ pub fn credential_key(profile_id: Uuid) -> String { format!("textbooklens/{profile_id}") } +pub(crate) async fn load_local_port(pool: &SqlitePool, profile_id: Uuid) -> AppResult { + let port: Option = sqlx::query_scalar("SELECT local_port FROM provider_profiles WHERE id = ? AND provider_kind IN ('ollama','lm_studio')") + .bind(profile_id.to_string()).fetch_optional(pool).await?.flatten(); + port.and_then(|port| u16::try_from(port).ok()) + .filter(|port| *port > 0) + .ok_or_else(|| AppError::new(AppErrorCode::DatabaseError)) +} + +pub(crate) async fn save_local_models( + pool: &SqlitePool, + _guard: &ProviderMutationGuard, + kind: ProviderKind, + port: u16, + models: &[crate::ai::local::LocalModel], +) -> AppResult> { + if !kind.is_local() || port == 0 { + return Err(AppError::new(AppErrorCode::InvalidInput)); + } + let mut transaction = pool.begin().await?; + let now = Utc::now().to_rfc3339(); + let mut profiles = Vec::new(); + for model in models { + let name: String = format!("{} · {}", kind.display_name(), model.id) + .chars() + .take(80) + .collect(); + sqlx::query("INSERT INTO provider_profiles (id,provider_kind,display_name,model_id,context_window_tokens,is_active,created_at,updated_at,validated_at,local_port,local_vision) VALUES (?,?,?,?,?,0,?,?,?,?,?) ON CONFLICT(provider_kind,local_port,model_id) WHERE local_port IS NOT NULL DO UPDATE SET context_window_tokens=excluded.context_window_tokens,updated_at=excluded.updated_at,validated_at=excluded.validated_at,local_vision=excluded.local_vision") + .bind(Uuid::new_v4().to_string()).bind(provider_kind_name(&kind)).bind(name).bind(&model.id).bind(model.context) + .bind(&now).bind(&now).bind(&now).bind(port as i64).bind(model.vision).execute(&mut *transaction).await?; + let row = sqlx::query("SELECT id,provider_kind,display_name,model_id,context_window_tokens,is_active,validated_at,kimi_api_region FROM provider_profiles WHERE provider_kind=? AND local_port=? AND model_id=?") + .bind(provider_kind_name(&kind)).bind(port as i64).bind(&model.id).fetch_one(&mut *transaction).await?; + let profile = profile_summary_from_row(&row, CredentialStatus::NotRequired)?; + for category in consent_categories() { + sqlx::query("INSERT OR IGNORE INTO provider_operation_consents (profile_id,category,decision,updated_at) VALUES (?,?,'ask',?)") + .bind(profile.id.to_string()).bind(consent_category_name(category)).bind(&now).execute(&mut *transaction).await?; + } + profiles.push(profile); + } + transaction.commit().await?; + Ok(profiles) +} + +pub(crate) async fn select_local_default( + pool: &SqlitePool, + _guard: &ProviderMutationGuard, + id: Uuid, +) -> AppResult<()> { + load_local_port(pool, id).await?; + let mut transaction = pool.begin().await?; + set_learning_default(&mut transaction, Some(id)).await?; + transaction.commit().await?; + Ok(()) +} + async fn provider_mutation_lock(pool: &SqlitePool) -> AppResult>> { let pool_identity: String = sqlx::query_scalar("SELECT file FROM pragma_database_list WHERE name = 'main'") @@ -114,7 +168,9 @@ pub async fn list_provider_profiles( for row in rows { let id = parse_uuid(row.try_get("id")?)?; let derived_key = credential_key(id); - let credential_status = if store.get(&derived_key).await.is_ok() { + let credential_status = if parse_provider_kind(row.try_get("provider_kind")?)?.is_local() { + CredentialStatus::NotRequired + } else if store.get(&derived_key).await.is_ok() { CredentialStatus::Available } else { CredentialStatus::Missing @@ -447,7 +503,7 @@ async fn delete_provider_profile_inner( profile_id: Uuid, ) -> AppResult<()> { let row = sqlx::query( - "SELECT is_active, EXISTS(SELECT 1 FROM app_settings WHERE id = 1 AND active_provider_profile_id = provider_profiles.id) AS legacy_selected, EXISTS(SELECT 1 FROM app_settings WHERE id = 1 AND default_learning_profile_id = provider_profiles.id) AS learning_selected, EXISTS(SELECT 1 FROM app_settings WHERE id = 1 AND default_vision_profile_id = provider_profiles.id) AS vision_selected FROM provider_profiles WHERE id = ?", + "SELECT provider_kind, is_active, EXISTS(SELECT 1 FROM app_settings WHERE id = 1 AND active_provider_profile_id = provider_profiles.id) AS legacy_selected, EXISTS(SELECT 1 FROM app_settings WHERE id = 1 AND default_learning_profile_id = provider_profiles.id) AS learning_selected, EXISTS(SELECT 1 FROM app_settings WHERE id = 1 AND default_vision_profile_id = provider_profiles.id) AS vision_selected FROM provider_profiles WHERE id = ?", ) .bind(profile_id.to_string()) .fetch_optional(pool) @@ -463,8 +519,13 @@ async fn delete_provider_profile_inner( let vision_selected = row.try_get::("vision_selected")?; let derived_key = credential_key(profile_id); - let old_secret = store.get(&derived_key).await?; - store.delete(&derived_key).await?; + let old_secret = if parse_provider_kind(row.try_get("provider_kind")?)?.is_local() { + None + } else { + let secret = store.get(&derived_key).await?; + store.delete(&derived_key).await?; + Some(secret) + }; let database_result: AppResult<()> = async { let mut transaction = pool.begin().await?; @@ -500,7 +561,9 @@ async fn delete_provider_profile_inner( .await; if let Err(error) = database_result { - if store.set(&derived_key, old_secret).await.is_err() { + if let Some(old_secret) = old_secret + && store.set(&derived_key, old_secret).await.is_err() + { return Err(AppError::credential_store( "credential restoration failed after provider deletion rollback; sensitive values omitted", )); @@ -555,12 +618,13 @@ async fn first_compatible_profile( operation: AiOperation, ) -> AppResult> { let rows = sqlx::query( - "SELECT id, provider_kind, model_id, validated_at FROM provider_profiles ORDER BY created_at, id", + "SELECT id, provider_kind, model_id, validated_at, local_vision FROM provider_profiles ORDER BY created_at, id", ) .fetch_all(&mut **transaction) .await?; for row in rows { let profile = ProfileCapability { + local_vision: row.try_get("local_vision")?, id: parse_uuid(row.try_get("id")?)?, kind: parse_provider_kind(row.try_get("provider_kind")?)?, model_id: row.try_get("model_id")?, @@ -577,6 +641,7 @@ async fn first_compatible_profile( } struct ProfileCapability { + local_vision: bool, id: Uuid, kind: ProviderKind, model_id: String, @@ -588,13 +653,14 @@ async fn load_profile_capability( profile_id: Uuid, ) -> AppResult { let row = sqlx::query( - "SELECT id, provider_kind, model_id, validated_at FROM provider_profiles WHERE id = ?", + "SELECT id, provider_kind, model_id, validated_at, local_vision FROM provider_profiles WHERE id = ?", ) .bind(profile_id.to_string()) .fetch_optional(pool) .await? .ok_or_else(|| AppError::new(AppErrorCode::NotFound))?; Ok(ProfileCapability { + local_vision: row.try_get("local_vision")?, id: parse_uuid(row.try_get("id")?)?, kind: parse_provider_kind(row.try_get("provider_kind")?)?, model_id: row.try_get("model_id")?, @@ -610,6 +676,13 @@ fn profile_supports_operation( profile: &ProfileCapability, operation: AiOperation, ) -> bool { + if profile.kind.is_local() { + return operation == AiOperation::TextLearning + || (operation == AiOperation::VisionLearning + && profile.kind == ProviderKind::Ollama + && profile.local_vision + && profile.validated_at.is_some()); + } match registry.operation_support(&profile.kind, &profile.model_id, operation) { CapabilitySupport::Supported => true, CapabilitySupport::Unknown => { @@ -668,6 +741,11 @@ fn profile_summary_from_row( credential_status: CredentialStatus, ) -> AppResult { let kind = parse_provider_kind(row.try_get("provider_kind")?)?; + let credential_status = if kind.is_local() { + CredentialStatus::NotRequired + } else { + credential_status + }; let kimi_api_region = row .try_get::, _>("kimi_api_region")? .map(|value| parse_kimi_api_region(&value)) @@ -732,6 +810,8 @@ fn provider_kind_name(value: &ProviderKind) -> &'static str { ProviderKind::Anthropic => "anthropic", ProviderKind::DeepSeek => "deepseek", ProviderKind::Kimi => "kimi", + ProviderKind::Ollama => "ollama", + ProviderKind::LmStudio => "lm_studio", } } @@ -742,6 +822,8 @@ fn parse_provider_kind(value: String) -> AppResult { "anthropic" => Ok(ProviderKind::Anthropic), "deepseek" => Ok(ProviderKind::DeepSeek), "kimi" => Ok(ProviderKind::Kimi), + "ollama" => Ok(ProviderKind::Ollama), + "lm_studio" => Ok(ProviderKind::LmStudio), _ => Err(AppError::new(AppErrorCode::DatabaseError)), } } diff --git a/src-tauri/src/domain/provider.rs b/src-tauri/src/domain/provider.rs index e86fcc3..216b5d3 100644 --- a/src-tauri/src/domain/provider.rs +++ b/src-tauri/src/domain/provider.rs @@ -14,6 +14,56 @@ pub enum ProviderKind { #[serde(rename = "deepseek")] DeepSeek, Kimi, + Ollama, + LmStudio, +} + +impl ProviderKind { + pub const fn is_local(&self) -> bool { + matches!(self, Self::Ollama | Self::LmStudio) + } + + pub const fn display_name(&self) -> &'static str { + match self { + Self::OpenAi => "OpenAI", + Self::Gemini => "Google Gemini", + Self::Anthropic => "Anthropic", + Self::DeepSeek => "DeepSeek", + Self::Kimi => "Kimi", + Self::Ollama => "Ollama", + Self::LmStudio => "LM Studio", + } + } +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, TS)] +#[serde(rename_all = "snake_case")] +#[ts(export_to = "provider.ts")] +pub enum LocalServiceStatus { + Connected, + NotInstalled, + Unavailable, + AuthenticationRequired, + NoModels, + NoUsableModels, +} + +#[derive(Clone, Debug, Serialize, Deserialize, TS)] +#[serde(rename_all = "camelCase")] +#[ts(export_to = "provider.ts")] +pub struct LocalServiceReport { + pub kind: ProviderKind, + pub status: LocalServiceStatus, + pub model_count: u32, +} + +#[derive(Clone, Debug, Serialize, Deserialize, TS)] +#[serde(rename_all = "camelCase")] +#[ts(export_to = "provider.ts")] +pub struct LocalModelConnectResult { + pub profiles: Vec, + pub default_profile_id: Option, + pub services: Vec, } #[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, TS)] @@ -106,6 +156,7 @@ pub struct ProviderCapabilityRegistryDto { pub enum CredentialStatus { Available, Missing, + NotRequired, } #[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, TS)] diff --git a/src-tauri/src/indexing/coordinator.rs b/src-tauri/src/indexing/coordinator.rs index 54f1ad5..786a03d 100644 --- a/src-tauri/src/indexing/coordinator.rs +++ b/src-tauri/src/indexing/coordinator.rs @@ -2447,6 +2447,8 @@ const fn provider_kind_name(value: &ProviderKind) -> &'static str { ProviderKind::Anthropic => "anthropic", ProviderKind::DeepSeek => "deepseek", ProviderKind::Kimi => "kimi", + ProviderKind::Ollama => "ollama", + ProviderKind::LmStudio => "lm_studio", } } diff --git a/src-tauri/src/learning/book_preparation.rs b/src-tauri/src/learning/book_preparation.rs index cd5990d..75a4214 100644 --- a/src-tauri/src/learning/book_preparation.rs +++ b/src-tauri/src/learning/book_preparation.rs @@ -468,9 +468,11 @@ impl BookLearningPreparationService { // Credential, consent, and other provider-adjacent state are forbidden above this point. self.require_snapshot_unchanged(&metadata, &snapshot.fingerprint) .await?; - self.sensitive_access - .require_credential(snapshot.binding.profile_id) - .await?; + if !snapshot.binding.provider_kind.is_local() { + self.sensitive_access + .require_credential(snapshot.binding.profile_id) + .await?; + } let consent = consent_snapshot( snapshot.binding.profile_id, self.sensitive_access @@ -588,10 +590,11 @@ impl BookLearningPreparationService { self.registry.discard(binding.preparation_id); return Err(error); } - if let Err(error) = self - .sensitive_access - .require_credential(binding.profile_id) - .await + if !binding.provider_kind.is_local() + && let Err(error) = self + .sensitive_access + .require_credential(binding.profile_id) + .await { self.registry.discard(binding.preparation_id); return Err(error); @@ -690,6 +693,7 @@ struct BookSummaryDraft { #[derive(Clone)] struct BookRegistryBinding { + provider_kind: ProviderKind, preparation_id: Uuid, book_id: Uuid, target: BookPreparationTarget, @@ -829,6 +833,7 @@ impl BookPreparationRegistry { .get(&preparation_id) .ok_or_else(|| AppError::new(AppErrorCode::RequestConflict))?; Ok(BookRegistryBinding { + provider_kind: entry.request.binding.provider_kind.clone(), preparation_id, book_id: entry.request.binding.book.book_id, target: entry.request.target.clone(), @@ -1170,18 +1175,11 @@ async fn load_provider_binding( { return Err(AppError::unsupported_provider_capability()); } - let provider = capabilities - .capabilities() - .iter() - .find(|candidate| candidate.kind == provider_kind) - .ok_or_else(AppError::unsupported_provider_capability)?; - let model = provider - .models - .iter() - .find(|candidate| candidate.id == model_id) - .ok_or_else(AppError::unsupported_provider_capability)?; let profile_window = u32::try_from(profile.try_get::("context_window_tokens")?) .map_err(|_| database_error())?; + let model = capabilities + .model_for_profile(&provider_kind, &model_id, profile_window) + .ok_or_else(AppError::unsupported_provider_capability)?; let context_window_tokens = profile_window.min(model.context_window_tokens); if context_window_tokens == 0 { return Err(database_error()); @@ -1199,8 +1197,8 @@ async fn load_provider_binding( Ok(ProviderBindingSnapshot { book, profile_id, + provider_display_name: provider_kind.display_name().to_owned(), provider_kind, - provider_display_name: provider.display_name.clone(), profile_display_name, model_id, model_display_name: model.display_name.clone(), @@ -1953,6 +1951,8 @@ fn parse_provider_kind(value: &str) -> AppResult { "anthropic" => Ok(ProviderKind::Anthropic), "deepseek" => Ok(ProviderKind::DeepSeek), "kimi" => Ok(ProviderKind::Kimi), + "ollama" => Ok(ProviderKind::Ollama), + "lm_studio" => Ok(ProviderKind::LmStudio), _ => Err(database_error()), } } @@ -1972,6 +1972,8 @@ fn provider_kind_name(value: &ProviderKind) -> &'static str { ProviderKind::Anthropic => "anthropic", ProviderKind::DeepSeek => "deepseek", ProviderKind::Kimi => "kimi", + ProviderKind::Ollama => "ollama", + ProviderKind::LmStudio => "lm_studio", } } diff --git a/src-tauri/src/learning/book_preparation_test.rs b/src-tauri/src/learning/book_preparation_test.rs index c43f332..476e63c 100644 --- a/src-tauri/src/learning/book_preparation_test.rs +++ b/src-tauri/src/learning/book_preparation_test.rs @@ -47,6 +47,35 @@ const FIXTURE_TIME: &str = "2026-08-06T00:00:00.000Z"; const QUESTION: &str = "How does orbital flux determine the synthetic example?"; const TEACHING_SENTINEL: &str = "Teach with one bounded counterexample."; +#[test] +fn local_model_book_questions_prepare_and_consume_without_credentials() { + let fixture = Fixture::new(); + tauri::async_runtime::block_on(async { + sqlx::query("UPDATE provider_profiles SET provider_kind='ollama',model_id='synthetic:small',context_window_tokens=8192,local_port=11434 WHERE id=?") + .bind(fixture.profile_id.to_string()).execute(fixture.pool()).await.unwrap(); + let book = seed_book(fixture.pool(), BookFormat::Pdf, "local").await; + fixture.access.reset_counts(); + let summary = fixture + .service + .prepare(PrepareBookLearningRequestMetadata::New { + book_id: book.book_id, + question: QUESTION.to_owned(), + }) + .await + .unwrap(); + let prepared = fixture + .service + .consume_for_execution(summary.preparation_id) + .await + .unwrap(); + assert_eq!( + prepared.provider_kind(), + &crate::domain::ProviderKind::Ollama + ); + assert_eq!(fixture.access.credential_calls(), 0); + }); +} + #[test] fn new_book_questions_are_deterministic_private_and_local_for_all_formats() { let fixture = Fixture::new(); diff --git a/src-tauri/src/learning/history.rs b/src-tauri/src/learning/history.rs index bfaa8b7..732701a 100644 --- a/src-tauri/src/learning/history.rs +++ b/src-tauri/src/learning/history.rs @@ -78,15 +78,11 @@ pub async fn prepare_conversation_followup( .ok_or_else(|| AppError::new(AppErrorCode::NotFound))?; let profile = providers::load_provider_profile_metadata(pool, profile_id).await?; let model = capabilities - .capabilities() - .iter() - .find(|provider| provider.kind == profile.kind) - .and_then(|provider| { - provider - .models - .iter() - .find(|model| model.id == profile.model_id) - }) + .model_for_profile( + &profile.kind, + &profile.model_id, + profile.context_window_tokens, + ) .ok_or_else(AppError::unsupported_provider_capability)?; if capabilities.operation_support(&profile.kind, &profile.model_id, AiOperation::TextLearning) != crate::domain::CapabilitySupport::Supported diff --git a/src-tauri/src/learning/preparation.rs b/src-tauri/src/learning/preparation.rs index a267d45..eeac298 100644 --- a/src-tauri/src/learning/preparation.rs +++ b/src-tauri/src/learning/preparation.rs @@ -19,8 +19,8 @@ use crate::{ credentials::CredentialStore, db::{providers, settings, teaching}, domain::{ - AiOperation, CapabilitySupport, ContentAnchor, ContextMode, DocumentLocator, - NormalizedRect, ProviderKind, ProviderOperationConsent, ProviderOperationConsentCategory, + AiOperation, ContentAnchor, ContextMode, DocumentLocator, NormalizedRect, ProviderKind, + ProviderOperationConsent, ProviderOperationConsentCategory, ProviderOperationConsentDecision, RegionAnchor, RegionLocator, TeachingInstructionDto, UiLanguage, }, @@ -464,11 +464,16 @@ impl LearningPreparationService { }, ) .await?; - let budget = InputBudget::new( + let mut budget = InputBudget::new( initial_binding.context_window_tokens, initial_binding.context_mode.clone(), initial_binding.default_max_output_tokens, ); + if initial_binding.provider_kind.is_local() && metadata.content_kind.will_send_image() { + // A bounded single image still consumes visual tokens in addition to the text prompt. + budget.usable_input = budget.usable_input.saturating_sub(2048); + budget.protocol_reserve = budget.protocol_reserve.saturating_add(2048); + } let (prepared_prompt, packed_context) = PromptPolicy.pack_and_prepare( PromptInput { operation: metadata.action.prompt_operation(), @@ -488,9 +493,11 @@ impl LearningPreparationService { // Everything below this barrier may touch credential, consent, or image/provider-adjacent // state. Mandatory retrieval and prompt packing above must succeed first. self.require_binding_unchanged(&initial_binding).await?; - self.sensitive_access - .require_credential(initial_binding.profile_id) - .await?; + if !initial_binding.provider_kind.is_local() { + self.sensitive_access + .require_credential(initial_binding.profile_id) + .await?; + } let consent = consent_snapshot( initial_binding.profile_id, self.sensitive_access @@ -631,10 +638,11 @@ impl LearningPreparationService { self.registry.discard(binding.preparation_id); return Err(error); } - if let Err(error) = self - .sensitive_access - .require_credential(binding.provider.profile_id) - .await + if !binding.provider.provider_kind.is_local() + && let Err(error) = self + .sensitive_access + .require_credential(binding.provider.profile_id) + .await { self.registry.discard(binding.preparation_id); return Err(error); @@ -1328,21 +1336,10 @@ async fn load_binding( } else { AiOperation::TextLearning }; - if capabilities.operation_support(&profile.kind, &profile.model_id, operation) - != CapabilitySupport::Supported - { + if !crate::db::local_capabilities::supports(pool, capabilities, &profile, operation).await? { return Err(AppError::unsupported_provider_capability()); } - let provider = capabilities - .capabilities() - .iter() - .find(|candidate| candidate.kind == profile.kind) - .ok_or_else(AppError::unsupported_provider_capability)?; - let model = provider - .models - .iter() - .find(|candidate| candidate.id == profile.model_id) - .ok_or_else(AppError::unsupported_provider_capability)?; + let model = crate::db::local_capabilities::model(pool, capabilities, &profile).await?; let capture_limits = if requires_vision { Some(RegionCaptureLimits::from_provider( model @@ -1363,8 +1360,8 @@ async fn load_binding( book, section_id, profile_id, + provider_display_name: profile.kind.display_name().to_owned(), provider_kind: profile.kind, - provider_display_name: provider.display_name.clone(), profile_display_name: profile.display_name, model_id: profile.model_id, model_display_name: model.display_name.clone(), diff --git a/src-tauri/src/learning/preparation_test.rs b/src-tauri/src/learning/preparation_test.rs index 72ee68d..a89431e 100644 --- a/src-tauri/src/learning/preparation_test.rs +++ b/src-tauri/src/learning/preparation_test.rs @@ -38,6 +38,62 @@ use super::{ const MODEL_ID: &str = "gpt-5.6"; const SELECTED_TEXT: &str = "PREPARATION_TEXT_SENTINEL spectral beacon selection"; +#[test] +fn local_model_selection_prepares_and_consumes_without_credential_access() { + let fixture = Fixture::new(); + tauri::async_runtime::block_on(async { + sqlx::query("UPDATE provider_profiles SET provider_kind='ollama',model_id='synthetic:small',context_window_tokens=8192,local_port=11434 WHERE id=?") + .bind(fixture.profile_id.to_string()).execute(fixture.pool()).await.unwrap(); + fixture.access.set_credential_available(false); + let mut metadata = fixture.text_metadata(); + metadata.model_id = "synthetic:small".to_owned(); + let summary = fixture.service.prepare(metadata).await.unwrap(); + fixture + .service + .consume(summary.preparation_id, None) + .await + .unwrap(); + assert_eq!(fixture.access.credential_calls(), 0); + }); +} + +#[test] +fn local_vision_prepares_authorizes_and_stages_without_cloud_credentials() { + let fixture = Fixture::new(); + tauri::async_runtime::block_on(async { + sqlx::query("UPDATE provider_profiles SET provider_kind='ollama',model_id='synthetic:vision',context_window_tokens=16384,local_port=11434,local_vision=1 WHERE id=?") + .bind(fixture.profile_id.to_string()).execute(fixture.pool()).await.unwrap(); + fixture.access.set_credential_available(false); + let bytes = include_bytes!("../../../fixtures/source/vision/tiny-blue.png").to_vec(); + let mut metadata = fixture.visual_metadata(&bytes); + metadata.model_id = "synthetic:vision".into(); + let summary = fixture.service.prepare(metadata).await.unwrap(); + assert!(summary.will_send_image); + let token = fixture + .service + .authorize(summary.preparation_id, LearningAuthorizationDecision::Allow) + .await + .unwrap() + .unwrap(); + let mut capture = fixture.capture_metadata(summary.preparation_id, token, &bytes); + capture.model_id = "synthetic:vision".into(); + fixture + .service + .stage_region_capture(capture, OwnedCaptureBytes::new(bytes)) + .await + .unwrap(); + let prepared = fixture + .service + .consume(summary.preparation_id, Some(token)) + .await + .unwrap(); + assert_eq!(prepared.model_id(), "synthetic:vision"); + assert!(prepared.requires_vision()); + assert!(prepared.capture().is_some()); + assert_eq!(fixture.access.credential_calls(), 0); + }); +} + #[test] fn preparation_requires_mandatory_packing_before_sensitive_access() { let fixture = Fixture::new(); diff --git a/src-tauri/src/lib.rs b/src-tauri/src/lib.rs index b45d1b4..dd3ca2c 100644 --- a/src-tauri/src/lib.rs +++ b/src-tauri/src/lib.rs @@ -95,6 +95,7 @@ pub fn run() { commands::credentials::replace_provider_profile_credential, commands::providers::list_provider_capabilities, commands::providers::list_provider_profiles, + commands::providers::connect_local_models, commands::providers::set_active_provider_profile, commands::providers::set_default_provider_profile, commands::providers::update_provider_operation_consent, diff --git a/src-tauri/src/retrieval/budget.rs b/src-tauri/src/retrieval/budget.rs index 64225e9..fd7bd15 100644 --- a/src-tauri/src/retrieval/budget.rs +++ b/src-tauri/src/retrieval/budget.rs @@ -27,7 +27,14 @@ impl InputBudget { ContextMode::Long => LONG_CONTEXT_CAP, }; let effective_window = model_context_tokens.min(mode_cap); - let output_reserve = requested_output_tokens.max(MINIMUM_OUTPUT_RESERVE); + // Small local models can explicitly reserve less output; the cloud-sized + // floor would otherwise leave no room for any question. + let output_reserve = + if effective_window < 8_192 && requested_output_tokens <= effective_window / 4 { + requested_output_tokens + } else { + requested_output_tokens.max(MINIMUM_OUTPUT_RESERVE) + }; let usable_input = effective_window .saturating_sub(output_reserve) .saturating_sub(PROTOCOL_RESERVE); diff --git a/src-tauri/src/retrieval/budget_test.rs b/src-tauri/src/retrieval/budget_test.rs index eaad570..07ad567 100644 --- a/src-tauri/src/retrieval/budget_test.rs +++ b/src-tauri/src/retrieval/budget_test.rs @@ -27,6 +27,15 @@ fn budget_exactly_caps_standard_long_and_reserves_output_plus_framing() { assert_eq!(model_bound.usable_input, 27_390); } +#[test] +fn small_local_context_leaves_room_for_a_question_and_bounded_answer() { + let budget = InputBudget::new(4096, ContextMode::Standard, 1024); + assert_eq!(budget.output_reserve, 1024); + assert_eq!(budget.usable_input, 2560); + assert!(budget.require_fits(2560).is_ok()); + assert!(budget.require_fits(2561).is_err()); +} + #[test] fn budget_boundaries_saturate_and_exact_fit_is_accepted() { let no_input = InputBudget::new(4_607, ContextMode::Long, u32::MAX); diff --git a/src-tauri/tauri.conf.json b/src-tauri/tauri.conf.json index f2a4dac..2455847 100644 --- a/src-tauri/tauri.conf.json +++ b/src-tauri/tauri.conf.json @@ -1,7 +1,7 @@ { "$schema": "https://schema.tauri.app/config/2", "productName": "TextbookLens", - "version": "0.1.0", + "version": "0.2.0", "identifier": "dev.textbooklens.desktop", "build": { "beforeDevCommand": "npm run dev", @@ -20,7 +20,7 @@ } ], "security": { - "csp": "default-src 'self'; img-src 'self' asset: blob: data:; style-src 'self' 'unsafe-inline'; font-src 'self' data:; connect-src 'self' ipc: http://ipc.localhost; script-src 'self'; worker-src 'self' blob:; frame-src 'self' blob:" + "csp": "default-src 'self'; img-src 'self' asset: blob: data:; style-src 'self' 'unsafe-inline' blob:; font-src 'self' data: blob:; connect-src 'self' ipc: http://ipc.localhost; script-src 'self' 'wasm-unsafe-eval'; worker-src 'self' blob:; frame-src 'self' blob:" } }, "bundle": { @@ -32,6 +32,12 @@ "migrations/*.sql": "migrations/", "../LICENSES.md": "licenses/LICENSES.md", "../THIRD_PARTY_NOTICES.md": "licenses/THIRD_PARTY_NOTICES.md" + }, + "windows": { + "webviewInstallMode": { + "type": "downloadBootstrapper", + "silent": true + } } } } diff --git a/src-tauri/tauri.offline.conf.json b/src-tauri/tauri.offline.conf.json new file mode 100644 index 0000000..a0780f7 --- /dev/null +++ b/src-tauri/tauri.offline.conf.json @@ -0,0 +1,11 @@ +{ + "$schema": "https://schema.tauri.app/config/2", + "bundle": { + "windows": { + "webviewInstallMode": { + "type": "offlineInstaller", + "silent": true + } + } + } +} diff --git a/src-tauri/tests/bindings.rs b/src-tauri/tests/bindings.rs index 72b3985..7fb7d24 100644 --- a/src-tauri/tests/bindings.rs +++ b/src-tauri/tests/bindings.rs @@ -720,6 +720,7 @@ fn export_bindings() { ProviderCapability::export_all(&config).unwrap(); ProviderCapabilityRegistryDto::export_all(&config).unwrap(); CredentialStatus::export_all(&config).unwrap(); + textbooklens_lib::domain::LocalModelConnectResult::export_all(&config).unwrap(); ProviderOperationConsentCategory::export_all(&config).unwrap(); ProviderOperationConsentDecision::export_all(&config).unwrap(); ProviderOperationConsent::export_all(&config).unwrap(); diff --git a/src-tauri/tests/credentials.rs b/src-tauri/tests/credentials.rs index cd4d45e..bc422e9 100644 --- a/src-tauri/tests/credentials.rs +++ b/src-tauri/tests/credentials.rs @@ -264,12 +264,13 @@ fn windows_credential_manager_disposable_round_trip() { .set(&key, SecretString::from("disposable-smoke-value")) .await?; let stored = store.get(&key).await?; - let matches = stored.expose_secret() == "disposable-smoke-value"; - let cleanup = store.delete(&key).await; - cleanup?; - Ok::(matches) + Ok::( + stored.expose_secret() == "disposable-smoke-value", + ) }); + let cleanup = tauri::async_runtime::block_on(store.delete(&key)); + cleanup.unwrap(); assert!(result.unwrap()); } diff --git a/src-tauri/tests/database_contract.rs b/src-tauri/tests/database_contract.rs index ae88da5..779b30d 100644 --- a/src-tauri/tests/database_contract.rs +++ b/src-tauri/tests/database_contract.rs @@ -412,7 +412,7 @@ fn late_v1_migrations_upgrade_real_v12_rows_without_data_loss() { .fetch_all(reopened.pool()) .await .unwrap(); - assert_eq!(versions, (1_i64..=15).collect::>()); + assert_eq!(versions, (1_i64..=17).collect::>()); let counts: (i64, i64, i64, i64) = sqlx::query_as( "SELECT (SELECT COUNT(*) FROM books), (SELECT COUNT(*) FROM annotations), (SELECT COUNT(*) FROM book_extractions), (SELECT COUNT(*) FROM provider_remote_resources)", ) diff --git a/src-tauri/tests/destructive_boundaries.rs b/src-tauri/tests/destructive_boundaries.rs index f3cb789..2d56c38 100644 --- a/src-tauri/tests/destructive_boundaries.rs +++ b/src-tauri/tests/destructive_boundaries.rs @@ -112,6 +112,18 @@ const MIGRATION_HASHES: &[(i64, &str, &[u8], &str)] = &[ include_bytes!("../migrations/0015_kimi_api_region_binding.sql"), "55556c7e0824176396be3798b28263bc8f0441c4603cf43098c60c1742effd75", ), + ( + 16, + "0016_local_model_profiles.sql", + include_bytes!("../migrations/0016_local_model_profiles.sql"), + "997894df81856f3552f0816f2e6a0dc4e437bc66d8fb05e8a6f7337cb12def4f", + ), + ( + 17, + "0017_local_vision.sql", + include_bytes!("../migrations/0017_local_vision.sql"), + "dbf5e77fb309dc66f28c44600b5267b96aabf9522587358add877f09c5775c28", + ), ]; const BOOK_ID: &str = "00000000-0000-4000-8000-000000000101"; @@ -127,7 +139,7 @@ const PROFILE_ID: &str = "00000000-0000-4000-8000-000000000109"; type SchemaRow = (String, String, String, Option); #[test] -fn migration_files_and_embedded_history_are_immutable_through_v15() { +fn migration_files_and_embedded_history_are_immutable_through_v16() { assert_eq!(MIGRATOR.migrations.len(), MIGRATION_HASHES.len()); assert_eq!( MIGRATOR @@ -136,7 +148,7 @@ fn migration_files_and_embedded_history_are_immutable_through_v15() { .filter(|migration| migration.no_tx) .map(|migration| migration.version) .collect::>(), - vec![2, 11] + vec![2, 11, 16] ); for ((version, name, bytes, expected_hash), migration) in @@ -175,7 +187,7 @@ fn every_historical_cutoff_upgrades_to_the_exact_fresh_contract_and_reopens() { block_on(fresh_database.pool().close()); drop(fresh_database); - for cutoff in 1..=15 { + for cutoff in 1..=16 { let temporary = TempDir::new().unwrap(); let database_path = temporary.path().join(format!("cutoff-{cutoff}.sqlite3")); let pool = block_on(raw_pool(&database_path)); diff --git a/src-tauri/tests/interrupted_operations.rs b/src-tauri/tests/interrupted_operations.rs index c81ae6c..78273ba 100644 --- a/src-tauri/tests/interrupted_operations.rs +++ b/src-tauri/tests/interrupted_operations.rs @@ -340,16 +340,23 @@ async fn queued_maintenance_is_fair_and_terminal_removal_keeps_continuation_leas let continuation = registry.operation_permit(book_id).unwrap(); let writer_gate = gate.clone(); + let (writer_queued_tx, writer_queued_rx) = tokio::sync::oneshot::channel(); let (writer_ready_tx, writer_ready_rx) = tokio::sync::oneshot::channel(); let (release_writer_tx, release_writer_rx) = tokio::sync::oneshot::channel(); let writer = tokio::spawn(async move { - let exclusive = writer_gate.acquire_maintenance().await.unwrap(); + let acquisition = writer_gate.acquire_maintenance(); + tokio::pin!(acquisition); + // Polling registers the writer's ticket. A scheduler yield alone does + // not prove the writer ran before the later reader was spawned. + assert!(futures_util::poll!(acquisition.as_mut()).is_pending()); + writer_queued_tx.send(()).unwrap(); + let exclusive = acquisition.await.unwrap(); writer_ready_tx.send(()).unwrap(); release_writer_rx.await.unwrap(); drop(exclusive); }); - tokio::task::yield_now().await; + writer_queued_rx.await.unwrap(); let reader_gate = gate.clone(); let late_reader = tokio::spawn(async move { reader_gate diff --git a/src/features/import/parsers/epub-xml-compatibility.test.ts b/src/features/import/parsers/epub-xml-compatibility.test.ts index 1c66d03..541e060 100644 --- a/src/features/import/parsers/epub-xml-compatibility.test.ts +++ b/src/features/import/parsers/epub-xml-compatibility.test.ts @@ -22,7 +22,7 @@ const boundaryXml = ` describe('EPUB.js xmldom compatibility', () => { it('keeps the browser-native and forced fallback parser paths compatible', () => { - expect(xmldomPackage.version).toBe('0.8.13'); + expect(xmldomPackage.version).toBe('0.8.15'); const nativeDocument = parseWithEpubJs(boundaryXml, 'application/xml'); const fallbackDocument = parseWithEpubJs( @@ -119,7 +119,7 @@ describe('EPUB.js xmldom compatibility', () => { it('round-trips valid XML node boundaries without creating extra markup', () => { const document = new XmldomParser().parseFromString( - boundaryXml, + boundaryXml.replace(/^<\?xml[^?]*\?>\s*/u, ''), 'application/xml', ); const serialized = strictSerialize(document); @@ -137,6 +137,31 @@ describe('EPUB.js xmldom compatibility', () => { expect.arrayContaining([Node.COMMENT_NODE, Node.CDATA_SECTION_NODE]), ); }); + + it('rejects a reserved XML processing-instruction target in strict serialization', () => { + // xmldom represents an XML declaration as a PI; strict DOM serialization + // now correctly rejects the reserved target instead of treating it as an ordinary PI. + const document = new XmldomParser().parseFromString( + boundaryXml, + 'application/xml', + ); + expect(() => strictSerialize(document)).toThrowError( + /processing instruction target/iu, + ); + }); + + it('rejects invalid entity-reference names at creation and strict serialization', () => { + const document = new DOMImplementation().createDocument( + null, + 'root', + null, + ) as XMLDocument & { createEntityReference(name: string): Node }; + expect(() => document.createEntityReference('invalid strictSerialize(reference)).toThrow(); + }); }); function strictSerialize(node: Node): string { diff --git a/src/features/import/parsers/pdf-parser.ts b/src/features/import/parsers/pdf-parser.ts index 9e3474a..ba73742 100644 --- a/src/features/import/parsers/pdf-parser.ts +++ b/src/features/import/parsers/pdf-parser.ts @@ -3,6 +3,7 @@ import { GlobalWorkerOptions, } from 'pdfjs-dist/legacy/build/pdf.mjs'; import pdfWorkerSrc from 'pdfjs-dist/legacy/build/pdf.worker.mjs?url'; +import { localPdfOptions } from '../../../lib/pdf-options'; import type { TextItem } from 'pdfjs-dist/types/src/display/api'; import type { @@ -52,7 +53,7 @@ export class PdfParser implements DocumentParser { let document: Awaited['promise']> | undefined; try { - loadingTask = getDocument({ data: new Uint8Array(context.source) }); + loadingTask = getDocument(localPdfOptions(context.source)); document = await loadingTask.promise; const metadata = await document.getMetadata().catch(() => null); const info = metadata?.info as @@ -135,10 +136,7 @@ export async function inspectLocalPdfPageQuality( signal: AbortSignal, ): Promise { throwIfAborted(signal); - const loadingTask = getDocument({ - data: new Uint8Array(source), - isEvalSupported: false, - } as never); + const loadingTask = getDocument(localPdfOptions(source) as never); let document: Awaited['promise']> | undefined; try { document = await loadingTask.promise; diff --git a/src/features/indexing/pdf-page-renderer.ts b/src/features/indexing/pdf-page-renderer.ts index 9f9a8f9..50dc32c 100644 --- a/src/features/indexing/pdf-page-renderer.ts +++ b/src/features/indexing/pdf-page-renderer.ts @@ -1,4 +1,5 @@ import { getDocument } from 'pdfjs-dist/legacy/build/pdf.mjs'; +import { localPdfOptions } from '../../lib/pdf-options'; import type { RenderedPdfPageDto } from './indexing-contract'; @@ -44,10 +45,7 @@ export async function renderPdfPageLocally( if (availableTotal <= 0) throw new PdfPageRenderError('RENDER_LIMIT_EXCEEDED'); const encodedLimit = Math.min(bounded.maxEncodedBytes, availableTotal); - const loadingTask = getDocument({ - data: new Uint8Array(source), - isEvalSupported: false, - } as never); + const loadingTask = getDocument(localPdfOptions(source) as never); let document: Awaited | undefined; try { document = await loadingTask.promise; diff --git a/src/features/learning/SelectionMenu.test.tsx b/src/features/learning/SelectionMenu.test.tsx index 94cd731..08f1fb3 100644 --- a/src/features/learning/SelectionMenu.test.tsx +++ b/src/features/learning/SelectionMenu.test.tsx @@ -71,6 +71,67 @@ const labels = { afterEach(cleanup); describe('SelectionMenu', () => { + it('keeps an expanded question form inside the viewport without closing the selection', async () => { + const measure = vi + .spyOn(HTMLElement.prototype, 'getBoundingClientRect') + .mockImplementation(function (this: HTMLElement) { + return new DOMRect( + 0, + 0, + 400, + this.getAttribute('role') === 'menu' && this.querySelector('textarea') + ? 220 + : 44, + ); + }); + try { + const onClose = vi.fn(); + render( + , + ); + await userEvent.click(screen.getByRole('menuitem', { name: 'Ask' })); + const menu = screen.getByRole('menu'); + expect(Number.parseFloat(menu.style.top) + 220).toBeLessThanOrEqual( + window.innerHeight, + ); + expect(Number.parseFloat(menu.style.left) + 400).toBeLessThanOrEqual( + window.innerWidth, + ); + await userEvent.type( + screen.getByRole('textbox', { name: 'Input' }), + 'A question', + ); + expect(onClose).not.toHaveBeenCalled(); + expect(screen.getByRole('textbox', { name: 'Input' })).toHaveValue( + 'A question', + ); + } finally { + measure.mockRestore(); + } + }); it('keeps visual capture bytes alive through the StrictMode effect replay', async () => { const bytes = new Uint8Array([137, 80, 78, 71]); const release = vi.fn(() => bytes.fill(0)); diff --git a/src/features/learning/SelectionMenu.tsx b/src/features/learning/SelectionMenu.tsx index 4dc3be2..a0dcca7 100644 --- a/src/features/learning/SelectionMenu.tsx +++ b/src/features/learning/SelectionMenu.tsx @@ -1,4 +1,4 @@ -import { useEffect, useRef, useState } from 'react'; +import { useEffect, useLayoutEffect, useRef, useState } from 'react'; import { invoke } from '@tauri-apps/api/core'; import { NoteEditor, type NoteEditorLabels } from '../notes/NoteEditor'; @@ -82,6 +82,20 @@ export function SelectionMenu({ summary: Readonly; } | null>(null); + useLayoutEffect(() => { + const element = menuRef.current; + if (!element) return; + const reposition = () => + setPosition( + clampMenuPosition(snapshot.position, element.getBoundingClientRect()), + ); + reposition(); + if (typeof ResizeObserver === 'undefined') return; + const observer = new ResizeObserver(reposition); + observer.observe(element); + return () => observer.disconnect(); + }, [snapshot, activeAction]); + useEffect(() => { activeSnapshot.current = snapshot; live.current = true; diff --git a/src/features/learning/selection-state.test.ts b/src/features/learning/selection-state.test.ts index 768b7ef..1aef914 100644 --- a/src/features/learning/selection-state.test.ts +++ b/src/features/learning/selection-state.test.ts @@ -15,6 +15,34 @@ const ids = { }; describe('learning selection state', () => { + it('canonicalizes both EPUB section fields before handing a selection to Rust', () => { + const locator = { + format: 'epub' as const, + cfi: 'epubcfi(/6/38!/4/2/1:0)', + sectionId: 'spine-18', + }; + const snapshot = menuSnapshotFromText( + { + text: 'selected', + anchor: { + locator, + sectionId: 'spine-18', + quote: { exact: 'selected', prefix: '', suffix: '' }, + }, + }, + { ...ids, position: { x: 10, y: 20 } }, + ); + expect( + preparationMetadata(snapshot, 'ask', 'Question').anchor, + ).toMatchObject({ + kind: 'text', + selection: { + sectionId: ids.sectionId, + locator: { sectionId: ids.sectionId }, + }, + }); + expect(locator.sectionId).toBe('spine-18'); + }); it('freezes text and reliable-region selections into the same menu model', () => { const text = menuSnapshotFromText( { diff --git a/src/features/learning/selection-state.ts b/src/features/learning/selection-state.ts index a1148b7..6478090 100644 --- a/src/features/learning/selection-state.ts +++ b/src/features/learning/selection-state.ts @@ -97,6 +97,8 @@ export function menuSnapshotFromText( ): Readonly { const anchor = clone(selection.anchor); anchor.sectionId = input.sectionId; + if (anchor.locator.format === 'epub') + anchor.locator.sectionId = input.sectionId; return freezeSnapshot({ ...input, anchor: { kind: 'text', selection: anchor }, diff --git a/src/features/library/LibraryDetails.tsx b/src/features/library/LibraryDetails.tsx index 85744fd..71b81dc 100644 --- a/src/features/library/LibraryDetails.tsx +++ b/src/features/library/LibraryDetails.tsx @@ -40,7 +40,7 @@ export function LibraryDetails({ role="grid" style={{ overflowX: 'auto' }} > -
+
{message('library.column.title')} {message('library.column.format')} @@ -48,6 +48,7 @@ export function LibraryDetails({ {message('library.column.indexStatus')} {message('library.column.lastOpened')} + {message('library.column.actions')}
{books.map((book, index) => ( +
{ 'Import progress', 'Index status', 'Last opened', + 'Actions', ]); const row = screen.getAllByRole('row')[1]; row.focus(); diff --git a/src/features/onboarding/OnboardingFlow.test.tsx b/src/features/onboarding/OnboardingFlow.test.tsx index bbd0bbc..1b85e97 100644 --- a/src/features/onboarding/OnboardingFlow.test.tsx +++ b/src/features/onboarding/OnboardingFlow.test.tsx @@ -176,6 +176,11 @@ function providerApi( ) => Promise = async () => profile, ): ProviderApi { return { + connectLocal: async () => ({ + profiles: [], + defaultProfileId: null, + services: [], + }), listCapabilities: async () => registry, listProfiles: async () => [], getSettings: async () => settings, diff --git a/src/features/onboarding/OnboardingProviderStep.tsx b/src/features/onboarding/OnboardingProviderStep.tsx index b0b20dd..b8e4ecf 100644 --- a/src/features/onboarding/OnboardingProviderStep.tsx +++ b/src/features/onboarding/OnboardingProviderStep.tsx @@ -4,6 +4,7 @@ import { toUserError, type UserFacingError } from '../../lib/errors'; import type { BookSummary } from '../../lib/generated/book'; import type { ProviderCapabilityRegistryDto } from '../../lib/generated/provider'; import { ProviderConnectForm } from '../providers/ProviderConnectForm'; +import { LocalModelConnect } from '../providers/LocalModelConnect'; import { TauriProviderApi, type ProviderApi, @@ -76,6 +77,12 @@ export function OnboardingProviderStep({

{message('onboarding.provider')}

{message('onboarding.provider.description')}

{message('onboarding.provider.bookStatus', { title: book.title })}

+ {error ? (

{error.message}

diff --git a/src/features/providers/AiServicesPage.test.tsx b/src/features/providers/AiServicesPage.test.tsx index 92073a9..46900a4 100644 --- a/src/features/providers/AiServicesPage.test.tsx +++ b/src/features/providers/AiServicesPage.test.tsx @@ -99,6 +99,11 @@ const registry = { }; function fakeApi(): ProviderApi { return { + connectLocal: vi.fn(async () => ({ + profiles: [], + defaultProfileId: null, + services: [], + })), listCapabilities: vi.fn(async () => registry), listProfiles: vi.fn(async () => [profile]), getSettings: vi.fn(async () => ({ @@ -122,6 +127,86 @@ function fakeApi(): ProviderApi { } afterEach(() => cleanup()); +it('shows discovered local vision support and lets the user select it for images', async () => { + const api = fakeApi(); + const localProfile = { + ...profile, + kind: 'ollama' as const, + displayName: 'Local vision', + modelId: 'synthetic:vision', + credentialStatus: 'not_required' as const, + }; + vi.mocked(api.listProfiles).mockResolvedValue([localProfile]); + vi.mocked(api.listCapabilities).mockResolvedValue({ + ...registry, + providers: [ + ...registry.providers, + { + ...registry.providers[0], + kind: 'ollama', + defaultModel: localProfile.modelId, + models: [ + { + ...registry.providers[0].models[0], + id: localProfile.modelId, + imageInput: 'supported', + imageLimits: { + maxImages: 1, + maxEncodedBytesEach: 2097152n, + maxTotalEncodedBytes: 2097152n, + maxDimensionPx: 2048, + maxDecodedPixelsEach: 1048576n, + }, + }, + ], + }, + ], + }); + renderPage(api); + expect(await screen.findByText(/Vision: Supported/)).toBeVisible(); + await userEvent.click(screen.getByRole('button', { name: 'Use for vision' })); + expect(api.setDefault).toHaveBeenCalledWith( + 'vision_learning', + localProfile.id, + ); + expect( + screen.queryByRole('button', { name: 'Replace key' }), + ).not.toBeInTheDocument(); +}); + +it('adds and selects an offline model with one click without asking for a key', async () => { + const api = fakeApi(); + const localProfile = { + ...profile, + kind: 'ollama', + displayName: 'Ollama · Synthetic local', + modelId: 'synthetic-local:small', + credentialStatus: 'not_required', + }; + Object.assign(api, { + connectLocal: vi.fn(async () => { + vi.mocked(api.listProfiles).mockResolvedValue([localProfile] as never); + return { + profiles: [localProfile], + defaultProfileId: profile.id, + services: [{ kind: 'ollama', status: 'connected', modelCount: 1 }], + }; + }), + }); + renderPage(api); + await userEvent.click( + await screen.findByRole('button', { name: 'Auto-connect local AI' }), + ); + expect( + await screen.findByRole('article', { name: localProfile.displayName }), + ).toBeVisible(); + expect(screen.getByText('Offline · No API key required')).toBeVisible(); + expect(screen.getByText('Learning default')).toBeVisible(); + expect( + screen.queryByRole('button', { name: 'Replace key' }), + ).not.toBeInTheDocument(); +}); + function languageSettings(language: UiLanguage) { const value = (uiLanguage: UiLanguage) => ({ onboardingCompleted: false, diff --git a/src/features/providers/AiServicesPage.tsx b/src/features/providers/AiServicesPage.tsx index a152eee..c886352 100644 --- a/src/features/providers/AiServicesPage.tsx +++ b/src/features/providers/AiServicesPage.tsx @@ -4,6 +4,7 @@ import { useMessage } from '../../app/LanguageProvider'; import { toUserError } from '../../lib/errors'; import { ProviderConnectForm } from './ProviderConnectForm'; import { ProviderProfileRow } from './ProviderProfileRow'; +import { LocalModelConnect } from './LocalModelConnect'; import { TauriProviderApi, type ProviderApi, @@ -68,6 +69,12 @@ export function AiServicesPage({

{message('aiServices.title')}

{message('aiServices.description')}

+ dispatch({ type: busy ? 'saving' : 'done' })} + /> {state.error ? (

{message('aiServices.error')}

diff --git a/src/features/providers/LocalModelConnect.tsx b/src/features/providers/LocalModelConnect.tsx new file mode 100644 index 0000000..4eb3488 --- /dev/null +++ b/src/features/providers/LocalModelConnect.tsx @@ -0,0 +1,85 @@ +import { useRef, useState } from 'react'; +import { useMessage } from '../../app/LanguageProvider'; +import type { + LocalModelConnectResult, + LocalServiceStatus, +} from '../../lib/generated/provider'; +import type { MessageKey } from '../../lib/i18n'; +import type { ProviderApi } from './api'; + +const statusMessages: Record = { + connected: 'localModels.status.connected', + not_installed: 'localModels.status.notInstalled', + unavailable: 'localModels.status.unavailable', + authentication_required: 'localModels.status.authentication', + no_models: 'localModels.status.noModels', + no_usable_models: 'localModels.status.noUsableModels', +}; + +export function LocalModelConnect({ + api, + busy, + onConnected, + onBusyChange, +}: { + api: Pick; + busy: boolean; + onConnected(): Promise; + onBusyChange(busy: boolean): void; +}) { + const message = useMessage(); + const inFlight = useRef(false); + const [running, setRunning] = useState(false); + const [result, setResult] = useState(null); + const [failed, setFailed] = useState(false); + async function connect() { + if (inFlight.current || busy) return; + inFlight.current = true; + setRunning(true); + onBusyChange(true); + setResult(null); + setFailed(false); + try { + const result = await api.connectLocal(); + setResult(result); + if (result.defaultProfileId) await onConnected(); + } catch { + setFailed(true); + } finally { + inFlight.current = false; + setRunning(false); + onBusyChange(false); + } + } + return ( +
+

{message('localModels.title')}

+

{message('localModels.description')}

+ +
+ {running ?

{message('localModels.wait')}

: null} + {result?.defaultProfileId ? ( +

+ {message('localModels.success', { count: result.profiles.length })} +

+ ) : null} + {result?.services.map((service) => ( +

+ {service.kind === 'ollama' ? 'Ollama' : 'LM Studio'} ·{' '} + {message(statusMessages[service.status])} +

+ ))} +
+ {failed ?

{message('localModels.error')}

: null} +
+ ); +} diff --git a/src/features/providers/ProviderConnectForm.tsx b/src/features/providers/ProviderConnectForm.tsx index a063d68..90855d8 100644 --- a/src/features/providers/ProviderConnectForm.tsx +++ b/src/features/providers/ProviderConnectForm.tsx @@ -15,6 +15,8 @@ const PROVIDERS: ProviderKind[] = [ 'kimi', ]; const PROVIDER_NAMES: Record = { + ollama: 'Ollama', + lm_studio: 'LM Studio', openai: 'OpenAI', gemini: 'Google Gemini', anthropic: 'Anthropic', diff --git a/src/features/providers/ProviderProfileRow.tsx b/src/features/providers/ProviderProfileRow.tsx index 6f00901..c04dff9 100644 --- a/src/features/providers/ProviderProfileRow.tsx +++ b/src/features/providers/ProviderProfileRow.tsx @@ -26,6 +26,7 @@ interface Props { export function ProviderProfileRow(props: Props) { const message = useMessage(); const { profile, registry } = props; + const local = profile.kind === 'ollama' || profile.kind === 'lm_studio'; const [mode, setMode] = useState<'none' | 'delete' | 'replace'>('none'); const [key, setKey] = useState(''); const trigger = useRef(null); @@ -59,18 +60,31 @@ export function ProviderProfileRow(props: Props) {

{profile.displayName}

- {profile.credentialStatus === 'available' - ? message('aiServices.connected') - : message('aiServices.credentialMissing')}{' '} + + {local + ? message('localModels.offline') + : profile.credentialStatus === 'available' + ? message('aiServices.connected') + : message('aiServices.credentialMissing')} + {' '} · {profile.modelId}

{message('aiServices.capability.text')}:{' '} - {capabilityLabel(message, model?.textChat ?? 'unknown')} ·{' '} - {message('aiServices.capability.vision')}:{' '} - {capabilityLabel(message, model?.imageInput ?? 'unknown')} ·{' '} - {message('aiServices.capability.structured')}:{' '} - {capabilityLabel(message, model?.strictStructuredOutput ?? 'unknown')} + {capabilityLabel( + message, + local ? 'supported' : (model?.textChat ?? 'unknown'), + )}{' '} + · {message('aiServices.capability.vision')}:{' '} + {capabilityLabel( + message, + model?.imageInput ?? (local ? 'unsupported' : 'unknown'), + )}{' '} + · {message('aiServices.capability.structured')}:{' '} + {capabilityLabel( + message, + local ? 'unsupported' : (model?.strictStructuredOutput ?? 'unknown'), + )}

{props.learningDefault || props.visionDefault ? (

@@ -93,15 +107,18 @@ export function ProviderProfileRow(props: Props) { > {message('aiServices.useForVision')} + {!local ? ( + + ) : null} -

) : null} - + {!local ? ( + + ) : null} ); } diff --git a/src/features/providers/api.ts b/src/features/providers/api.ts index bd2f039..4e39f6e 100644 --- a/src/features/providers/api.ts +++ b/src/features/providers/api.ts @@ -3,6 +3,7 @@ import { invoke } from '@tauri-apps/api/core'; import { toUserError } from '../../lib/errors'; import type { AiOperation, + LocalModelConnectResult, ProviderCapabilityRegistryDto, ProviderKind, ProviderOperationConsentCategory, @@ -19,6 +20,7 @@ export interface SaveProviderProfileRequest { } export interface ProviderApi { + connectLocal(): Promise; listCapabilities(): Promise; listProfiles(operation?: AiOperation): Promise; getSettings(): Promise; @@ -40,6 +42,13 @@ export interface ProviderApi { } export class TauriProviderApi implements ProviderApi { + async connectLocal() { + try { + return await invoke('connect_local_models'); + } catch (error) { + throw toUserError(error); + } + } async listCapabilities() { return invoke('list_provider_capabilities'); } diff --git a/src/features/reader/ReaderController.test.ts b/src/features/reader/ReaderController.test.ts index eb7da1e..4735e1e 100644 --- a/src/features/reader/ReaderController.test.ts +++ b/src/features/reader/ReaderController.test.ts @@ -26,7 +26,9 @@ class FakeApi implements ReaderApi { readonly getReaderSettings = vi.fn(); readonly updateReaderSettings = vi.fn(); readonly saveReadingProgress = vi.fn(); - readonly listReaderSections = vi.fn(async () => []); + readonly listReaderSections = vi.fn( + async () => [], + ); readonly ensurePdfPageSections = vi.fn(async () => []); readonly searchBook = vi.fn(async () => []); readonly listAnnotationMarkers = vi.fn( @@ -58,6 +60,65 @@ function adapter(format: ReaderAdapter['format']) { } describe('ReaderController', () => { + it('does not retain an EPUB source when chapter lookup finishes after switching books', async () => { + const api = new FakeApi(); + api.current = bootstrap('epub'); + let release!: ( + sections: Awaited>, + ) => void; + api.listReaderSections.mockImplementationOnce( + () => + new Promise((resolve) => { + release = resolve; + }), + ); + const epub = adapter('epub'); + const docx = adapter('docx'); + const controller = new ReaderController(api, { + epub: () => epub, + docx: () => docx, + }); + const opening = controller.open(bookId); + await vi.waitFor(() => expect(api.listReaderSections).toHaveBeenCalled()); + api.current = bootstrap('docx'); + await controller.open('new-book'); + release([]); + await opening; + expect(epub.open).not.toHaveBeenCalled(); + expect(docx.open).toHaveBeenCalledOnce(); + expect(controller.sourceForTesting()).toBeNull(); + controller.dispose(); + }); + it('supplies persisted EPUB bindings before the rendition can emit a location', async () => { + const api = new FakeApi(); + api.current = bootstrap('epub'); + const id = '33333333-3333-4333-8333-333333333333'; + const sections = [ + { + id, + parentId: null, + ordinal: 0, + title: 'Synthetic chapter', + locator: { + format: 'epub' as const, + cfi: 'epubcfi(/6/38!/4/2)', + sectionId: id, + }, + }, + ]; + api.listReaderSections.mockResolvedValue(sections); + const epub = adapter('epub'); + const controller = new ReaderController(api, { epub: () => epub }); + await controller.open(bookId); + expect(epub.open).toHaveBeenCalledWith( + expect.objectContaining({ + kind: 'document_bytes', + sectionBindings: sections, + }), + null, + ); + controller.dispose(); + }); it('opens only the matching adapter and releases document bytes after opening', async () => { const api = new FakeApi(); const pdf = adapter('pdf'); diff --git a/src/features/reader/ReaderController.ts b/src/features/reader/ReaderController.ts index 70c950c..3399a5b 100644 --- a/src/features/reader/ReaderController.ts +++ b/src/features/reader/ReaderController.ts @@ -79,7 +79,7 @@ export class ReaderController { } this.#adapter = adapter; this.#bookId = bookId; - this.#source = + const source: ReaderSource = bootstrap.book.format === 'docx' ? { kind: 'sanitized_html', @@ -88,10 +88,14 @@ export class ReaderController { : { kind: 'document_bytes', bytes: toArrayBuffer(await this.api.readBookSource(bookId)), + ...(bootstrap.book.format === 'epub' + ? { sectionBindings: await this.api.listReaderSections(bookId) } + : {}), }; if (generation !== this.#openGeneration || adapter !== this.#adapter) return; - await adapter.open(this.#source, bootstrap.lastLocator); + this.#source = source; + await adapter.open(source, bootstrap.lastLocator); if (generation !== this.#openGeneration || adapter !== this.#adapter) return; if ( diff --git a/src/features/reader/ReaderLayout.test.tsx b/src/features/reader/ReaderLayout.test.tsx index 6db32d5..f9af1ee 100644 --- a/src/features/reader/ReaderLayout.test.tsx +++ b/src/features/reader/ReaderLayout.test.tsx @@ -23,6 +23,36 @@ afterEach(() => { }); describe('ReaderLayout', () => { + it('moves focus after a contents jump without scrolling the outer reader', async () => { + const locator = { + format: 'pdf' as const, + startPage: 10, + endPage: 10, + rectsByPage: null, + }; + render( + true} + />, + ); + const main = screen.getByRole('main', { name: '阅读内容' }); + const focus = vi.spyOn(main, 'focus'); + await userEvent.click(screen.getByRole('button', { name: '目录' })); + await userEvent.click(screen.getByRole('button', { name: 'Page ten' })); + expect(focus).toHaveBeenCalledWith({ preventScroll: true }); + expect(screen.queryByRole('dialog')).not.toBeInTheDocument(); + }); it('renders exactly the minimal toolbar contract and keeps the adapter mount stable across drawers', async () => { const user = userEvent.setup(); const readerContainerRef = createRef(); diff --git a/src/features/reader/ReaderLayout.tsx b/src/features/reader/ReaderLayout.tsx index 4c742ea..688b27b 100644 --- a/src/features/reader/ReaderLayout.tsx +++ b/src/features/reader/ReaderLayout.tsx @@ -319,7 +319,9 @@ export function ReaderLayout({ const found = (await onNavigate?.(locator)) ?? false; if (found) { setActiveLayer(null); - document.querySelector('.reader-main')?.focus(); + rootRef.current + ?.querySelector('.reader-main') + ?.focus({ preventScroll: true }); } return found; }; @@ -330,7 +332,7 @@ export function ReaderLayout({ className={`reader-layout theme-${settings?.theme ?? 'system'}`} style={style} > -
+
0 ? bounds.width : 720), + bounds?.width && bounds.width > 0 ? bounds.width : (size?.width ?? 720), height: - size?.height ?? - (bounds?.height && bounds.height > 0 ? bounds.height : 720), + bounds?.height && bounds.height > 0 + ? bounds.height + : (size?.height ?? 720), }; }; diff --git a/src/features/reader/ReaderPage.tsx b/src/features/reader/ReaderPage.tsx index 949ceae..0848a49 100644 --- a/src/features/reader/ReaderPage.tsx +++ b/src/features/reader/ReaderPage.tsx @@ -282,7 +282,7 @@ export function ReaderPage() { bookId, sectionId, profile, - position: selectionPosition(), + position: selection.position ?? selectionPosition(), }), ); }, @@ -301,7 +301,12 @@ export function ReaderPage() { }); }, onMarkersResolved: () => setPanelContent(undefined), - onFailure: (error) => setPanelContent(error.message), + onFailure: (error) => + setPanelContent( + error.code === 'FILE_CORRUPTED' + ? messageRef.current('reader.pdfRenderFailed') + : error.message, + ), }, markerLayer, ); diff --git a/src/features/reader/contracts.ts b/src/features/reader/contracts.ts index 8552559..45880c5 100644 --- a/src/features/reader/contracts.ts +++ b/src/features/reader/contracts.ts @@ -8,11 +8,22 @@ import type { } from '../../lib/generated/document'; export type ReaderSource = - | { kind: 'document_bytes'; bytes: ArrayBuffer } + | { + kind: 'document_bytes'; + bytes: ArrayBuffer; + sectionBindings?: readonly ReaderSectionBinding[]; + } | { kind: 'sanitized_html'; html: string }; +export interface ReaderSectionBinding { + id: string; + locator: DocumentLocator; +} + export interface SelectionSnapshot { text: string; + /** Outer viewport position; EPUB selections originate in a separate iframe. */ + position?: { x: number; y: number }; anchor: { locator: DocumentLocator; quote: TextQuote; diff --git a/src/features/reader/epub/EpubReaderAdapter.test.ts b/src/features/reader/epub/EpubReaderAdapter.test.ts index 0cb0382..a98ec54 100644 --- a/src/features/reader/epub/EpubReaderAdapter.test.ts +++ b/src/features/reader/epub/EpubReaderAdapter.test.ts @@ -4,6 +4,82 @@ import { EpubReaderAdapter } from './EpubReaderAdapter'; import { hashEpubRegionElement } from './epub-region-capture'; describe('EpubReaderAdapter', () => { + it('uses persisted EPUB IDs for selection and progress after empty spine entries', async () => { + const id = '33333333-3333-4333-8333-333333333333'; + const cfi = 'epubcfi(/6/38!/4/2/1:0)'; + const document = new DOMParser().parseFromString( + '

synthetic selection

', + 'text/html', + ); + const range = document.createRange(); + range.selectNodeContents(document.querySelector('p')!); + const handlers = new Map void>(); + const rendition = { + display: vi.fn(async () => {}), + on: vi.fn((name: string, handler: (...args: unknown[]) => void) => { + handlers.set(name, handler); + }), + off: vi.fn(), + destroy: vi.fn(), + annotations: { add: vi.fn() }, + hooks: { content: { register: vi.fn() } }, + }; + const book = { + open: vi.fn(async () => {}), + ready: Promise.resolve(), + renderTo: () => rendition, + getRange: async () => range, + destroy: vi.fn(), + spine: { get: () => ({ index: 18 }) }, + }; + const onSelection = vi.fn(); + const onProgress = vi.fn(); + const adapter = new EpubReaderAdapter( + window.document.body, + { + onSelection, + onProgress, + onMarkerActivate: vi.fn(), + onFailure: vi.fn(), + }, + () => book, + ); + const source = { + kind: 'document_bytes' as const, + bytes: new ArrayBuffer(1), + sectionBindings: [ + { + id, + locator: { + format: 'epub' as const, + cfi: 'epubcfi(/6/38!/4/2)', + sectionId: id, + }, + }, + ], + }; + await adapter.open(source); + handlers.get('relocated')?.({ start: { cfi } }); + expect(adapter.getProgress().locator).toEqual({ + format: 'epub', + cfi, + sectionId: id, + }); + handlers.get('selected')?.(cfi); + await vi.waitFor(() => + expect(onSelection).toHaveBeenCalledWith( + expect.objectContaining({ + anchor: expect.objectContaining({ + sectionId: id, + locator: { format: 'epub', cfi, sectionId: id }, + }), + }), + ), + ); + handlers.get('relocated')?.({ start: { cfi: 'epubcfi(/6/2!/4/2)' } }); + expect(adapter.getProgress().locator).toBeNull(); + adapter.dispose(); + }); it('opens byte data in continuous vertical flow, restores CFI, and destroys rendition/book', async () => { const destroyBook = vi.fn(); const destroyRendition = vi.fn(); diff --git a/src/features/reader/epub/EpubReaderAdapter.ts b/src/features/reader/epub/EpubReaderAdapter.ts index ff0febe..1b5757b 100644 --- a/src/features/reader/epub/EpubReaderAdapter.ts +++ b/src/features/reader/epub/EpubReaderAdapter.ts @@ -11,11 +11,21 @@ import type { RegionSelectionResult, SelectionSnapshot, AnnotationMarker, + ReaderSectionBinding, } from '../contracts'; import { groupOverlappingMarkers } from '../markers/MarkerLayer'; import { iframeClientPointToLocal } from '../region-preview-geometry'; import { recoverEpubCfi } from './epub-markers'; -import { sanitizeEpubDocument, snapshotEpubRange } from './epub-selection'; +import { + epubSelectionPosition, + sanitizeEpubDocument, + snapshotEpubRange, +} from './epub-selection'; +import { sectionIdForEpubCfi } from '../section-resolution'; +import { + rememberEpubResourceReferences, + restoreEpubResourceReferences, +} from './epub-resource-references'; import { captureEpubRegion, hashEpubRegionElement, @@ -43,7 +53,12 @@ type BookLike = { getRange(cfi: string): Promise; load?: (...args: unknown[]) => Promise; destroy(): void; - spine: { get(cfi: string): SpineSectionLike | undefined }; + spine: { + get(cfi: string): SpineSectionLike | undefined; + hooks?: { + content: { register(handler: (document: Document) => void): void }; + }; + }; locations?: { generate(size: number): Promise; percentageFromCfi(cfi: string): number; @@ -87,6 +102,7 @@ export class EpubReaderAdapter implements ReaderAdapter { #cfi = ''; #generation = 0; #locationsReady = false; + #sectionBindings: readonly ReaderSectionBinding[] | null = null; #markerCfis: string[] = []; #contents = new Map(); #regionAbort: AbortController | null = null; @@ -94,8 +110,8 @@ export class EpubReaderAdapter implements ReaderAdapter { #regionCleanup: (() => void) | null = null; #regionAttach: ((contents: EpubContentsLike) => void) | null = null; #regionCapture: RegionSelectionResult['capture'] = null; - #selected = (cfi: string) => { - void this.captureSelection(cfi); + #selected = (cfi: string, contents?: EpubContentsLike) => { + void this.captureSelection(cfi, contents); }; #relocated = (location: { start?: { cfi?: string } }) => { if (location.start?.cfi) this.#cfi = location.start.cfi; @@ -115,7 +131,7 @@ export class EpubReaderAdapter implements ReaderAdapter { private readonly container: HTMLElement, private readonly events: ReaderAdapterEvents, private readonly factory: BookFactory = () => - ePub({ replacements: 'none' }) as unknown as BookLike, + ePub({ replacements: 'blobUrl' }) as unknown as BookLike, ) {} async open( @@ -125,10 +141,15 @@ export class EpubReaderAdapter implements ReaderAdapter { if (source.kind !== 'document_bytes') throw new TypeError('EPUB reader requires in-memory document bytes'); this.dispose(); + this.#sectionBindings = source.sectionBindings ?? null; const generation = this.#generation; this.container.classList.add('epub-reader'); const book = this.factory(); this.#book = book; + book.spine.hooks?.content.register((document) => { + sanitizeEpubDocument(document); + rememberEpubResourceReferences(document); + }); await book.open(source.bytes); await book.ready; if (generation !== this.#generation || this.#book !== book) { @@ -146,6 +167,7 @@ export class EpubReaderAdapter implements ReaderAdapter { if (generation !== this.#generation || rendition !== this.#rendition) return; sanitizeEpubDocument(contents.document); + restoreEpubResourceReferences(contents.document); this.#contents.set(contents.document, contents); this.#regionAttach?.(contents); }); @@ -154,9 +176,17 @@ export class EpubReaderAdapter implements ReaderAdapter { this.container.addEventListener('keydown', this.#keyDown); this.#cfi = initial?.format === 'epub' ? initial.cfi : ''; await rendition.display(this.#cfi || undefined); - void book.locations?.generate(1024).then(() => { - if (generation === this.#generation) this.#locationsReady = true; - }); + void book.locations + ?.generate(1024) + .then(() => { + if (generation === this.#generation) { + this.#locationsReady = true; + this.events.onProgress(this.getProgress()); + } + }) + .catch(() => { + /* CFI navigation still works if a location index cannot be generated. */ + }); } getSelectionSnapshot(): SelectionSnapshot | null { return this.#selection; @@ -165,6 +195,7 @@ export class EpubReaderAdapter implements ReaderAdapter { if (locator.format !== 'epub' || !this.#rendition) return { found: false }; await this.#rendition.display(locator.cfi); this.#cfi = locator.cfi; + this.events.onProgress(this.getProgress()); return { found: true }; } async showAnnotations( @@ -250,8 +281,7 @@ export class EpubReaderAdapter implements ReaderAdapter { >; const resolver = { getRange: (cfi: string) => this.#book!.getRange(cfi), - sectionIdForCfi: (cfi: string) => - exactSectionId(this.#book, cfi) ?? '', + sectionIdForCfi: (cfi: string) => this.resolveSectionId(cfi) ?? '', }; const primary = await resolveEpubRegionAnchor(resolver, region); if (primary) { @@ -319,15 +349,17 @@ export class EpubReaderAdapter implements ReaderAdapter { this.#cfi && this.#locationsReady ? (this.#book?.locations?.percentageFromCfi(this.#cfi) ?? 0) : 0; + const sectionId = this.resolveSectionId(this.#cfi); return { fraction, - locator: this.#cfi - ? { - format: 'epub', - cfi: this.#cfi, - sectionId: sectionId(this.#book, this.#cfi), - } - : null, + locator: + this.#cfi && sectionId + ? { + format: 'epub', + cfi: this.#cfi, + sectionId, + } + : null, }; } @@ -424,10 +456,11 @@ export class EpubReaderAdapter implements ReaderAdapter { const element = pointTarget(contents, event); if (!element) return; const cfi = contents.cfiFromNode(element); - const currentSection = `spine-${contents.sectionIndex}`; + const currentSection = this.resolveSectionId(cfi); if ( !cfi || - exactSectionId(this.#book, cfi) !== currentSection || + !currentSection || + this.#book?.spine.get(cfi)?.index !== contents.sectionIndex || element.getBoundingClientRect().width <= 0 || element.getBoundingClientRect().height <= 0 ) { @@ -507,7 +540,7 @@ export class EpubReaderAdapter implements ReaderAdapter { region.locator.format !== 'epub' || currentContents !== selected.contents || !selected.element.isConnected || - exactSectionId(this.#book, selected.cfi) !== selected.sectionId || + this.resolveSectionId(selected.cfi) !== selected.sectionId || currentHash !== region.contentSha256 ) { throw new EpubRegionSelectionError('epub_region_content_changed'); @@ -610,22 +643,38 @@ export class EpubReaderAdapter implements ReaderAdapter { this.#book = null; this.#selection = null; this.#locationsReady = false; + this.#sectionBindings = null; this.#markerCfis = []; this.#contents.clear(); this.#regionAttach = null; this.container.replaceChildren(); this.container.classList.remove('epub-reader'); } - private async captureSelection(cfi: string): Promise { + private resolveSectionId(cfi: string): string | null { + return this.#sectionBindings === null + ? exactSectionId(this.#book, cfi) + : (sectionIdForEpubCfi(this.#sectionBindings, cfi) ?? null); + } + + private async captureSelection( + cfi: string, + contents?: EpubContentsLike, + ): Promise { const generation = this.#generation; const book = this.#book; if (!book) return; - const range = await book.getRange(cfi); + const selection = contents?.document.defaultView?.getSelection(); + const range = selection?.rangeCount + ? selection.getRangeAt(0).cloneRange() + : await book.getRange(cfi); if (generation !== this.#generation || book !== this.#book) return; - const snapshot = snapshotEpubRange(range, sectionId(book, cfi), cfi); + const sectionId = this.resolveSectionId(cfi); + if (!sectionId) return; + const snapshot = snapshotEpubRange(range, sectionId, cfi); if (!snapshot) return; this.#selection = { text: snapshot.text, + position: epubSelectionPosition(range), anchor: { locator: { format: 'epub', cfi, sectionId: snapshot.sectionId }, quote: snapshot.quote, @@ -635,9 +684,6 @@ export class EpubReaderAdapter implements ReaderAdapter { this.events.onSelection(this.#selection); } } -function sectionId(book: BookLike | null, cfi: string): string { - return `spine-${book?.spine.get(cfi)?.index ?? 0}`; -} function exactSectionId(book: BookLike | null, cfi: string): string | null { const section = book?.spine.get(cfi); return section ? `spine-${section.index}` : null; diff --git a/src/features/reader/epub/epub-region-capture.ts b/src/features/reader/epub/epub-region-capture.ts index e74bca5..b23ff84 100644 --- a/src/features/reader/epub/epub-region-capture.ts +++ b/src/features/reader/epub/epub-region-capture.ts @@ -5,6 +5,7 @@ import type { TextQuote, } from '../../../lib/generated/document'; import { epubRegionContainer } from './epub-region-selection'; +import { canonicalEpubAttribute } from './epub-resource-references'; export const EPUB_REGION_CAPTURE_LIMITS = Object.freeze({ maxScale: 2, @@ -471,7 +472,10 @@ function appendCanonical(node: Node, parts: string[]): void { ) .filter((attribute) => !attribute.name.toLowerCase().startsWith('on')) .sort((a, b) => a.name.localeCompare(b.name)) - .map((attribute) => `${attribute.name}=${attribute.value.normalize('NFC')}`) + .map( + (attribute) => + `${attribute.name}=${canonicalEpubAttribute(element, attribute).normalize('NFC')}`, + ) .join(';'); parts.push(`<${element.tagName.toLowerCase()}|${attributes}>`); } diff --git a/src/features/reader/epub/epub-resource-references.test.ts b/src/features/reader/epub/epub-resource-references.test.ts new file mode 100644 index 0000000..d70e9a3 --- /dev/null +++ b/src/features/reader/epub/epub-resource-references.test.ts @@ -0,0 +1,58 @@ +import { expect, it } from 'vitest'; +import { hashEpubRegionElement } from './epub-region-capture'; +import { + rememberEpubResourceReferences, + restoreEpubResourceReferences, +} from './epub-resource-references'; + +it('keeps a region hash stable across blob URL replacement and reopening', async () => { + const source = new DOMParser().parseFromString( + '
Synthetic plot
A local diagram
', + 'text/html', + ); + rememberEpubResourceReferences(source); + const expected = await hashEpubRegionElement(source.querySelector('figure')!); + for (const url of ['blob:synthetic-first', 'blob:synthetic-reopened']) { + const rendered = new DOMParser().parseFromString( + source.documentElement.outerHTML.replace( + 'src="../Images/plot.png"', + `src="${url}"`, + ), + 'text/html', + ); + restoreEpubResourceReferences(rendered); + expect(rendered.querySelector('img')!.getAttribute('src')).toBe(url); + expect(await hashEpubRegionElement(rendered.querySelector('figure')!)).toBe( + expected, + ); + expect( + rendered.querySelector('[data-textbooklens-resource-references]'), + ).toBeNull(); + rendered.querySelector('img')!.setAttribute('src', 'blob:changed-image'); + expect( + await hashEpubRegionElement(rendered.querySelector('figure')!), + ).not.toBe(expected); + } + source.querySelector('img')!.setAttribute('src', '../Images/different.png'); + rememberEpubResourceReferences(source); + expect(await hashEpubRegionElement(source.querySelector('figure')!)).not.toBe( + expected, + ); +}); + +it('overwrites untrusted book markers and safely ignores malformed rendered markers', async () => { + const document = new DOMParser().parseFromString( + '', + 'text/html', + ); + rememberEpubResourceReferences(document); + expect( + document + .querySelector('img')! + .getAttribute('data-textbooklens-resource-references'), + ).not.toBe('fake'); + document + .querySelector('img')! + .setAttribute('data-textbooklens-resource-references', 'not-valid'); + expect(() => restoreEpubResourceReferences(document)).not.toThrow(); +}); diff --git a/src/features/reader/epub/epub-resource-references.ts b/src/features/reader/epub/epub-resource-references.ts new file mode 100644 index 0000000..43b451c --- /dev/null +++ b/src/features/reader/epub/epub-resource-references.ts @@ -0,0 +1,98 @@ +// Blob URLs change between openings. Keep archive-relative resource references +// private to each DOM element so persisted region hashes remain stable. +const references = new WeakMap< + Element, + ReadonlyMap +>(); +const marker = 'data-textbooklens-resource-references'; +const resourceAttributes = new Set([ + 'src', + 'srcset', + 'href', + 'xlink:href', + 'poster', +]); + +/** Called on the archive document before EPUB.js substitutes resource URLs. */ +export function rememberEpubResourceReferences(document: Document): void { + for (const element of document.querySelectorAll('*')) { + // Never trust a marker supplied by the book itself. + element.removeAttribute(marker); + const entries = [...element.attributes] + .filter( + (attribute) => + resourceAttributes.has(attribute.name.toLowerCase()) && + isArchivePath(attribute.value), + ) + .map( + (attribute) => [attribute.name.toLowerCase(), attribute.value] as const, + ); + if (entries.length === 0) continue; + references.set( + element, + new Map( + entries.map(([name, value]) => [ + name, + { archive: value, rendered: value }, + ]), + ), + ); + // Encoding prevents the library's HTML string replacement from replacing + // these original paths along with the visible resource attributes. + element.setAttribute( + marker, + btoa(encodeURIComponent(JSON.stringify(entries))), + ); + } +} + +/** Called only on rendered documents produced by the preparation hook above. */ +export function restoreEpubResourceReferences(document: Document): void { + for (const element of document.querySelectorAll(`[${marker}]`)) { + const encoded = element.getAttribute(marker)!; + element.removeAttribute(marker); + if (encoded.length > 65_536) continue; + try { + const entries: unknown = JSON.parse(decodeURIComponent(atob(encoded))); + if (!Array.isArray(entries)) continue; + const valid = entries.filter( + (entry): entry is [string, string] => + Array.isArray(entry) && + entry.length === 2 && + typeof entry[0] === 'string' && + resourceAttributes.has(entry[0]) && + typeof entry[1] === 'string' && + isArchivePath(entry[1]), + ); + references.set( + element, + new Map( + valid.map(([name, value]) => [ + name, + { archive: value, rendered: element.getAttribute(name) ?? '' }, + ]), + ), + ); + } catch { + /* Invalid metadata cannot establish a stable resource identity. */ + } + } +} + +export function canonicalEpubAttribute( + element: Element, + attribute: Attr, +): string { + const reference = references.get(element)?.get(attribute.name.toLowerCase()); + return reference?.rendered === attribute.value + ? reference.archive + : attribute.value; +} + +function isArchivePath(value: string): boolean { + return ( + value.length > 0 && + value.length <= 8_192 && + !/^(?:[a-z][a-z\d+.-]*:|\/\/|#)/iu.test(value.trim()) + ); +} diff --git a/src/features/reader/epub/epub-selection.test.ts b/src/features/reader/epub/epub-selection.test.ts index df300e5..fd972f3 100644 --- a/src/features/reader/epub/epub-selection.test.ts +++ b/src/features/reader/epub/epub-selection.test.ts @@ -3,6 +3,19 @@ import { sanitizeEpubDocument, snapshotEpubRange } from './epub-selection'; import { recoverEpubCfi } from './epub-markers'; describe('EPUB selection security and CFI recovery', () => { + it('captures the actual occurrence when a selected word repeats in a chapter', () => { + const document = new DOMParser().parseFromString( + '

first eigenvalue here

second eigenvalue there

', + 'text/html', + ); + const node = document.querySelectorAll('p')[1]!.firstChild!; + const range = document.createRange(); + range.setStart(node, 7); + range.setEnd(node, 17); + expect( + snapshotEpubRange(range, 'section', 'epubcfi(/6/38!/4/4/1:7)')?.quote, + ).toMatchObject({ exact: 'eigenvalue', suffix: ' there' }); + }); it('creates an immutable CFI snapshot and strips executable/external content', () => { const document = new DOMParser().parseFromString( '

hello world

bad', @@ -34,4 +47,26 @@ describe('EPUB selection security and CFI recovery', () => { ), ).toBe('epubcfi(/6/4)'); }); + + it('confines restored archive assets and rejects remote URLs and refresh navigation', () => { + const document = new DOMParser().parseFromString( + 'link', + 'text/html', + ); + sanitizeEpubDocument(document); + expect(document.querySelector('meta[http-equiv="refresh"]')).toBeNull(); + expect( + document.querySelectorAll('meta[http-equiv="Content-Security-Policy"]'), + ).toHaveLength(1); + expect( + document + .querySelector('meta[data-textbooklens-csp]') + ?.getAttribute('content'), + ).toContain("connect-src 'none'"); + expect(document.querySelector('img')?.hasAttribute('src')).toBe(false); + expect(document.querySelectorAll('img')[1]?.getAttribute('src')).toBe( + 'blob:http://localhost/synthetic-image', + ); + expect(document.querySelector('a')?.hasAttribute('href')).toBe(false); + }); }); diff --git a/src/features/reader/epub/epub-selection.ts b/src/features/reader/epub/epub-selection.ts index 97757b4..ad2ef7a 100644 --- a/src/features/reader/epub/epub-selection.ts +++ b/src/features/reader/epub/epub-selection.ts @@ -8,23 +8,41 @@ export function snapshotEpubRange( ): { text: string; sectionId: string; cfi: string; quote: TextQuote } | null { const text = range.toString(); if (!text || !cfi) return null; - const root = - range.commonAncestorContainer.nodeType === Node.ELEMENT_NODE - ? (range.commonAncestorContainer as Element) - : range.commonAncestorContainer.parentElement; - const sectionText = root?.ownerDocument?.body.textContent ?? text; - const start = sectionText.indexOf(text); + const body = range.startContainer.ownerDocument?.body; + if ( + !body || + !body.contains(range.startContainer) || + !body.contains(range.endContainer) + ) + return null; + const sectionText = body.textContent ?? ''; + const prefix = range.cloneRange(); + prefix.selectNodeContents(body); + prefix.setEnd(range.startContainer, range.startOffset); + const start = [...prefix.toString()].length; return { text, sectionId, cfi, - quote: createTextQuote( - sectionText, - start < 0 ? 0 : [...sectionText.slice(0, start)].length, - start < 0 - ? [...text].length - : [...sectionText.slice(0, start + text.length)].length, - ), + quote: createTextQuote(sectionText, start, start + [...text].length), + }; +} + +/** Translate the rendered iframe selection into the application's viewport. */ +export function epubSelectionPosition( + range: Range, +): { x: number; y: number } | undefined { + if (typeof range.getBoundingClientRect !== 'function') return undefined; + const rect = range.getBoundingClientRect(); + if (rect.width === 0 && rect.height === 0) return undefined; + const frame = range.startContainer.ownerDocument?.defaultView?.frameElement; + if (!(frame instanceof HTMLElement)) return { x: rect.left, y: rect.bottom }; + const bounds = frame.getBoundingClientRect(); + const sx = frame.offsetWidth ? bounds.width / frame.offsetWidth : 1; + const sy = frame.offsetHeight ? bounds.height / frame.offsetHeight : 1; + return { + x: bounds.left + (frame.clientLeft + rect.left) * sx, + y: bounds.top + (frame.clientTop + rect.bottom) * sy, }; } @@ -33,6 +51,27 @@ export function sanitizeEpubDocument(document: Document): void { const frame = document.defaultView?.frameElement; if (frame instanceof HTMLIFrameElement) frame.setAttribute('sandbox', 'allow-same-origin'); + // This also confines CSS imports and image/font URLs that are not ordinary + // href/src attributes. It is inserted before the archived document renders. + const head = document.head ?? document.querySelector('head'); + document.querySelectorAll('meta[http-equiv]').forEach((meta) => { + if ( + ['refresh', 'content-security-policy'].includes( + meta.getAttribute('http-equiv')!.toLowerCase(), + ) + ) + meta.remove(); + }); + if (head) { + const policy = document.createElementNS(head.namespaceURI, 'meta'); + policy.setAttribute('data-textbooklens-csp', ''); + policy.setAttribute('http-equiv', 'Content-Security-Policy'); + policy.setAttribute( + 'content', + "default-src 'none'; img-src blob: data:; style-src 'unsafe-inline' blob: data:; font-src blob: data:; script-src 'none'; connect-src 'none'; frame-src 'none'; form-action 'none'", + ); + head.prepend(policy); + } document .querySelectorAll('script,iframe,object,embed,form') .forEach((node) => node.remove()); @@ -49,5 +88,7 @@ export function sanitizeEpubDocument(document: Document): void { } function isArchiveResource(value: string): boolean { - return !/^(?:https?:|javascript:|file:|data:text\/html)/iu.test(value); + const url = value.trim(); + if (/^(?:blob:|data:image\/)/iu.test(url)) return true; + return !/^(?:[a-z][a-z\d+.-]*:|\/\/)/iu.test(url); } diff --git a/src/features/reader/pdf/PdfReaderAdapter.ts b/src/features/reader/pdf/PdfReaderAdapter.ts index d2bbdb4..6592ffa 100644 --- a/src/features/reader/pdf/PdfReaderAdapter.ts +++ b/src/features/reader/pdf/PdfReaderAdapter.ts @@ -3,6 +3,7 @@ import type { NormalizedRect, } from '../../../lib/generated/document'; import { getDocument } from 'pdfjs-dist/legacy/build/pdf.mjs'; +import { localPdfOptions } from '../../../lib/pdf-options'; import { EventBus, PDFLinkService, @@ -109,7 +110,16 @@ export class PdfReaderAdapter implements ReaderAdapter { #annotationRefreshFrame: number | null = null; #eventBus: EventBus | null = null; #scaleObserver: MutationObserver | null = null; - #onPageRendered = () => { + #onPageRendered = (event?: { error?: unknown }) => { + if (event?.error) { + this.events.onFailure({ + code: 'FILE_CORRUPTED', + message: 'This PDF page could not be displayed.', + nextStep: 'Reopen the book or try another page.', + diagnosticId: null, + }); + return; + } if (this.#annotationItems.length) this.#annotationsNeedRefresh = true; if ( !this.#annotationsNeedRefresh || @@ -734,10 +744,9 @@ function validPersistedPdfRegion( function loadPdf(bytes: ArrayBuffer): PdfLoadingTask { // The parser/viewer is deliberately data-only: PDF.js receives no URL and cannot fetch a remote source. - return getDocument({ - data: new Uint8Array(bytes), - isEvalSupported: false, - } as never) as unknown as PdfLoadingTask; + return getDocument( + localPdfOptions(bytes) as never, + ) as unknown as PdfLoadingTask; } function createPdfViewer( diff --git a/src/features/reader/section-resolution.test.ts b/src/features/reader/section-resolution.test.ts index 5c0a2b2..8b0cf66 100644 --- a/src/features/reader/section-resolution.test.ts +++ b/src/features/reader/section-resolution.test.ts @@ -12,6 +12,105 @@ const sections: ReaderSection[] = [ ]; describe('reader section resolution', () => { + it('resolves skipped EPUB spine entries by CFI instead of the normalized ordinal', () => { + const epubSections: ReaderSection[] = [12, 20, 38].map((step, ordinal) => ({ + id: `00000000-0000-4000-8000-00000000000${ordinal + 1}`, + parentId: null, + ordinal, + title: `Synthetic chapter ${ordinal}`, + locator: { + format: 'epub', + cfi: `epubcfi(/6/${step}[chapter-${ordinal}]!/4/2)`, + sectionId: `00000000-0000-4000-8000-00000000000${ordinal + 1}`, + }, + })); + const locator = { + format: 'epub' as const, + cfi: 'epubcfi(/6/38[chapter-2]!/4/94,/1:0,/1:9)', + sectionId: 'spine-18', + }; + expect( + sectionIdForTextSelection( + { + text: 'selection', + anchor: { + locator, + sectionId: 'spine-18', + quote: { exact: 'selection', prefix: '', suffix: '' }, + }, + }, + epubSections, + ), + ).toBe(epubSections[2]!.id); + expect( + sectionIdForRegionSelection( + { + sectionId: 'spine-18', + cfi: locator.cfi, + rect: { x: 0, y: 0, width: 0.5, height: 0.5 }, + text: 'selection', + capture: null, + anchor: { + kind: 'region', + region: { + locator, + rect: { x: 0, y: 0, width: 0.5, height: 0.5 }, + contentSha256: 'a'.repeat(64), + textFallback: null, + }, + }, + }, + epubSections, + ), + ).toBe(epubSections[2]!.id); + // Normalized ordinal zero is not the unindexed image-only spine zero. + expect( + sectionIdForTextSelection( + { + text: 'selection', + anchor: { + locator: { + ...locator, + cfi: 'epubcfi(/6/2!/4/2)', + sectionId: 'spine-0', + }, + sectionId: 'spine-0', + quote: { exact: 'selection', prefix: '', suffix: '' }, + }, + }, + epubSections, + ), + ).toBeUndefined(); + }); + + it('does not accept an EPUB section ID that contradicts its CFI', () => { + const id = '33333333-3333-4333-8333-333333333333'; + const epub: ReaderSection = { + id, + parentId: null, + ordinal: 0, + title: 'Synthetic', + locator: { format: 'epub', cfi: 'epubcfi(/6/12!/4/2)', sectionId: id }, + }; + expect( + sectionIdForTextSelection( + { + text: 'selection', + anchor: { + sectionId: id, + locator: { + format: 'epub', + cfi: 'epubcfi(/6/38!/4/2)', + sectionId: id, + }, + quote: { exact: 'selection', prefix: '', suffix: '' }, + }, + }, + [epub], + ), + ).toBeUndefined(); + }); + it('maps PDF text and region selections to the section owning their page', () => { expect( sectionIdForTextSelection( @@ -82,7 +181,7 @@ describe('reader section resolution', () => { title: 'EPUB section', locator: { format: 'epub' as const, - cfi: 'epubcfi(/6/6)', + cfi: 'epubcfi(/6/6!/4/2)', sectionId: '33333333-3333-4333-8333-333333333333', }, }; diff --git a/src/features/reader/section-resolution.ts b/src/features/reader/section-resolution.ts index 2fe556c..9ae3b85 100644 --- a/src/features/reader/section-resolution.ts +++ b/src/features/reader/section-resolution.ts @@ -1,3 +1,4 @@ +import { EpubCFI } from 'epubjs'; import type { ReaderSection } from './api'; import type { RegionSelectionResult, SelectionSnapshot } from './contracts'; @@ -6,6 +7,9 @@ export function sectionIdForTextSelection( sections: readonly ReaderSection[], ): string | undefined { const supplied = selection.anchor.sectionId; + if (selection.anchor.locator.format === 'epub') { + return resolveEpubSection(sections, selection.anchor.locator.cfi, supplied); + } if (supplied && sections.some((section) => section.id === supplied)) { return supplied; } @@ -13,9 +17,6 @@ export function sectionIdForTextSelection( if (locator.format === 'pdf') { return sectionIdForPdfRange(sections, locator.startPage, locator.endPage); } - if (locator.format === 'epub') { - return sectionIdForEpubRendition(sections, supplied); - } return locator.format === 'docx' ? sectionIdForDocxBlock(sections, locator.startBlockId) : undefined; @@ -25,6 +26,16 @@ export function sectionIdForRegionSelection( region: Readonly, sections: readonly ReaderSection[], ): string | undefined { + if ( + region.anchor.kind === 'region' && + region.anchor.region.locator.format === 'epub' + ) { + return resolveEpubSection( + sections, + region.anchor.region.locator.cfi, + region.sectionId, + ); + } if ( region.sectionId && sections.some((section) => section.id === region.sectionId) @@ -37,23 +48,49 @@ export function sectionIdForRegionSelection( if (locator.format === 'pdf') { return sectionIdForPdfRange(sections, locator.page, locator.page); } - if (locator.format === 'epub') { - return sectionIdForEpubRendition(sections, region.sectionId); - } - return sectionIdForDocxBlock(sections, locator.blockId); + return locator.format === 'docx' + ? sectionIdForDocxBlock(sections, locator.blockId) + : undefined; } -function sectionIdForEpubRendition( - sections: readonly ReaderSection[], - renditionSectionId: string | null | undefined, +/** CFI identifies the original spine; import ordinals may omit image-only entries. */ +export function sectionIdForEpubCfi( + sections: readonly Pick[], + cfi: string, ): string | undefined { - const match = /^spine-(\d+)$/u.exec(renditionSectionId ?? ''); - if (!match) return undefined; - const ordinal = Number(match[1]); - return sections.find( + const spine = epubSpineIndex(cfi); + if (spine === undefined) return undefined; + const matches = sections.filter( (section) => - section.locator.format === 'epub' && section.ordinal === ordinal, - )?.id; + section.locator.format === 'epub' && + epubSpineIndex(section.locator.cfi) === spine, + ); + return matches.length === 1 ? matches[0]!.id : undefined; +} + +function resolveEpubSection( + sections: readonly ReaderSection[], + cfi: string, + supplied: string | null | undefined, +): string | undefined { + const resolved = sectionIdForEpubCfi(sections, cfi); + if (!resolved) return undefined; + if (supplied?.startsWith('spine-')) { + return supplied === `spine-${epubSpineIndex(cfi)}` ? resolved : undefined; + } + return !supplied || supplied === resolved ? resolved : undefined; +} + +function epubSpineIndex(cfi: string): number | undefined { + if (!cfi.startsWith('epubcfi(') || !cfi.endsWith(')')) return undefined; + try { + const parsed = new EpubCFI(cfi); + return Number.isSafeInteger(parsed.spinePos) && parsed.spinePos >= 0 + ? parsed.spinePos + : undefined; + } catch { + return undefined; + } } function sectionIdForDocxBlock( diff --git a/src/features/teaching/api.ts b/src/features/teaching/api.ts index 9c3dbd0..23778a3 100644 --- a/src/features/teaching/api.ts +++ b/src/features/teaching/api.ts @@ -69,7 +69,7 @@ export class TauriTeachingApi implements TeachingApi { return profiles.some( (profile) => profile.id === settings.defaultLearningProfileId && - profile.credentialStatus === 'available', + profile.credentialStatus !== 'missing', ); } catch (error) { throw toUserError(error); diff --git a/src/lib/generated/provider.ts b/src/lib/generated/provider.ts index d37c09e..d82dac2 100644 --- a/src/lib/generated/provider.ts +++ b/src/lib/generated/provider.ts @@ -4,19 +4,25 @@ export type AiOperation = "text_learning" | "vision_learning" | "structured_page export type CapabilitySupport = "supported" | "unsupported" | "unknown"; -export type CredentialStatus = "available" | "missing"; +export type CredentialStatus = "available" | "missing" | "not_required"; export type ImageLimits = { maxImages: number, maxEncodedBytesEach: bigint, maxTotalEncodedBytes: bigint, maxDimensionPx: number, maxDecodedPixelsEach: bigint, }; export type KimiApiRegion = "cn" | "international"; +export type LocalModelConnectResult = { profiles: Array, defaultProfileId: string | null, services: Array, }; + +export type LocalServiceReport = { kind: ProviderKind, status: LocalServiceStatus, modelCount: number, }; + +export type LocalServiceStatus = "connected" | "not_installed" | "unavailable" | "authentication_required" | "no_models" | "no_usable_models"; + export type ProviderCapability = { kind: ProviderKind, displayName: string, defaultModel: string, fileCapabilities: ProviderFileCapabilities, models: Array, }; export type ProviderCapabilityRegistryDto = { schemaVersion: number, providers: Array, }; export type ProviderFileCapabilities = { fileExtraction: boolean, fileOcr: boolean, maxFileBytes: bigint | null, }; -export type ProviderKind = "openai" | "gemini" | "anthropic" | "deepseek" | "kimi"; +export type ProviderKind = "openai" | "gemini" | "anthropic" | "deepseek" | "kimi" | "ollama" | "lm_studio"; export type ProviderModelCapability = { id: string, displayName: string, contextWindowTokens: number, defaultMaxOutputTokens: number, textChat: CapabilitySupport, imageInput: CapabilitySupport, nativePdfInput: CapabilitySupport, pdfInput: CapabilitySupport, strictStructuredOutput: CapabilitySupport, imageLimits: ImageLimits | null, lastVerified: string, }; diff --git a/src/lib/messages/en.ts b/src/lib/messages/en.ts index 73c835d..feb790f 100644 --- a/src/lib/messages/en.ts +++ b/src/lib/messages/en.ts @@ -1,6 +1,31 @@ import type { MessageCatalog } from '../i18n'; export const en = { + 'library.column.actions': 'Actions', + 'reader.pdfRenderFailed': + 'This PDF page could not be displayed. Reopen the book or try another page.', + 'localModels.title': 'Local AI · Offline', + 'localModels.description': + 'Already have Ollama or LM Studio and downloaded models? Start the local service, add models, and enable text answers with one click. No API key required. Ollama vision models can also be selected for image questions.', + 'localModels.connect': 'Auto-connect local AI', + 'localModels.connecting': 'Connecting local AI…', + 'localModels.wait': + 'Finding software, starting services, and testing a model. Initial loading may take a few minutes. Keep the app open.', + 'localModels.success': + 'Added or updated {count} local models and selected a default for text answers.', + 'localModels.offline': 'Offline · No API key required', + 'localModels.status.connected': 'Connected and ready for text answers', + 'localModels.status.notInstalled': 'Installed software not found', + 'localModels.status.unavailable': + 'Could not start or reach the service. Check that the software runs locally, then retry.', + 'localModels.status.authentication': + 'The local service requires authentication. Configure local access in that software, then retry.', + 'localModels.status.noModels': + 'No downloaded local text models found. Cloud models and remote devices are excluded.', + 'localModels.status.noUsableModels': + 'Models were found, but the answer test failed. Check available memory and try the model in its original app.', + 'localModels.error': + 'Local model connection did not finish. Please try again.', 'nav.onboarding': 'Get started', 'nav.library': 'Library', 'nav.settings': 'Settings', diff --git a/src/lib/messages/zh-CN.ts b/src/lib/messages/zh-CN.ts index e45c6c2..a86bcdc 100644 --- a/src/lib/messages/zh-CN.ts +++ b/src/lib/messages/zh-CN.ts @@ -1,4 +1,28 @@ export const zhCN = { + 'library.column.actions': '操作', + 'reader.pdfRenderFailed': + '此 PDF 页面无法显示，请重新打开教材或尝试其他页面。', + 'localModels.title': '本地 AI · 离线使用', + 'localModels.description': + '已安装 Ollama 或 LM Studio 并下载模型？一键启动本机服务、添加模型并用于文字问答，无需 API 密钥。支持识图的 Ollama 模型还可设为视觉服务。', + 'localModels.connect': '一键连接本地 AI', + 'localModels.connecting': '正在连接本地 AI…', + 'localModels.wait': + '正在检测软件、启动服务并测试模型。首次加载可能需要几分钟，请保持应用打开。', + 'localModels.success': + '已添加或更新 {count} 个本地模型，并设为文字问答默认模型。', + 'localModels.offline': '离线 · 无需 API 密钥', + 'localModels.status.connected': '已连接，文字问答可用', + 'localModels.status.notInstalled': '未找到已安装的软件', + 'localModels.status.unavailable': + '服务未能启动或连接，请确认软件可以在本机正常运行后重试', + 'localModels.status.authentication': + '本机服务要求身份验证，请在该软件中配置允许本机访问后重试', + 'localModels.status.noModels': + '未找到已下载的本地文字模型；云模型和远程设备模型不参与离线连接', + 'localModels.status.noUsableModels': + '发现了模型，但未能完成回答测试，请检查运行内存和模型是否能在原软件中使用', + 'localModels.error': '本地模型连接未完成，请稍后重试。', 'nav.onboarding': '开始使用', 'nav.library': '图书馆', 'nav.settings': '设置', diff --git a/src/lib/messages/zh-TW.ts b/src/lib/messages/zh-TW.ts index 97653ae..ed29fc6 100644 --- a/src/lib/messages/zh-TW.ts +++ b/src/lib/messages/zh-TW.ts @@ -1,6 +1,30 @@ import type { MessageCatalog } from '../i18n'; export const zhTW = { + 'library.column.actions': '操作', + 'reader.pdfRenderFailed': + '此 PDF 頁面無法顯示，請重新開啟教材或嘗試其他頁面。', + 'localModels.title': '本機 AI · 離線使用', + 'localModels.description': + '已安裝 Ollama 或 LM Studio 並下載模型？一鍵啟動本機服務、新增模型並用於文字問答，無需 API 金鑰。支援圖片辨識的 Ollama 模型也可設為視覺服務。', + 'localModels.connect': '一鍵連接本機 AI', + 'localModels.connecting': '正在連接本機 AI…', + 'localModels.wait': + '正在偵測軟體、啟動服務並測試模型。首次載入可能需要幾分鐘，請保持應用程式開啟。', + 'localModels.success': + '已新增或更新 {count} 個本機模型，並設為文字問答預設模型。', + 'localModels.offline': '離線 · 無需 API 金鑰', + 'localModels.status.connected': '已連接，文字問答可用', + 'localModels.status.notInstalled': '未找到已安裝的軟體', + 'localModels.status.unavailable': + '服務無法啟動或連接，請確認軟體可在本機正常執行後重試', + 'localModels.status.authentication': + '本機服務要求驗證身分，請在該軟體中設定允許本機存取後重試', + 'localModels.status.noModels': + '未找到已下載的本機文字模型；雲端模型和遠端裝置模型不參與離線連接', + 'localModels.status.noUsableModels': + '找到模型，但未能完成回答測試，請檢查記憶體和模型是否能在原軟體中使用', + 'localModels.error': '本機模型連接未完成，請稍後重試。', 'nav.onboarding': '開始使用', 'nav.library': '圖書館', 'nav.settings': '設定', diff --git a/src/lib/pdf-options.test.ts b/src/lib/pdf-options.test.ts new file mode 100644 index 0000000..1183bc7 --- /dev/null +++ b/src/lib/pdf-options.test.ts @@ -0,0 +1,24 @@ +import { describe, expect, it } from 'vitest'; +import { localPdfOptions } from './pdf-options'; + +describe('local PDF resources', () => { + it('uses bundled same-origin decoders and fonts without accepting a source URL', () => { + const source = new Uint8Array([1, 2, 3]).buffer; + const options = localPdfOptions(source); + expect([...options.data]).toEqual([1, 2, 3]); + for (const url of [ + options.wasmUrl, + options.cMapUrl, + options.standardFontDataUrl, + ]) { + expect(new URL(url).origin).toBe(window.location.origin); + expect(new URL(url).pathname).toMatch(/^\/pdfjs\//); + } + expect(options).toMatchObject({ + isEvalSupported: false, + stopAtErrors: true, + cMapPacked: true, + }); + expect(options).not.toHaveProperty('url'); + }); +}); diff --git a/src/lib/pdf-options.ts b/src/lib/pdf-options.ts new file mode 100644 index 0000000..2d82e11 --- /dev/null +++ b/src/lib/pdf-options.ts @@ -0,0 +1,13 @@ +/** Only document bytes and bundled same-origin resources reach PDF.js. */ +export function localPdfOptions(source: ArrayBuffer) { + const root = new URL(import.meta.env.BASE_URL, window.location.href); + return { + data: new Uint8Array(source), + isEvalSupported: false, + stopAtErrors: true, + wasmUrl: new URL('pdfjs/wasm/', root).href, + cMapUrl: new URL('pdfjs/cmaps/', root).href, + cMapPacked: true, + standardFontDataUrl: new URL('pdfjs/standard_fonts/', root).href, + }; +} diff --git a/src/styles/fluent.css b/src/styles/fluent.css index d2038c2..41fcf15 100644 --- a/src/styles/fluent.css +++ b/src/styles/fluent.css @@ -2,6 +2,188 @@ min-height: 100vh; } +.library-command-bar { + align-items: end; + margin-block-end: var(--space-5); +} +.library-command-bar > label { + display: grid; + gap: var(--space-1); + min-width: 0; +} +.library-command-bar input[type='search'] { + width: min(20rem, 100%); + min-height: var(--control-height); +} +.library-command-bar select { + min-height: var(--control-height); +} +.library-command-bar__filter { + margin: 0; + padding-block: var(--space-2); +} +.library-grid { + list-style: none; + margin: 0; + padding: 0; + align-items: start; +} +.library-card { + min-width: 0; + display: flex; + flex-direction: column; + gap: var(--space-2); + padding: var(--space-3); + border: var(--border-width) solid var(--color-border); + border-radius: var(--radius-medium); + background: var(--color-surface); +} +.library-card[data-selected='true'] { + border-color: var(--color-accent); +} +.library-item { + display: grid; + gap: var(--space-2); + width: 100%; + min-width: 0; + padding: var(--space-2); + text-align: start; + border-color: transparent; + background: transparent; +} +.library-item__icon { + justify-self: start; + font-size: 0.75rem; + font-weight: 700; + letter-spacing: 0.04em; + color: var(--color-accent); +} +.library-item__title { + display: -webkit-box; + -webkit-box-orient: vertical; + -webkit-line-clamp: 3; + overflow: hidden; + overflow-wrap: anywhere; + font-weight: 600; + min-height: 3em; +} +.library-item__import-status, +.library-item__selected { + font-size: 0.875rem; + color: var(--color-muted-text); +} +.library-item__status:empty { + display: none; +} +.library-details { + border: var(--border-width) solid var(--color-border); + border-radius: var(--radius-medium); + background: var(--color-surface); +} +.library-details__row, +.library-details__header { + display: grid; + grid-template-columns: + minmax(16rem, 2fr) 4rem 6rem minmax(10rem, 1fr) + 9rem 7rem; + min-width: 54rem; + align-items: center; +} +.library-details__row { + border-top: var(--border-width) solid var(--color-border); +} +.library-details__row[data-selected='true'] { + background: var(--color-surface-raised); +} +.library-details :is([role='gridcell'], [role='columnheader']) { + min-width: 0; + padding: var(--space-3); + overflow-wrap: anywhere; +} +.library-details__header { + background: var(--color-surface-raised); + font-weight: 600; +} +.library-details__title { + width: 100%; + text-align: start; + border-color: transparent; + background: transparent; + white-space: normal; +} +.reader-header { + position: sticky; + top: 0; + z-index: 4; +} +.reader-toolbar__location strong { + min-width: 0; +} +.reader-toolbar__location > span { + flex: 0 0 auto; +} +.reader-toolbar__location > span:last-child { + max-width: 10rem; +} +.reader-main > h1 { + font-size: 1.35rem; + line-height: 1.4; + margin-block: 0 var(--space-4); + overflow-wrap: anywhere; +} +.reader-drawer > header { + position: sticky; + top: 0; + z-index: 1; + background: var(--color-surface); +} +.reader-document-resize-handle[data-reader-resize-handle='e']::before { + content: ''; + position: absolute; + width: 2px; + height: 32px; + top: calc(50% - 16px); + left: 4px; + background: var(--color-border); +} +.reader-document-resize-handle[data-reader-resize-handle='s']::before { + content: ''; + position: absolute; + height: 2px; + width: 32px; + left: calc(50% - 16px); + top: 4px; + background: var(--color-border); +} +.learning-selection-menu { + z-index: 20; + display: flex; + flex-wrap: wrap; + align-items: start; + gap: var(--space-1); + width: max-content; + max-width: calc(100vw - 16px); + max-height: calc(100dvh - 16px); + box-sizing: border-box; + overflow: auto; + padding: var(--space-2); + border: var(--border-width) solid var(--color-border); + border-radius: var(--radius-medium); + background: var(--color-surface); + box-shadow: var(--elevation-1); +} +.learning-selection-menu > form { + display: grid; + gap: var(--space-2); + width: 100%; +} +.learning-selection-menu textarea { + width: 100%; + min-width: 0; + min-height: 5rem; + box-sizing: border-box; +} + .app-header { display: flex; flex-wrap: wrap; @@ -221,7 +403,7 @@ main { } .reader-toolbar { - position: sticky; + position: relative; z-index: 4; top: 0; min-width: 0; @@ -257,6 +439,7 @@ main { justify-content: center; overflow: hidden; white-space: nowrap; + gap: var(--space-2); } .reader-toolbar__location :is(strong, span:last-child) { @@ -359,6 +542,7 @@ main { .reader-document-frame { position: relative; + isolation: isolate; width: min(100%, var(--reader-width, var(--reader-column-width))); max-width: 100%; height: min(75vh, 56rem); @@ -369,6 +553,10 @@ main { border: var(--border-width) solid var(--color-border); border-radius: var(--radius-small); background: var(--color-surface); + box-sizing: border-box; + padding-inline-end: 12px; + padding-block-end: 12px; + max-height: max(15rem, calc(100dvh - 12rem)); } .reader-document { @@ -405,23 +593,23 @@ main { .reader-document-resize-handle[data-reader-resize-handle='e'] { inset-block: 0.75rem; - inset-inline-end: -0.1rem; - width: 0.7rem; + inset-inline-end: 0; + width: 10px; cursor: ew-resize; } .reader-document-resize-handle[data-reader-resize-handle='s'] { inset-inline: 0.75rem; - inset-block-end: -0.1rem; - height: 0.7rem; + inset-block-end: 0; + height: 10px; cursor: ns-resize; } .reader-document-resize-handle[data-reader-resize-handle='se'] { inset-inline-end: 0; inset-block-end: 0; - width: 1.1rem; - height: 1.1rem; + width: 12px; + height: 12px; cursor: nwse-resize; } diff --git a/vite.config.ts b/vite.config.ts index 7dee06b..1e67c12 100644 --- a/vite.config.ts +++ b/vite.config.ts @@ -12,6 +12,8 @@ export default defineConfig({ }, }, test: { + dir: '.', + include: ['src/**/*.test.{ts,tsx}', 'scripts/**/*.test.mjs'], environment: 'jsdom', setupFiles: ['./src/test/setup.ts'], exclude: ['e2e/**', 'node_modules/**', 'dist/**'],