Skip to content

HANDOFF.md

Latest commit

 

History

History
124 lines (92 loc) · 4.64 KB

HANDOFF.md

File metadata and controls

124 lines (92 loc) · 4.64 KB
Raw

Developer Handoff

Product Direction

AI Study Notes is an iPad study notebook for handwritten learning. The goal is to combine a GoodNotes-style writing surface with an AI assistant that can understand the full page while focusing on the selected region.

The main loop is:

Write with Apple Pencil -> select a region -> ask the AI sidebar -> keep writing and organizing notes

Current Implementation

iPad App

  • SwiftUI app structure
  • iPad-only project configuration
  • PencilKit canvas wrapped in SwiftUI
  • Custom writing toolbar instead of the default PencilKit floating tool picker
  • Pen, highlighter, eraser, lasso, color, width, and preset controls
  • Page model with templates, page size, optional background image, and PencilKit drawing data
  • Local notebook persistence
  • Page thumbnail sidebar
  • Page creation, selection, duplication, deletion, reordering, and batch actions
  • PDF import using PDFKit and UIDocumentPickerViewController
  • AI selection workflow with full-page and selected-area context
  • Resizable right sidebar for notebook-level chat
  • Markdown-friendly AI response display and copy actions

Backend

The backend lives in worker/ and is designed as a Cloudflare Worker proxy. It receives notebook context and forwards multimodal requests to an AI provider. Secrets must be configured through the deployment platform and must not be committed.

Expected endpoints:

GET /health
POST /ai/chat

Required secret:

npx wrangler secret put DASHSCOPE_API_KEY

Optional non-secret model variables:

  • QWEN_FLASH_MODEL
  • QWEN_PLUS_MODEL

Local Setup

brew install xcodegen
xcodegen generate
open AIStudyNotes.xcodeproj

Run the AIStudyNotes scheme on an iPad simulator or a connected iPad.

Backend Configuration

The app reads AIBackendURL from Info.plist, generated by XcodeGen from project.yml.

For public development, keep this value empty or use a placeholder. For private deployment, set it to your own Worker base URL and regenerate the project:

AIBackendURL: "https://your-worker.example.com"
xcodegen generate

If AIBackendURL is empty, the app uses MockAIClient for local UI testing.

Important Files

  • project.yml: XcodeGen configuration and app Info.plist values
  • AIStudyNotes/AIStudyNotesApp.swift: app entry point
  • AIStudyNotes/Models/Notebook.swift: notebook model
  • AIStudyNotes/Models/NotePage.swift: page model, template, size, background, and drawing data
  • AIStudyNotes/Models/NotebookChatMessage.swift: AI chat and selection models
  • AIStudyNotes/Models/CanvasTool.swift: writing tool definitions and presets
  • AIStudyNotes/Stores/NotebookStore.swift: main app state and notebook operations
  • AIStudyNotes/Views/NotebookView.swift: notebook shell and page sidebar
  • AIStudyNotes/Views/NotePageView.swift: page canvas, toolbar, selection, and AI sidebar
  • AIStudyNotes/Views/PencilCanvasView.swift: PencilKit bridge
  • AIStudyNotes/Services/AIClient.swift: mock and remote AI clients
  • AIStudyNotes/Services/NotebookPersistence.swift: local persistence
  • AIStudyNotes/Services/PageRenderer.swift: page and selection rendering
  • worker/src/index.ts: Cloudflare Worker backend

Product Constraints

  • Keep the visible writing tools custom and compact.
  • Do not replace the main writing UI with the default PKToolPicker.
  • Keep the AI assistant in a resizable sidebar rather than in floating cards over the page.
  • Preserve the meaning of selection: the AI receives the full page and treats the selected region as the focus.
  • Keep notebook chat history scoped to the notebook.
  • Do not commit production API keys, tokens, private service URLs, or signing credentials.

Suggested Next Work

  1. Improve notebook library management for multiple notebooks.
  2. Add export flows for PDF or images.
  3. Add OCR search and handwriting-to-text experiments.
  4. Improve lasso actions for moving, scaling, and copying handwritten content.
  5. Add shape recognition and object tools such as images, stickers, and text boxes.
  6. Add optional audio recording linked to page timelines.
  7. Design a migration strategy for persisted notebooks as the model evolves.

Verification Checklist

  • xcodegen generate succeeds.
  • The AIStudyNotes scheme builds for an iPad simulator.
  • Writing remains smooth after page resize and sidebar resize.
  • PDF import creates correctly sized pages with rendered backgrounds.
  • Page thumbnails update after drawing and page operations.
  • AI selection captures both full-page and selected-area context.
  • Empty backend configuration falls back to MockAIClient.
  • Public repository scans do not show committed secrets or private endpoints.