Skip to content

Add Vision-based Image OCR converter, docs, and tests - #9

Merged
etherlink-intern merged 1 commit into
mainfrom
codex/identify-acceptable-document-types-for-ingestion
Jun 2, 2026
Merged

etherlink-intern merged 1 commit into
mainfrom
codex/identify-acceptable-document-types-for-ingestion

Conversation

@etherlink-intern

Copy link
Copy Markdown
Owner

Motivation

  • Add on-device OCR support so image inputs can be converted to Markdown using Apple Vision when available.
  • Wire image-format recognition into the format inference so image files are routed to an OCR converter instead of returning unsupportedFormat by default.
  • Provide example app wiring for an Import button so apps can easily hand picked documents or photos to the converter.

Description

  • Add ImageOCRConverter which uses conditional imports for Vision, CoreGraphics, and ImageIO to run text recognition and return a MarkdownDocument, and otherwise throws ConversionError.unsupportedFormat.
  • Extend DocumentFormat with image kinds (png, jpeg, heic, tiff, gif), add imageFormats set, and map common content-types and extensions.
  • Register ImageOCRConverter in MarkItDown.defaultConverters so images are handled by the pipeline when supported.
  • Add Docs/ImportButtonIntegration.md with SwiftUI fileImporter/PhotosPicker examples and update README.md; expand tests in SwiftMarkItDownTests to cover image format inference and conditional OCR rendering tests.

Testing

  • Ran the unit test suite with swift test, which executed SwiftMarkItDownTests (including conditional OCR tests) and completed successfully.
  • The tests include platform-conditional OCR rendering tests that exercise Vision-based recognition when the test host provides Vision/CoreGraphics/ImageIO, and they assert the expected unsupportedFormat behavior when those frameworks are unavailable.
  • Verified repository-level CLI smoke tests with Scripts/smoke-test.sh to ensure basic CLI scenarios remain unchanged and they passed.

Codex Task

@etherlink-intern
etherlink-intern marked this pull request as ready for review June 2, 2026 10:45
@etherlink-intern
etherlink-intern merged commit 59070fa into main Jun 2, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant