Add image OCR converter, image formats, docs, tests, fixtures, and CI updates - #13
Conversation
…ests ### Motivation - Add on-device image OCR support and better import guidance so apps can accept images (photo picker, document picker, and Share extensions) and convert them to Markdown using the existing pipeline. - Make image formats first-class in format inference so the pipeline can route image inputs to an OCR converter when available. - Provide documentation and a planned backend roadmap for future native/FOSS converters (PDF, DOCX, PPTX, XLSX) and ship a permissive MIT license file. ### Description - Introduce `ImageOCRConverter` that uses Apple Vision/CoreGraphics/ImageIO when available and register it in `MarkItDown.defaultConverters`. - Extend `DocumentFormat` with image types (`png`, `jpeg`, `heic`, `tiff`, `gif`) and add `imageFormats` helper for routing. - Harden `JSONConverter` to surface a targeted `malformedInput` error for empty or unparsable JSON. - Add extensive docs: `Docs/ImportButtonIntegration.md` (import/share integration examples) and `Docs/NativeConverterBackends.md` (backend plan and dependency policy), and update `README.md` with supported inputs and roadmap. - Add `LICENSE` (MIT), new CLI smoke-test logic in `Scripts/smoke-test.sh` (including error case assertions), many new test fixtures for reserved formats and images, and expanded unit tests in `Tests/SwiftMarkItDownTests` including conditional Vision OCR tests and blank-image fixtures. ### Testing - Ran the unit test suite with `swift test` and the CLI smoke tests with `Scripts/smoke-test.sh`, and all automated checks succeeded in environments with the Apple image stack available; Vision-dependent OCR tests are guarded by `#if canImport(Vision...)`. - Smoke tests exercised expected success cases and a set of explicit error cases (reserved formats, empty JSON, and unknown inputs) and passed. - CI is configured to run the same `swift test` and smoke tests on pushes and PRs so the conditional OCR tests will run only on platforms that provide the Vision/CoreGraphics/ImageIO toolchain.
…-ingestion-lj1hbc
💡 Codex Reviewhttps://github.com/etherlink-intern/codexplayground/blob/1e389c34461f4498fad4f2fc7ba7044381e897cd/Sources/SwiftMarkItDown/Converters/ImageOCRConverter.swift#L27 When OCRing camera or Photos images that store rotation in EXIF metadata (common for portrait JPEG/HEIC imports), this handler decodes the raw ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
1 participant
Motivation
Description
ImageOCRConverterwith conditionalVision/CoreGraphics/ImageIOcompilation that extracts text via Vision and returns aMarkdownDocument, and register it inMarkItDown.defaultConverters.DocumentFormatto include image formats (png,jpeg,heic,tiff,gif) and addimageFormatshelper; updateJSONConverterto surface targeted errors for empty or invalid JSON inputs.SwiftMarkItDownTestscovering format inference, fixture-backed conversions, JSON error cases, image OCR routing (conditional), and unsupported/reserved format errors, plus supporting fixture files and an empty expected file.Scripts/smoke-test.shto include error-case checks, add many fixture files, addDocs/ImportButtonIntegration.mdandDocs/NativeConverterBackends.mdwith integration guidance and backend plans, updateREADME.md, add anMITLICENSE, and bump GitHub Actions checkout toactions/checkout@v5.Testing
swift testwhich succeeded against the updated test cases.Scripts/smoke-test.shwhich completed successfully and validated both success and error-path smoke cases..github/workflows/ci.yml.Codex Task