Add Image OCR converter, image format inference, docs, and expanded tests - #12
Conversation
### Motivation
- Ensure CI exercises every import surface so supported, image, empty, reserved, and unknown document paths are validated before merge.
- Surface deterministic, user-friendly errors for malformed inputs instead of leaking low-level parser/runtime errors.
- Provide app integration docs and a path toward native/FOSS backends while keeping unsupported formats visible to import UIs.
### Description
- Added image formats to `DocumentFormat` and an `imageFormats` helper, and registered `ImageOCRConverter` in `MarkItDown.defaultConverters` so image inputs can be routed to OCR when available.
- Implemented `ImageOCRConverter` behind `#if canImport(Vision) && canImport(CoreGraphics) && canImport(ImageIO)` to run Vision OCR and return a `MarkdownDocument`, and otherwise return `ConversionError.unsupportedFormat`.
- Hardened `JSONConverter` to throw `ConversionError.malformedInput("JSON input is empty.")` for empty payloads and a deterministic `"The input could not be parsed as JSON."` message for parse failures.
- Added deterministic CI fixtures (`Tests/Fixtures`) including empty files, minimal OpenXML mock packages, and a tiny PNG, updated unit tests (`Tests/SwiftMarkItDownTests`) to cover supported/reserved/empty/unknown paths, added `Scripts/smoke-test.sh` checks for both successful conversions and expected stderr messages for error cases, and added `Docs/ImportButtonIntegration.md`, `Docs/NativeConverterBackends.md`, and `LICENSE` (MIT).
### Testing
- Ran `swift test`, which passed all unit tests (Vision OCR tests run conditionally on Apple platforms and the suite passed in CI with conditional behavior).
- Ran `Scripts/smoke-test.sh`, which built the CLI and validated successful fixture conversions and expected error messages for empty JSON, reserved unsupported formats, and unknown files.
- Ran `git diff --check` with no issues found.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1ef104addc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| request.usesLanguageCorrection = true | ||
| request.minimumTextHeight = 0.01 | ||
|
|
||
| let handler = VNImageRequestHandler(cgImage: image, options: [:]) |
There was a problem hiding this comment.
For camera/photo-library JPEG or HEIC inputs whose pixels are stored with an EXIF orientation other than .up, this handler discards the orientation metadata from CGImageSource and runs Vision as if the image were upright. Those are common outputs from iPhone photos, so OCR can miss or badly order text on rotated images; read kCGImagePropertyOrientation from the image source and use the VNImageRequestHandler(cgImage:orientation:options:) initializer.
Useful? React with 👍 / 👎.
| } else if let url { | ||
| continuation.resume(returning: url) |
There was a problem hiding this comment.
In the share-extension sample, any provider that reaches loadFileRepresentation resumes with the provider's temporary file URL, but convertFile reads it only after the completion handler has returned. On iOS those temporary files are only guaranteed during the callback, so document/image shares can fail with a missing file; copy the file or read its Data inside this closure before resuming.
Useful? React with 👍 / 👎.
…-ingestion-bji288
1 participant
Motivation
Description
ImageOCRConverterthat uses Apple Vision/CoreGraphics/ImageIO when available and register it inMarkItDown.defaultConverters.DocumentFormatwith image types (png,jpeg,heic,tiff,gif) and addimageFormatshelper for routing.JSONConverterto surface a targetedmalformedInputerror for empty or unparsable JSON.Docs/ImportButtonIntegration.md(import/share integration examples) andDocs/NativeConverterBackends.md(backend plan and dependency policy), and updateREADME.mdwith supported inputs and roadmap.LICENSE(MIT), new CLI smoke-test logic inScripts/smoke-test.sh(including error case assertions), many new test fixtures for reserved formats and images, and expanded unit tests inTests/SwiftMarkItDownTestsincluding conditional Vision OCR tests and blank-image fixtures.Testing
swift testand the CLI smoke tests withScripts/smoke-test.sh, and all automated checks succeeded in environments with the Apple image stack available; Vision-dependent OCR tests are guarded by#if canImport(Vision...).swift testand smoke tests on pushes and PRs so the conditional OCR tests will run only on platforms that provide the Vision/CoreGraphics/ImageIO toolchain.Codex Task