Add Nayana OCR, layout detection and Gemma-judged VQA - #9
adithya-s-k wants to merge 14 commits into
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8f6df0a742
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| with self.index_cache.lease(info["sha256"], download) as path: | ||
| with self._connection(path) as db: |
There was a problem hiding this comment.
When an index has already been downloaded, DiskCache.lease() takes the hit path and never invokes download, so the size and SHA-256 checks above are skipped. If the persistent cache is corrupted or modified between server runs, _connection can therefore open a valid-but-altered SQLite database under the original snapshot ID and silently serve different task mappings. Verify the cached file against info["sha256"] before opening it, as the local-index path already does.
Useful? React with 👍 / 👎.
| storage = json.loads(manifest.read_text()).get("storage") | ||
| env[ | ||
| "NAYANA_CORPUS_MANIFEST" | ||
| if storage == "bucket-parquet" | ||
| else "NAYANA_SNAPSHOT" | ||
| ] = str(path) |
There was a problem hiding this comment.
When snapshot is a small prepared snapshot's manifest.json, this branch correctly reads that file to detect its storage type but then assigns the file itself to NAYANA_SNAPSHOT. The child process passes that value to Catalog, which treats it as a directory and attempts to read <manifest.json>/manifest.json, so nayana-smoke --snapshot /path/to/manifest.json cannot start even though both this helper and the smoke command explicitly handle file-valued snapshots. Assign manifest.parent for the non-corpus case.
Useful? React with 👍 / 👎.
adithya-s-k
changed the title
1 participant
Adds experiment 05 with one OpenEnv environment for full-page OCR, section OCR, six-class layout detection, multiple-choice VQA, and descriptive VQA. The Gradio playground provides indexed navigation, scoring, reference reveal, and layout overlays.
The pinned 22-language Nayana corpus is served from the HuggingEnvs bucket. Per-language indexes provide stable task IDs and document-disjoint splits; bounded caches, physical-block sampling, and prefetch support local, Space, and HF Jobs access. Descriptive answers use strict Gemma grading through HF Inference Providers with DeepInfra; provider failures return errors without assigning rewards.
Includes the installable environment, Docker deployment, TRL GRPO runner, HF Jobs launchers, notebook, regression tests, and setup/reproduction documentation. The published corpus manifest is the only committed data artifact. Generated reports and checkpoints use ignored
artifacts/; CI publishes its verification reports as workflow artifacts.Validation: 55 regression tests passed from a clean export, along with HTTP/WebSocket/media smoke, lint, and repository index checks. The deployed environment also passed multilingual UI, notebook, and live judge calibration checks.
GPU optimizer training remains unverified. The 50-million-pixel image limit rejects some indexed candidates; unattended full-corpus runs need a versioned resize or eligibility policy. Source and derived data retain CognitiveLab attribution and CC BY-NC 4.0; code is Apache-2.0.