Dataset Annotation App Browser-only YOLO detect dataset annotator with NDJSON-first export, overlap-friendly box editing, TP/FP review states, and lossless re-import.
Ultralytics supports multiple dataset/task types and many detection datasets, but the relevant scope here is detect: normalized xywh boxes, zero-indexed classes, overlapping objects like person + tie, NDJSON with a first-line dataset record plus one image record per line, and platform flows around ZIP/TAR/NDJSON upload plus NDJSON export/versioning. The supported-datasets page spans many detection taxonomies, so class presets should be an extensible registry rather than a one-off list. (Ultralytics Docs)
Goals
-
Upload multiple images in-browser.
-
Support class presets, with COCO-80 as the first built-in pack, plus custom classes.
-
Annotate images with axis-aligned boxes.
-
Support overlapping boxes as layers with explicit z-order.
-
Support box review states:
TP,FP,ignore. -
Support image review states:
complete,incomplete,negative. -
Export a zip containing:
- canonical
dataset.ndjson - images in split folders
- optional YOLO compatibility artifacts
- sidecar metadata for round-trip re-import
- canonical
-
Re-import any dataset zip generated by the tool without losing classes, annotations, colors, z-order, or TP/FP review state.
-
No backend, no server persistence, no auth.
-
Next.js + React + TypeScript, Clean Architecture, SOLID, BusinessRepo layout.
Non-goals
- No cloud sync, collaboration, or user accounts.
- No model training or inference in v1.
- No SAM or in-browser foundation-model auto-labeling in v1.
- No segmentation, pose, OBB, or classification in v1.
- No runtime scraping of Ultralytics docs.
- No “background” or “FP” exported as a training class.
The annotation UX should mirror the proven affordances in Ultralytics’ own editor: click-drag box creation, resize via 8 handles, center move, delete, z-order controls, undo/redo, class creation during annotation, and import of prior annotations. (Ultralytics Docs)
A. Create project
-
Start from:
- fresh image upload
- re-imported project zip
- raw NDJSON + images zip
-
Create a local project record in IndexedDB.
-
Assign default split policy:
trainrequired,valauto-created if missing. -
Manual split overrides per image and bulk split editing.
B. Manage classes
-
Built-in class pack registry.
-
Ship
COCO-80first because Ultralytics exposes the full class map, includingpersonandtie. (Ultralytics Docs) -
Allow:
- activate subset of preset classes
- add custom classes
- rename classes
- recolor classes
- search/filter classes
-
Unified class table applies edits across dataset.
C. Annotate
-
Layout:
- left panel: image list/grid with split + review state + counts
- center: annotation canvas
- right panel: class manager + annotation stack + properties
-
Core interactions:
- draw box by dragging
- select box
- drag 8 handles to resize
- drag center to move
- delete selected box
- multi-select boxes
- bring forward / send backward / bring to front / send to back
- zoom, pan, fit-to-screen
- keyboard shortcuts for save, undo, redo, delete, z-order, class quick-select
-
Overlap handling:
- translucent fills
- annotation stack list
- cycle selection for stacked boxes
- explicit z-order controls
-
Box attributes:
- class
- normalized coordinates
- review state:
tp | fp | ignore - locked / hidden
- z-index
- notes (optional)
D. Review
-
Each image has:
completeincompletenegative
-
Export rule:
- default is to block export for
incompleteimages - optional override with clear warning
- default is to block export for
-
Reason:
- partially annotated datasets can work, but Ultralytics recommends labeling all target objects and excluding unannotated images for best results. (Ultralytics Docs)
E. Export
-
Canonical export is NDJSON.
-
Only
TPboxes are exported into training annotations. -
FPandignoreremain in tool metadata only. -
If an image has zero exportable boxes, export it as a negative image.
-
Mirror Ultralytics split behavior:
trainrequired- if
valmissing and dataset has at least 2 images, auto-create avalsubset at roughly 10% by default, while allowing manual override in UI. (Ultralytics Docs)
F. Re-import
-
Prefer
meta/project.jsonfor UI state. -
Prefer
dataset.ndjsonfor canonical dataset content. -
Fallback path:
- raw NDJSON + images
- optional stretch: YOLO detect folder structure import
Because Platform NDJSON snapshots may contain URLs and annotations but not the image bytes themselves, this app should always include the real image files in the zip. I’m not fully certain every local-only NDJSON workflow resolves relative image files the same way when URLs are absent, so the safest interoperability choice is to ship NDJSON + images + optional YOLO txt/YAML compatibility artifacts together. (Ultralytics Docs)
dataset-annotation-app-export.zip
├── dataset.ndjson # canonical export
├── data.yaml # compatibility export
├── images/
│ ├── train/
│ ├── val/
│ └── test/ # optional
├── labels/
│ ├── train/
│ ├── val/
│ └── test/ # optional
└── meta/
└── project.json # UI-only round-trip metadata
dataset.ndjson rules
-
First line: dataset record
-
Subsequent lines: image records
-
taskmust bedetect -
class_namesmust be zero-indexed -
image records use:
type: "image"file: "<basename.ext>"split: "train" | "val" | "test"width,heightannotations: { boxes: [...] }
-
Negative image:
annotations: { boxes: [] }
YOLO compatibility export
labels/{split}/{stem}.txt- Empty file when image has no exported boxes
data.yamlwithnames,train,val,testas needed
Ultralytics’ docs define detect labels as normalized class x_center y_center width height, allow zero-indexed classes, and say images with no objects do not require a .txt file. The official NDJSON converter writes an empty label file when there are no annotation lines. (Ultralytics Docs)
type Split = "train" | "val" | "test";
type ImageReviewState = "complete" | "incomplete" | "negative";
type BoxReviewState = "tp" | "fp" | "ignore";
type ClassDef = {
id: number; // zero-indexed
name: string;
color: string;
source: "preset" | "custom" | "imported";
};
type BoxAnnotation = {
id: string;
classId: number;
x: number; // normalized 0..1
y: number; // normalized 0..1
w: number; // normalized 0..1
h: number; // normalized 0..1
zIndex: number;
review: BoxReviewState;
locked: boolean;
hidden: boolean;
note?: string;
};
type ImageRecord = {
id: string;
fileName: string;
storedBlobKey: string;
width: number;
height: number;
split: Split;
reviewState: ImageReviewState;
annotations: BoxAnnotation[];
hash: string;
};
type Project = {
id: string;
name: string;
description?: string;
version: number;
createdAt: string;
updatedAt: string;
classes: ClassDef[];
images: ImageRecord[];
exportOptions: {
includeYaml: boolean;
includeTxtLabels: boolean;
blockIncompleteImages: boolean;
};
};Important rule
-
dataset.ndjsonremains training-safe and minimal. -
meta/project.jsonstores UI-only fields such as:- box review state
- z-order
- class colors
- image completeness
- local hashes
- any future editor preferences
Ultralytics describes NDJSON as extensible, but I would still keep UI-only metadata out of the training NDJSON to reduce compatibility risk. (Ultralytics Docs)
Repo name
dataset-annotation-app
BusinessRepo structure
/app
/common
/infra
/tests
/docs
Makefile
Recommended internal structure
/app-openlabel
/annotate
/import
/export
/settings
/common
/domain
/dataset
/annotations
/classes
/application
/use-cases
/services
/infrastructure
/persistence
/zip
/parsers
/serializers
/ui
/components
/layouts
/hooks
/hoc
/infra
/github-actions
/hosting
/tests
/unit
/integration
/e2e
/docs
/adr
/prd
Architecture rules
-
Clean Architecture.
-
Pure domain objects.
-
Serialization isolated from UI.
-
IndexedDB repository behind interfaces.
-
Export/import services pure and testable.
-
React components presentational where possible.
-
Hooks for composition.
-
HOCs only for narrow cross-cutting concerns:
withClientOnlywithErrorBoundarywithHotkeys
-
Do not use HOCs as the primary state model.
Suggested libraries
- Zustand for app state
- Dexie for IndexedDB
- Zod for schema validation
fflateorzip.jsfor zip import/exportreact-dropzonefor uploads- SVG overlay for box editing
- Playwright for e2e
- Vitest + React Testing Library for unit/integration
- User can upload several images and annotate them locally.
- User can select COCO classes or create new ones during annotation.
- Overlapping boxes work correctly and can be reordered.
- Box coordinates export normalized to 0..1 with zero-indexed class IDs.
- Negative images export with empty detect annotations and empty compatibility txt files.
- Re-import of an exported zip preserves classes, annotations, splits, TP/FP review state, image completeness, colors, and z-order.
- Export warns or blocks on incomplete images by default.
- Undo/redo, keyboard shortcuts, and autosave work reliably. (Ultralytics Docs)
-
NDJSON only, no sidecar metadata Rejected. It weakens round-trip fidelity for TP/FP review state, z-order, and editor metadata.
-
Encode FP as a “background” class Rejected. Issue #8889 is specifically about background/negative images in multi-class detection, and the guidance is to use empty annotations for images with no objects of interest rather than inventing a special exported class. (GitHub)
-
Backend upload/storage service Rejected. Violates your explicit no-backend requirement.
-
Multi-task v1 Rejected. Detect-only is the correct scope boundary.
-
Browser memory / CPU
- Large projects can freeze the tab during image decode or zip generation.
- Mitigate with workers, lazy thumbnails, virtualization, and debounced autosave.
-
Partial annotation risk
- Mixed reviewed/unreviewed images can poison training quality.
- Mitigate with image completeness state and export gating. (Ultralytics Docs)
-
Image format mismatch
- Ultralytics Platform accepts more formats than browsers decode consistently.
- v1 should prioritize JPEG, PNG, WebP, BMP, and TIFF where browser support is practical; HEIC, DNG, JP2, MPO should be non-goals or progressive enhancement. (Ultralytics Docs)
-
Import trust boundary
-
Malicious zips, path traversal, malformed NDJSON, and hostile filenames are the main threats.
-
Mitigate with:
- zip path normalization
- size limits
- schema validation
- filename sanitization
- escaping rendered labels/classes
- no remote auto-fetch by default
-
-
Data loss
- Browser-only apps are fragile if state is only in memory.
- Mitigate with IndexedDB autosave + explicit export reminders.
-
Ops / cost
- Near-zero backend cost.
- Static hosting only.
- No server blast radius.
-
SRE
- Main failure modes move to client memory pressure, corrupted local state, and failed imports.
- Recovery is export/re-import plus autosave snapshots.
-
Product
- Strong for individual annotators.
- Weak for collaboration and large centralized data ops.
Build a detect-only, browser-only annotation app with NDJSON as canonical export, YOLO txt/YAML as compatibility output, and sidecar metadata for lossless round-trip editing.
-
Scaffold the BusinessRepo.
-
Implement import + persistence first.
-
Implement box editor second.
-
Implement export + re-import third.
-
Add tests before polishing UI.
-
Write ADRs for:
- NDJSON canonical export
- sidecar metadata
- detect-only scope
- no-backend architecture
This prompt encodes the verified constraints around detect labels, NDJSON structure, negative images, split handling, class creation, editor affordances, and safe export semantics. (Ultralytics Docs)
You are acting as a staff+ product engineer, principal frontend architect, and UX lead.
First, review these sources before proposing or coding anything:
- https://docs.ultralytics.com/datasets/detect/#supported-datasets
- https://docs.ultralytics.com/platform/data/
- https://docs.ultralytics.com/platform/data/annotation/
- https://docs.ultralytics.com/reference/data/converter/
- https://github.com/ultralytics/ultralytics/issues/8889
Project goal:
Build a browser-only Next.js + React + TypeScript application called `dataset-annotation-app` for Ultralytics YOLO object-detection dataset annotation.
Hard constraints:
- No backend.
- No API routes.
- No server actions.
- No remote database.
- Browser-only persistence via IndexedDB/local browser storage.
- Detect task only in v1.
- Use Clean Architecture and SOLID.
- Use BusinessRepo structure:
/app
/common
/infra
/tests
/docs
Makefile
- Use composition and hooks by default.
- Use HOCs only for narrow cross-cutting concerns.
- Never add any `Co-Authored-By` lines.
User preferences to honor:
- Clear, concise, practical output.
- No hype or marketing language.
- Optimize for senior technical / product / executive audience.
- Staff/principal-level rigor.
- Call out trade-offs and risks directly.
- Prefer long-term maintainability.
- BusinessRepo naming and structure rules apply.
- Clean Architecture, SOLID, composition over inheritance.
- Testability, observability, reliability, and security by default.
Product requirements:
1. Upload several images locally.
2. Allow selecting existing YOLO class presets and creating custom classes.
3. Ship a pluggable class-pack registry.
4. Include COCO-80 as the first built-in preset.
5. Support overlapping axis-aligned boxes as layers.
6. Support:
- draw box by drag
- resize with 8 handles
- move by dragging center
- delete
- multi-select
- z-order controls
- zoom / pan
- undo / redo
- keyboard shortcuts
7. Allow creating classes during annotation.
8. Support a review workflow with:
- box review state: TP / FP / ignore
- image review state: complete / incomplete / negative
9. Important semantic rule:
- TP boxes are exported as training annotations
- FP boxes are NOT exported as training boxes
- ignore boxes are NOT exported as training boxes
- if an image has zero TP boxes, export it as a negative image
10. Export a zip that contains:
- `dataset.ndjson` as the canonical source of truth
- actual image files in split folders
- `data.yaml` as compatibility output
- YOLO txt label files as compatibility output
- `meta/project.json` for UI-only round-trip metadata
11. Re-import any dataset generated by this tool and preserve:
- classes
- annotations
- TP/FP/ignore review state
- image completeness state
- z-order
- class colors
- splits
12. Default export should block incomplete images unless the user explicitly overrides with a warning.
13. Do not scrape docs at runtime.
14. Do not implement model inference or SAM in v1.
15. Do not export FP as a background class.
Export contract:
- `dataset.ndjson` is canonical.
- `dataset.ndjson` must use detect task only.
- First line is the dataset record.
- Remaining lines are image records.
- Image records should use:
- `type: "image"`
- `file: "<basename.ext>"`
- `split: "train" | "val" | "test"`
- `width`
- `height`
- `annotations: { boxes: [...] }`
- Detect boxes must be normalized `[class_id, x_center, y_center, width, height]`.
- Zero-index all class IDs.
- For negative images, export `annotations: { boxes: [] }`.
- Keep UI-only metadata out of training annotations.
- Store UI-only metadata in `meta/project.json`.
Zip structure:
dataset-annotation-app-export.zip
├── dataset.ndjson
├── data.yaml
├── images/
│ ├── train/
│ ├── val/
│ └── test/
├── labels/
│ ├── train/
│ ├── val/
│ └── test/
└── meta/
└── project.json
Import requirements:
- First-class support for re-importing this app’s own zip format.
- Fallback support for raw NDJSON + images zip.
- Optional stretch: parse YOLO detect folder structure.
Architecture requirements:
- Next.js app router is acceptable, but runtime must remain browser-only.
- No server dependence in the user flow.
- Recommended libraries:
- Zustand
- Dexie
- Zod
- fflate or zip.js
- react-dropzone
- Vitest
- React Testing Library
- Playwright
- Prefer an SVG overlay for annotation editing unless you can justify a better browser-only approach.
- Add HOCs only where they genuinely help, for example:
- withClientOnly
- withErrorBoundary
- withHotkeys
- Keep domain logic out of React components.
State model to implement:
- Project
- ClassDef
- ImageRecord
- BoxAnnotation
- ExportOptions
- ImportResult
- ValidationIssue
Suggested domain rules:
- Unique sanitized file names per project export.
- Preserve original file names in metadata.
- Hash images locally for dedupe and stable IDs.
- Autosave to IndexedDB after meaningful edits.
- Validate all imported files and schemas before mutating project state.
- Normalize coordinates strictly to 0..1.
- Warn on zero-area or out-of-bounds boxes.
- Track image completeness separately from box review state.
UX requirements:
- Left panel: image list/grid with split + counts + review badge.
- Center: annotation canvas.
- Right panel: annotation stack, class picker, box properties, review tools.
- Clear support for overlapping boxes.
- Annotation list should expose z-order clearly.
- Keyboard shortcuts should cover save, undo, redo, delete, zoom, z-order, class quick-select.
- Accessibility:
- visible focus states
- keyboard operability for major actions
- non-color-only status indicators
Definition of done:
- A user can upload several images.
- A user can choose COCO classes or add new classes.
- A user can annotate overlapping person/tie-style boxes correctly.
- A user can mark boxes TP/FP/ignore.
- A user can mark images complete/incomplete/negative.
- Exported zip is valid and round-trippable.
- Re-import restores the project faithfully.
- Unit, integration, and e2e tests cover core workflows.
- The repo includes a proper README and ADRs.
Deliverables:
1. Final PRD in this format:
1. Title
2. Context
3. Goals / Non-goals
4. Proposal
5. Alternatives
6. Trade-offs and Risks
7. Impact
8. Decision
9. Next Steps
2. Final repository tree.
3. Key ADRs:
- detect-only scope
- NDJSON canonical export
- sidecar metadata strategy
- browser-only persistence strategy
4. Full implementation plan.
5. Actual code, file by file, with paths.
6. Tests:
- unit tests for serializers/parsers/coordinate transforms
- integration tests for store and import/export
- e2e tests for upload → annotate → export → re-import
7. README with:
- local setup
- architecture overview
- export format
- import format
- known limitations
8. Makefile with at least:
- install
- dev
- lint
- typecheck
- test
- e2e
- build
Important execution behavior:
- Do not ask follow-up questions unless there is a hard blocker.
- Make reasonable assumptions and state them explicitly.
- Be opinionated.
- Reject weak designs clearly.
- Do not hand-wave implementation details.
- Output concrete code, not vague pseudo-code.