Open-source dataset inspection and quality analysis toolkit for ML datasets.
Inspect your datasets before training models!
DatasetLens helps you understand what is inside your datasets before they reach your training pipeline.
It can help identify:
- File formats
- Dataset composition
- Supported and unsupported files
- Broken or corrupt files
- Duplicate files
- Encoding and language information
- Dataset-level statistics
The goal is simple: "Know your dataset before you train your model."
🚧 Early Development
DatasetLens is actively evolving.
- v0.1.0-alpha — Early development build (Deleted, due to crashes)
- v0.2.0-alphs — Stable text-only build <----- Typo error must not die! 💀
- v0.3.0-alpha — Image inspection added
Current development is focused on stability, inspection quality, and expanding supported modalities.
Star this repo for upcoming features or issue a request instead! 🐱
- Text Inspector ✅
- Image Inspector ✅
- Duplicate Detection
- Advanced Text Inspection
- Advanced Image Inspection
- 3D Voxel
- Point Clouds
- TIFF
- Audio
- Video
- Graph
- NLP
For deeper project information:
- 📚
Documents.md— Project documentation and technical information - 👥
CONTRIBUTORS.md— Contributors and acknowledgements