TODO.md

StreamLib Development Roadmap

Phase 1: Core Infrastructure ✓ COMPLETED

• Design specification
• Project structure
• CMake build system
• Core types and base stream

Phase 2: Basic Streams ✓ COMPLETED

• Implement base stream operations
• Implement file_stream
  • POSIX implementation
  • Windows implementation (CreateFileA, ReadFile/WriteFile, mmap)
  • mmap support with offset
• Implement mem_stream
• Unit tests for core streams

Phase 3: Compression Support ✓ COMPLETED

• Design compression_stream interface
• Implement gzip support (zlib)
• Implement bzip2 support
• Implement xz/lzma support
• Implement zstd support
• Auto-detection from magic bytes
• Unit tests for each codec (20 tests total)

Phase 4: Archive Support ✓ COMPLETED

• Design archive_stream interface
• Implement libarchive wrapper
• Support reading archive entries
• Support nested compression (transparent decompression in archives)
• Archive format detection by magic bytes (not extension)
• Unit tests for archives

Phase 5: Path Walker ✓ COMPLETED

• Implement directory walking
  • POSIX (readdir)
  • Windows (FindFirstFileA/FindNextFileA)
• Implement archive expansion
• Capability filtering (files/dirs)
• Transparent decompression with magic byte detection
• Unit tests for walker (9 tests total)

Phase 6: Advanced Features ✓ COMPLETED

• Printf support (stream_vprintf) - Implemented with fallback
• mmap emulation for non-mmapable streams - Implemented for compression_stream
• [~] stream_copy utility - Not needed
• [~] stream_read_all utility - Not needed (use mmap instead)
• Performance benchmarks

Phase 7: Documentation & Examples ✓ COMPLETED

• [~] API reference (Doxygen) - Using annotated headers instead
• User guide (README.md with comprehensive examples)
• Example programs
  • walk_tree - Directory and archive walker with decompression
  • Compressed file reading (compression tests)
  • Large file processing with mmap (large_file_mmap.c)
  • Mmap emulation demo (test_mmap_emulation.c)
• Compression format table with magic bytes

Phase 8: Testing & Polish (Mostly Complete)

• Comprehensive test suite
  • 20 compression tests (all 4 formats)
  • 9 walker tests
  • 7 basic stream tests
  • Magic byte detection verification
  • Transparent decompression tests
  • Verified with real-world .vgz files (161 files across 6 archives)
• Memory leak testing (valgrind)
  • Zero leaks in compression tests (202 allocs, 202 frees)
  • Zero leaks in walker tests (222 allocs, 222 frees)
  • Zero leaks with real archives (tested with 45+ .vgz files)
  • Perfect allocation/deallocation balance across all tests
• Cross-platform CI (GitHub Actions)
  • Linux: GCC + Clang, full + minimal builds
  • macOS: full + minimal builds (with proper zstd linking)
  • Valgrind memory leak checks in CI
  • Static analysis (cppcheck)
  • Code coverage reporting
  • Release build workflow
  • Windows CI testing (vcpkg + full/minimal builds)
• Large file tests (>4GB)
• Error handling coverage

Future Enhancements

• Write support for compression streams - Fully implemented and documented
• Create/modify archive support - Fully implemented for all libarchive formats (TAR, ZIP, 7z, CPIO, ISO9660, SHAR)
• Additional compression formats (lz4, brotli)
• Async I/O support
• Network stream support (HTTP, FTP)
• Encryption/decryption streams

Known Limitations

• Large file tests: Not yet tested with files >4GB