True streaming packages: Binary/Xml format entities as Packages for progressive parse/render #35

Design intent (network streaming)

Imagine an Excel file arriving over the network. A streaming parser should:

1. Read bytes / XML tokens incrementally as they arrive
2. Recognize a local complete format unit (one BIFF record, one cell/row element, …)
3. Immediately emit that unit as a Package
4. Let a renderer stream-render packages without waiting for the whole file

So: Network Stream → Parser (incremental) → Package (complete local fragment) → Renderer.

What is still wrong after #33 / PR #34

1. Reader/Binary and Reader/Xml format types are not Packages
   BinaryPackage / XmlPackage are thin wrappers around the real format objects (XlsBiffRecord, worksheet XML handling). The format entities themselves should be the packages.

2. Parsing is still bulk, not network-incremental

   • Binary: workbook stream is fully buffered (ReadStream() → byte[]) before records are walked
   • OpenXML: ZipWorker.Extract unpacks the entire ZIP to disk before any sheet XML is read

   That cannot progressively emit packages while data is still in flight.

3. Consumers still think in “whole workbook → DataSet”
   Progressive render needs a package stream API first; DataSet can remain a batch consumer on top.

Target shape

Binary

• BIFF record types are BinaryPackage (inheritance or the record hierarchy lives under the package model)
• BinaryPackageParser reads a Stream and emits a package as soon as type+size+body for one record is complete (no need to buffer the entire workbook stream up front when the source allows)
• OLE directory/FAT positioning is an IO concern; once the workbook stream is positioned, record parse is strictly streaming

XML (OpenXML)

• Sheet semantic units under Reader/Xml are XmlPackage (cell / row / merge / hyperlink / …)
• XmlPackageParser emits a package when a local XML fragment is complete (e.g. finished </c> or </row>)
• Prefer streaming ZIP entry inflate for sheetN.xml instead of full-archive extract to temp

Renderer contract

• IEnumerable<IPackage> / pull parser TryRead(out IPackage) suitable for progressive UI or markdown emission
• Batch DataSet builder remains optional downstream

Acceptance criteria

• Format entities in Binary/Xml are Packages (not wrapped by a parallel unrelated type tree)
• Binary parser can emit record packages from a forward-only workbook stream without requiring a full prior byte[] copy of that stream (API-level; buffering only one record)
• Xml parser emits packages at local completeness boundaries (cell/row/…)
• OpenXML sheet reading can use a streamed ZIP entry (document remaining gaps if full OPC streaming is phased)
• src/test.sh stays green
• Document how a progressive renderer would subscribe to the package stream

Related

• refactor excel formart reader as package-packageparser system #1 original package-parser request
• Refactor package/parser to streaming BinaryPackage + XmlPackage model #33 first streaming refactor (insufficient — wrappers + bulk IO)