0.4.0 round 4: LoadFromArrays, CreateBytesAsync, migration lessons

CHANGELOG.md

@@ -12,6 +12,7 @@ All notable changes to SheetStream. Versions follow [Semantic Versioning](https:
 - **Pivot tables**: `Worksheet.AddPivotTable` over a table or range written earlier, with row, column, filter and data fields (Sum, Count, Average, Max, Min and more), styles and grand totals. The cache is marked refresh-on-open, so Excel computes the pivot when the file opens; verified against Excel 16 through COM automation (`tests/excel-check-pivot.ps1`).
 - **Cached formula results**: `Worksheet.CacheFormulaResults()` opts a sheet into a small evaluator (`+ - * / %`, `SUM`, `AVERAGE`, `COUNT`, `COUNTA`, `MIN`, `MAX`, `ROUND`, `ABS` over cells already written) so viewers that do not calculate show totals instead of blanks. Anything it cannot prove is written without a value, as before.
 - **Async reading** (.NET 8+): `ExcelReader.OpenAsync`, `ReadRowsAsync`, `ReadAsync<T>` and `ReadDataTableAsync`, with cancellation between rows; non-seekable uploads are buffered asynchronously and spill to a temp file above 64 MB.
+- **From the first production migration (LogisticaHQ)**: `Worksheet.LoadFromArrays` replaces EPPlus' `Cells.LoadFromArrays` one for one; `ExcelPackage.CreateBytesAsync` lets a builder await a query per sheet; the migration guide gained "Things EPPlus did for you silently" (the code-page encoding provider that ExcelDataReader imports depend on, AutoFit timing, default date formats).
 - **Reader**: `ReadMergedRanges` and `ReadHyperlinks` (issue #6) read a sheet's merged cells and links, external URLs resolved from the sheet relationships, without parsing the rows.
 - **Benchmarks**: 1,000,000-row and 1,000 × 10,000-column write scenarios, and read benchmarks against MiniExcel, ExcelDataReader, ClosedXML, EPPlus and NPOI.
 - **Web sample** (`samples/SheetStream.Sample.Web`): a minimal API with a download button, a 50,000-row streamed export and a `Cells[...]` report with charts; double-click `run.cmd`. A [DotNetFiddle](https://dotnetfiddle.net/BR59ju) runs the quick start in the browser.

README.md

@@ -79,6 +79,8 @@ Need bytes for a web download, like EPPlus' `GetAsByteArray()`?
 
 ```csharp
 byte[] bytes = ExcelPackage.CreateBytes(wb => wb.Worksheets.Add("Export").LoadFromCollection(orders));
+byte[] bytes = await ExcelPackage.CreateBytesAsync(async wb =>            // Await a query per sheet
+    wb.Worksheets.Add("Export").LoadFromCollection(await db.Orders.ToListAsync()));
 ```
 
 ## 🧭 Feature tour
@@ -94,6 +96,7 @@ var result = sheet.LoadFromCollection(orders, new LoadOptions
 });
 sheet.LoadFromDataTable(dataTable);
 sheet.LoadFromDataReader(sqlCommand.ExecuteReader()); // Streams a large query row by row
+sheet.LoadFromArrays(listOfObjectArrays);              // First array is the header row (EPPlus LoadFromArrays)
 ```
 
 Control columns with attributes: `[SheetColumn("Invoice #", Order = 0, NumberFormat = "$#,##0.00")]`, `[SheetColumn(Ignore = true)]`, or `[DisplayName("...")]`. `IDictionary<string, object>` rows (ExpandoObject, Dapper) work too.

docs/migrating-from-epplus.md

@@ -138,10 +138,15 @@ The file is built into a temporary file and streamed to the response, so memory
 | EPPlus | SheetStream |
 |---|---|
 | `new ExcelPackage()` … `GetAsByteArray()` | `ExcelPackage.CreateBytes(wb => …)` |
+| `await package.GetAsByteArrayAsync()` with queries awaited between sheets | `await ExcelPackage.CreateBytesAsync(async wb => { … await db.QueryAsync(…) … })` |
 | `new ExcelPackage(file)` … `SaveAs(file)` | `ExcelPackage.Create(path)` … `Save()` |
 | `new ExcelPackage(templateFile)` to fill a template | `ExcelPackage.OpenTemplate(template, output)` + `Worksheets.Open("Sheet")` |
+| `new ExcelPackage(file)`, change cells, `Save()` (small files) | `ExcelPackage.Edit(path)` + `Worksheets.Edit("Sheet")` + `Cells[…]` |
 | `ws.Cells["A1"].LoadFromCollection(list, true)` | `ws.LoadFromCollection(list)` |
 | `ws.Cells["A1"].LoadFromDataTable(table, true)` | `ws.LoadFromDataTable(table)` |
+| `ws.Cells["A1"].LoadFromArrays(listOfObjectArrays)` | `ws.LoadFromArrays(listOfObjectArrays)` (first array = header row, styled with `HeaderStyle`) |
+| `ws.Cells[1, 1, 1, n].Style.Font.Bold = true` on a header row | `var bold = Styles.Create(o => o.Font.Bold = true); ws.Row(1).WriteCells(bold, headers)` |
+| `ws.Cells[ws.Dimension.Address].AutoFitColumns()` at the end | `ws.AutoFitColumns()` **before the first row** (it measures rows as they stream) |
 | `ws.Cells[r, c].Value = v` | `ws.Row(r).WriteCell(v)` or `ws.Cells[r, c].Value = v` |
 | `ws.Cells[r, c].Formula = "SUM(A1:A9)"` | `row.WriteFormula("SUM(A1:A9)")` |
 | `cell.Style.Numberformat.Format = "0.00%"` | `Styles.Create(o => o.NumberFormat = "0.00%")` |
@@ -164,16 +169,32 @@ The file is built into a temporary file and streamed to the response, so memory
 
 These EPPlus features have no SheetStream equivalent today: formula calculation (`Calculate()`; SheetStream only writes cached results for simple aggregates, see `CacheFormulaResults`), pivot tables with calculated fields, grouping or slicers (plain pivots are supported and refreshed on open), sparklines, editing large arbitrary workbooks in place (small workbooks can be edited with `ExcelPackage.Edit`; large ones are filled as templates), VBA, encryption and digital signatures. If an exporter depends on one of them, keep that exporter on EPPlus 4.5.3.3 (LGPL) or another library.
 
-## After removing the EPPlus package
+## Things EPPlus did for you silently
 
-Removing EPPlus can break code that never called it. EPPlus 4.5.3.3 depends on `System.Text.Encoding.CodePages` and registers the code-page encoding provider when it loads. Other libraries in the same application may rely on that registration without declaring it; **ExcelDataReader** is the usual case, because legacy `.xls` files need code-page encodings. With EPPlus gone, those imports fail, sometimes silently with empty text.
+Removing the package can break code that never called it. These were found migrating a warehouse management system whose exports moved to SheetStream.
+
+### 1. It registered the code-page encoding provider
+
+EPPlus depends on `System.Text.Encoding.CodePages` and registers `CodePagesEncodingProvider` as a side effect: 4.5.3.3 when it loads, 5 and later as soon as `ExcelPackage` is touched, which the start-up licence call does. Other libraries in the same application rely on that registration without declaring it. **ExcelDataReader** is the usual case, because legacy `.xls` files need code-page encodings. With EPPlus gone, every import fails with:
+
+```
+System.ArgumentException: No data is available for encoding 1252.
+```
 
 Register the provider yourself at startup, before the first import:
 
 ```csharp
-// Program.cs, before building the app. On .NET Framework or netstandard targets add the
-// System.Text.Encoding.CodePages package; on .NET 8 it is part of the shared framework.
+// Program.cs, before building the host. On .NET 5+ the provider is part of the shared framework;
+// on .NET Framework or netstandard targets add the System.Text.Encoding.CodePages package.
 System.Text.Encoding.RegisterProvider(System.Text.CodePagesEncodingProvider.Instance);
 ```
 
-Then run one `.xls` import in a test to prove it. This was found while migrating a warehouse system whose exports moved to SheetStream but whose imports stayed on ExcelDataReader. For `.xlsx` imports, `ExcelReader` needs no provider.
+Then run one `.xls` import in a test to prove it. For `.xlsx` imports, SheetStream's `ExcelReader` needs no provider.
+
+### 2. It let you AutoFit at the end
+
+EPPlus code calls `AutoFitColumns()` after the last row, when every value is known. SheetStream writes column widths before the data, so `sheet.AutoFitColumns()` must be called **before the first row**; it measures the rows as they stream past and writes the widths at save time. A straight port that keeps the call at the end throws, and the analyzer that ships in the package flags it in the IDE (SS0001).
+
+### 3. It wrote dates as raw serial numbers
+
+EPPlus, 4 through 8, writes a `DateTime` as its serial number with the General format, so a cell showed `45123.5` unless you set `Numberformat.Format`. SheetStream applies a built-in date style when you give none: `mm-dd-yy` for dates and `m/d/yy h:mm` for date-times, which Excel shows in the user's locale. Pass a style with your own format (`Styles.Create(o => o.NumberFormat = "yyyy-mm-dd")`, or `[SheetColumn(NumberFormat = "...")]` for `LoadFromCollection`) when the report needs a fixed format. Tell your users that date columns now look like dates.

src/SheetStream/ExcelPackage.cs

@@ -112,6 +112,34 @@ public static byte[] CreateBytes(Action<Workbook> build, PackageOptions? options
             }
         }
 
+        /// <summary>
+        /// Like <see cref="CreateBytes"/>, but the builder can await between sheets (a query per sheet, say).
+        /// The workbook itself is still written synchronously into memory, so there is no I/O to await after
+        /// the builder returns; this exists so callers need not run every query first and build afterwards.
+        /// </summary>
+        /// <example>
+        /// <code>
+        /// var bytes = await ExcelPackage.CreateBytesAsync(async wb =>
+        /// {
+        ///     wb.Worksheets.Add("Orders").LoadFromCollection(await db.Orders.ToListAsync(ct));
+        ///     wb.Worksheets.Add("Customers").LoadFromCollection(await db.Customers.ToListAsync(ct));
+        /// });
+        /// </code>
+        /// </example>
+        public static async System.Threading.Tasks.Task<byte[]> CreateBytesAsync(Func<Workbook, System.Threading.Tasks.Task> build, PackageOptions? options = null)
+        {
+            if (build == null) throw new ArgumentNullException(nameof(build));
+            using (var ms = new MemoryStream())
+            {
+                using (var package = new ExcelPackage(ms, leaveOpen: true, options))
+                {
+                    await build(package.Workbook).ConfigureAwait(false);
+                    package.Save();
+                }
+                return ms.ToArray();
+            }
+        }
+
         /// <summary>The single workbook of this package.</summary>
         public Workbook Workbook { get; }
 

src/SheetStream/Worksheet.Load.cs

@@ -108,6 +108,41 @@ public LoadResult LoadFromDataTable(DataTable table, LoadOptions? options = null
                 i => table.Rows[index][i]);
         }
 
+        /// <summary>
+        /// Writes rows given as arrays, the direct replacement for EPPlus' <c>Cells.LoadFromArrays</c>. With
+        /// <see cref="LoadOptions.PrintHeaders"/> (the default) the first array is the header row and gets
+        /// <see cref="LoadOptions.HeaderStyle"/>; every later array is a data row. Rows may be shorter than the
+        /// header (missing cells stay empty); longer rows have their extra values dropped, because a table or
+        /// filter needs a fixed column count. Values are written like <see cref="RowWriter.WriteCell(object?, CellStyle?)"/>.
+        /// </summary>
+        /// <example>
+        /// <code>
+        /// sheet.LoadFromArrays(new[] { new object?[] { "Sku", "Qty" }, new object?[] { "A-1", 3 } }, new LoadOptions { TableName = "Stock" });
+        /// </code>
+        /// </example>
+        public LoadResult LoadFromArrays(IEnumerable<object?[]> rows, LoadOptions? options = null)
+        {
+            if (rows == null) throw new ArgumentNullException(nameof(rows));
+            options ??= new LoadOptions();
+            using (var e = rows.GetEnumerator())
+            {
+                if (!e.MoveNext())
+                    return new LoadResult(CurrentRow + 1, CurrentRow, options.StartColumn, options.StartColumn - 1, 0);
+
+                // The first array sizes the sheet. Without headers it is data, so it must be written too.
+                var first = e.Current ?? new object?[0];
+                var headers = new string[first.Length];
+                for (var i = 0; i < headers.Length; i++)
+                    headers[i] = options.PrintHeaders
+                        ? (first[i] == null ? Internal.CellReference.ColumnLetters(options.StartColumn + i) : Convert.ToString(first[i], System.Globalization.CultureInfo.InvariantCulture) ?? "")
+                        : Internal.CellReference.ColumnLetters(options.StartColumn + i);
+                var pendingFirst = !options.PrintHeaders;
+                return LoadCore(headers, new CellStyle?[headers.Length], options,
+                    () => { if (pendingFirst) { pendingFirst = false; return true; } return e.MoveNext(); },
+                    i => e.Current != null && i < e.Current.Length ? e.Current[i] : null);
+            }
+        }
+
         /// <summary>
         /// Writes everything an <see cref="IDataReader"/> returns, streaming row by row; the natural fit for
         /// exporting a large SQL query without materialising it.

tests/SheetStream.Tests/LoadFromArraysTests.cs

@@ -0,0 +1,90 @@
+using System.IO;
+using System.Threading.Tasks;
+using SheetStream.Reading;
+
+namespace SheetStream.Tests;
+
+/// <summary>
+/// LoadFromArrays (the EPPlus Cells.LoadFromArrays replacement) and CreateBytesAsync, both requested by the
+/// first production migration.
+/// </summary>
+public class LoadFromArraysTests
+{
+    private static readonly object?[][] Rows =
+    {
+        new object?[] { "Sku", "Description", "Qty" },
+        new object?[] { "A-1", "Bolt", 3 },
+        new object?[] { "B-2", "Nut" },                 // Shorter: Qty stays empty.
+        new object?[] { "C-3", "Washer", 7, "extra" },  // Longer: the extra value is dropped.
+    };
+
+    [Fact]
+    public void FirstArrayIsTheHeaderAndTheRestAreRows()
+    {
+        var bytes = ExcelPackage.CreateBytes(wb =>
+        {
+            var bold = wb.Styles.Create(o => o.Font.Bold = true);
+            var result = wb.Worksheets.Add("Stock").LoadFromArrays(Rows, new LoadOptions { HeaderStyle = bold, TableName = "Stock" });
+            Assert.Equal(1, result.FirstRow);
+            Assert.Equal(4, result.LastRow);
+            Assert.Equal(3, result.LastColumn);
+            Assert.Equal(3, result.DataRows);
+            Assert.Equal(new[] { "Sku", "Description", "Qty" }, result.Headers);
+            Assert.NotNull(result.Table);
+        });
+
+        using var reader = ExcelReader.Open(new MemoryStream(bytes));
+        var rows = reader.ReadRows("Stock").ToList();
+        Assert.Equal(4, rows.Count);
+        Assert.Equal("Sku", rows[0].GetString(0));
+        Assert.Equal(3.0, rows[1].GetDouble(2));
+        Assert.Null(rows[2][2]);
+        Assert.Equal(3, rows[3].Values.Count); // "extra" was not written.
+    }
+
+    [Fact]
+    public void WithoutHeadersTheFirstArrayIsData()
+    {
+        var bytes = ExcelPackage.CreateBytes(wb =>
+            wb.Worksheets.Add("Data").LoadFromArrays(Rows, new LoadOptions { PrintHeaders = false, StartColumn = 2 }));
+
+        using var reader = ExcelReader.Open(new MemoryStream(bytes));
+        var rows = reader.ReadRows("Data").ToList();
+        Assert.Equal(4, rows.Count);
+        Assert.Null(rows[0][0]);                      // Column A is skipped (StartColumn = 2).
+        Assert.Equal("Sku", rows[0].GetString(1));    // The "header" array is an ordinary row here.
+        Assert.Equal("Bolt", rows[1].GetString(2));
+    }
+
+    [Fact]
+    public void EmptyInputWritesNothing()
+    {
+        var bytes = ExcelPackage.CreateBytes(wb =>
+        {
+            var result = wb.Worksheets.Add("Empty").LoadFromArrays(System.Array.Empty<object?[]>());
+            Assert.Equal(0, result.DataRows);
+        });
+        using var reader = ExcelReader.Open(new MemoryStream(bytes));
+        Assert.Empty(reader.ReadRows("Empty"));
+    }
+
+    [Fact]
+    public async Task CreateBytesAsyncAwaitsBetweenSheets()
+    {
+        var bytes = await ExcelPackage.CreateBytesAsync(async wb =>
+        {
+            wb.Worksheets.Add("First").LoadFromArrays(await FakeQueryAsync("one"));
+            wb.Worksheets.Add("Second").LoadFromArrays(await FakeQueryAsync("two"));
+        });
+
+        using var reader = ExcelReader.Open(new MemoryStream(bytes));
+        Assert.Equal(new[] { "First", "Second" }, reader.SheetNames);
+        Assert.Equal("two", reader.ReadRows("Second").Skip(1).First().GetString(0));
+    }
+
+    private static async Task<object?[][]> FakeQueryAsync(string value)
+    {
+        await Task.Yield(); // Really asynchronous, so the builder resumes on another continuation.
+        return new[] { new object?[] { "Value" }, new object?[] { value } };
+    }
+}