Repository navigation
0.4.0 round 3: async reader, cached formula results, pivot tables - #11
Merged
Merged
Conversation
…sync The reader now has an asynchronous story matching the writer's LoadFromAsyncEnumerable (net8.0 only, in ExcelReader.Async.cs): - OpenAsync(Stream) copies a non-seekable stream (request body, HttpClient) with asynchronous reads; up to 64 MB stays in memory, anything larger spills into a DeleteOnClose temp file so a big upload never doubles the managed heap. OpenAsync(path) opens the file for overlapped I/O; the package directory and metadata are still parsed synchronously (ZipArchive has no async API), and the docs say so. - ReadRowsAsync parses the sheet with XmlReader's async methods, checks cancellation between rows and disposes the XML reader and entry stream when the consumer stops early. ReadAsync<T> (reflection or registered mapper), ReadAsync<T>(mapper) and ReadDataTableAsync build on it. The cell typing (CellValue), row-buffer placement (PlaceCell), row-to-object mapping (MapRow) and DataTable column rules are factored into helpers shared by the sync and async parsers, so the two can never disagree; the sync loops are otherwise unchanged. 16 tests compare every async API with the sync one row by row, cover the non-seekable and spill paths, cancellation and early exit. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…MIN/MAX/ROUND/ABS Worksheet.CacheFormulaResults() makes WriteFormula(formula, style) also write the cached value when the formula is a simple aggregate over cells already written on the same sheet, so non-calculating viewers and ExcelReader show numbers instead of blanks. Anything outside the subset (other functions, other sheets, forward references, text in arithmetic, division by zero) is written exactly as before, without a value; a wrong value is never written. - Internal/FormulaValueStore: per-column double[] indexed by row, 8 bytes per cell; empty/text/unknown states are NaN payloads in the same slot. - Internal/FormulaEvaluator: recursive-descent parser that evaluates as it parses, fails without exceptions, honours precedence, unary minus, %, absolute and sheet-qualified references naming the current sheet. - RowWriter records every value through a single bool branch per cell; the default path is unchanged. Formula results feed later formulas. - Cells[...] mode evaluates at flush; edit mode records loaded values; copied template rows are opaque so references to them get no value. - 92 tests: grammar, refusals, store growth, streaming, disabled, Cells, edit, template, LoadFromCollection; files validated with the OpenXml SDK. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A single far-away cell now costs an 8 KB page instead of an array sized to its row, which bounds the memory a crafted input file could demand in edit mode with caching on. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… by Excel on open The source sheet is already streamed out when the pivot is written, so the file carries the pivot layout plus an empty cache (recordCount 0, saveData 0, refreshOnLoad 1, Excel 2016 version stamps). Excel rebuilds the cache and lays the pivot out the moment the workbook opens; verified through Excel COM (tests/excel-check-pivot.ps1) for row, column, filter, multi-data, outline and data-only layouts, including a pivot next to its own data and three pivots sharing one source. - Worksheet.AddPivotTable(name, sourceSheet, range, row, column, headers) and AddPivotTable(name, ExcelTable, row, column); PivotTable with AddRowField, AddColumnField, AddFilterField, AddDataField(function, displayName, numberFormat), StyleName, grand-total and compact/outline switches. - Parts: xl/pivotTables/pivotTableN.xml, xl/pivotCache/pivotCacheDefinitionN.xml and pivotCacheRecordsN.xml with their .rels; workbook <pivotCaches> after calcPr; content types registered; cacheId = part number, unique per pivot. - LoadResult now exposes Headers, Range and Table so a later sheet can pivot over loaded data; ExcelTable exposes its captured column names internally. - Data field captions equal to a source header are rejected up front (Excel fails the refresh with "field name already exists"). - Showcase gains a "Pivot" sheet; docs/pivot-tables.md explains the refresh-on-open design and the limits. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ady scans this repository Code scanning rejects SARIF from an advanced configuration while default setup is enabled, so the custom job could never upload. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…e fuzz seed The fuzz test's seed came from string.GetHashCode(), which is randomised per process, so each run explored different mutations; the CI run hit s="-1" and the reader indexed the style table at -1. The index is now range-checked on both sides, the seed is a stable FNV-1a hash so failures reproduce, and a direct regression test covers the case. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…instead of duplicating them Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
EPPlus 4.5.3.3 registers CodePagesEncodingProvider on load; ExcelDataReader .xls imports depend on it without saying so, and break once EPPlus is removed. Found during the LogisticaHQ migration. Also updates the list of what does not translate for pivots and edit mode. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ExcelReader.OpenAsync,ReadRowsAsync,ReadAsync<T>,ReadDataTableAsync, with cancellation between rows; non-seekable uploads are buffered asynchronously and spill to a temp file above 64 MB.Worksheet.CacheFormulaResults()opts a sheet into a small evaluator (+ - * / %,SUM,AVERAGE,COUNT,COUNTA,MIN,MAX,ROUND,ABSover cells already written), so viewers that do not calculate show totals. Anything it cannot prove is written without a value. The value store is paged so a far-away cell costs 8 KB, not an array sized to its row.Worksheet.AddPivotTableover a table or range written earlier; row, column, filter and data fields; refresh-on-open cache; verified against Excel 16 through COM automation (tests/excel-check-pivot.ps1).Test plan
dotnet build -c Release: 0 warningsdotnet test: 230 + 35 + 13 passing🤖 Generated with Claude Code