Repository navigation
Workbooks and sheets replace the current-sheet reader, across the library, C ABI and bindings (v6.0.0) - #136
Merged
Merged
Conversation
XlsxReader, XlsbReader and XlsReader become XlsxWorkbook, XlsbWorkbook and XlsWorkbook. The internal XLSB parsing helper that held the name becomes XlsbWorkbookParts. No member or behaviour changes. The old names are recorded as removed in PublicAPI.Unshipped.txt. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Add IExcelWorkbook, IExcelSheet, ExcelSheetList<TSheet> and the XLSX, XLSB, XLS and CSV sheet structs, with Sheets, FirstSheet and TryGetSheet on each workbook. The v5 reader interfaces inherit the new ones so callers can move over file by file; nothing is removed yet. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Adds sheet overloads next to the reader ones: ExcelParser<T>.Parse, Excel.InferSchema, ExcelDataReader and ToArrowRecordBatch. ExcelEnumerable and AsyncRowEnumerator now constrain to IExcelSheet<TEnumerator>; CsvEnumerable holds a CsvSheet. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…a reader CliCommands.Open returns the workbook plus the selected sheet; the write and schema paths take IExcelSheet. The parallel-CSV internals open the CSV sheet directly through OpenSheet. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
NativeHandle now holds an IExcelWorkbook and its own current sheet index; the exported C functions keep their status codes and messages. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… AOT check Benchmarks, the fuzz harness, FuzzTests and the AOT sanity app obtain a sheet from the workbook (FirstSheet / SheetAt) instead of enumerating a reader with a current sheet. The differential harness is typed over IExcelWorkbook. Sheet lookup stays inside every measured method. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Move the tests directly under tests/ExcelReader.Tests/Reader off the v5 reader-with-a-current-sheet members: enumerators, parsers, schema inference and ExcelDataReader take a sheet, navigation tests assert the v6 workbook and sheet equivalents. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Csv, Xls, Xlsb, Xlsx and Internal reader tests read through Sheets, FirstSheet, SheetAt and TryGetSheet instead of the reader's current sheet. The dispose tests assert on the workbook's Sheets and on a sheet taken before the dispose. The CSV dialect helper drops its v5 overloads. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…oks and sheets The v5 "reader with a current sheet" surface is gone; only the workbook/sheet API remains. - IExcelRowReader, IExcelRowReader<TEnumerator> -> IExcelWorkbook (the workbook) or IExcelSheet / IExcelSheet<TEnumerator> (one sheet) - ExcelRowReaderExtensions.Sheets(), ExcelSheet -> for n in [0, SheetCount): workbook.SheetAt(n), or foreach over workbook.Sheets - MoveToSheet(int) -> Sheets[n] / SheetAt(n) - TryMoveToSheet(name) -> TryGetSheet(name, out sheet) - SheetName, SheetNameAt(n) -> sheet.Name - SheetVisibility, SheetVisibilityAt(n) -> sheet.Visibility - workbook GetEnumerator() / GetAsyncEnumerator(ct) -> sheet.GetEnumerator() / sheet.GetAsyncEnumerator(ct) - Excel.Open / OpenAsync returning IExcelRowReader -> returning IExcelWorkbook / ValueTask<IExcelWorkbook> - Excel.InferSchema(IExcelRowReader, ...) -> Excel.InferSchema(IExcelSheet, ...) - ExcelParser<T>.Parse(XlsxWorkbook | XlsWorkbook | XlsbWorkbook | IExcelRowReader | CsvReader) -> Parse(XlsxSheet | XlsSheet | XlsbSheet | IExcelSheet | CsvSheet) - ExcelDataReader(IExcelRowReader, int) -> ExcelDataReader(IExcelSheet, int) - ToArrowRecordBatch(this IExcelRowReader, ...) -> ToArrowRecordBatch(this IExcelSheet, ...) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…verloads Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The concurrency theories now run over memory, path, FileStream, a seekable stream and a non-seekable stream. A new theory obtains sheets by index and by name and opens their enumerators from twelve threads at once, racing the first shared-string load. A fact checks that a disposed workbook refuses a new enumerator while a live one reads to the end. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e notes The quick start, the feature notes and the reader table now use the v6 workbook and sheet types, ARCHITECTURE.md gains a paragraph on the workbook, sheet and enumerator split and the reference-counted lifetime, and the style guide example names a member that still exists. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Re-wrap the reflowed open paragraph, align the reader table and list XlsbWorkbookParts.cs in the partial-class layout. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The dispose-while-live check now drains the live enumerator and compares every cell, shared strings included, with a sequential read, and runs over memory, FileStream and a seekable stream, where an early release of the zip or the shared table would break the read. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…workbook Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
rust/.cargo/config.toml points EXCELREADER_NATIVE_LIB_DIR at a local build that the publish job never makes, so the package verification failed with "no .so file found" and excelreader 5.2.0 never reached crates.io. Remove the override in that job so the verification downloads the native library from the release, as users do. Let the derive step skip a version that is already published, so the job can be re-run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rewrite the reading, parsing, CSV, encryption and writing guides for the workbook and sheet API, and add a v5 to v6 migration guide. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each read opens its own enumerator, so reads on one workbook handle no longer invalidate one another and the live-session interlock is removed. xl_parse_typed_ex, xl_parse_arrow_ex and xl_infer_schema_ex fold into the unsuffixed functions. XL_ABI_VERSION is 6. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
xl_rows_open returns a cursor with its own enumerator, so several cursors read one workbook at once and survive xl_close. The workbook handle no longer has a current sheet: xl_move_to_sheet and xl_sheet_name are removed, and xl_sheet_visibility_at and xl_sheet_index are added. The struct filled by the decoded read-all is renamed xl_rows_decoded. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…en functions xl_open_file and xl_open_memory take the options pointer; NULL means the defaults. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Cursors and typed parses over all sheets of one handle match a sequential read in XLSX, XLSB and XLS, and a cursor reads to the end after its workbook id is closed. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
…t to v6 A reading call now clears the thread's error first, so a non-OK status never shows an earlier call's message; xl_sheet_name_at reports a message for a bad index like every other function. The C smoke test binds the v6 exports. The parallel tests assert per-sheet content, not only row counts. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
xl::Workbook hands out xl::Sheet values; rows(), read_all_decoded() and infer_schema() move to the sheet, and parse_sheet, typed_reader, parse_arrow and arrow_stream take one. xl::RowCursor owns its native cursor. move_to_sheet and the index-less sheet_name are removed. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Workbook hands out Sheet values; rows(), read_all_*, typed_chunks, infer_schema, parse_sheet and the Arrow functions take a sheet and open their own native cursor. move_to_sheet and the index-less sheet_name are removed. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Workbook.sheets is a sequence of Sheet objects, looked up by index, slice or name. The read methods move from Workbook to Sheet and each opens its own native cursor. move_to_sheet, sheet_name and the sheets() method are removed. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The v5 versions asserted that a fresh reader could start, which proved the workbook's single cursor slot was free. That slot is gone, so a leaked reader or stream passed. They now spy on xl_typed_reader_close and on the Arrow stream release, and the parse_typed test keeps a row cursor alive while it runs. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
DecodedRows and TableView own plain native allocations, so they are Send and Sync and can be returned from scoped threads. Workbook::sheet takes its error codes from the native side. The migration guide says read-alls now start from the first row. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…drop jthread RowCursor::read_all_decoded returns what the cursor has left; the sheet form reads the whole sheet, and the migration guide says so. The first ABI check runs one small read so later threads do not race the runtime's start-up. The sheets test uses std::thread for libc++. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…arm the runtime once Sheets.__getitem__ is overloaded so sheets[i] types as a Sheet, and two Sheet objects for the same workbook and index compare equal. The first library load runs one small read on one thread so later threads do not race the runtime's start-up. ArrowStream no longer holds the workbook. The migration guide says reads now start from the first row. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Workbook.close() takes the handle under a lock, so two threads closing the same workbook call xl_close once and neither raises. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Sheet::infer_schema_parse_text passes XL_INFER_PARSE_TEXT, and parse_sheet_parallel / parse_arrow_parallel pass a degree of parallelism, matching what the Python and C++ bindings expose. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Park each closing thread at its first read of the handle so both pass the is-None check; fails every time on the unlocked close, passes with the lock. Drops the stub test, which also passed on the old code. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The sources already compile against v6; the compare option failed to configure because xlsxio's C sources need the C language enabled. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The benchmark hardcoded format="xlsb", so a path of any other format failed. open_workbook now infers the format from the path; the CSV legs keep their explicit format. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The profile predated the v6 row cursor and sheet-index signatures, so ILC found no data for ReadApi.NextRow/NextRowView or the typed entry points. Python, XLSX 65K, alternated old/new/old/new: read_all -1.1%, rows -0.9%, read_all_columnar -1.2%, to_polars -3.5%; CSV parse_typed and to_polars flat. Rust and C++ on XLSB (unprofiled by design): within noise. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
xlnt 1.5.0 and zsv declare a CMake minimum that CMake 4 no longer accepts; raise the policy minimum for those fetched projects. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #136 +/- ##
==========================================
+ Coverage 89.88% 90.03% +0.15%
==========================================
Files 185 190 +5
Lines 13911 13963 +52
Branches 2550 2558 +8
==========================================
+ Hits 12504 12572 +68
+ Misses 941 933 -8
+ Partials 466 458 -8 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Benchmark ResultsBaseline: master Moved by at least 10%
64 within ±10%. Full results per groupExcelReader.Benchmarks.ArrowConversionBenchmark
ExcelReader.Benchmarks.ChunkedParseBenchmark
ExcelReader.Benchmarks.ColdStartBenchmark
ExcelReader.Benchmarks.CsvParallelParseBenchmark
ExcelReader.Benchmarks.CsvParallelVsSepBenchmark
ExcelReader.Benchmarks.CsvParseBenchmark
ExcelReader.Benchmarks.CsvReadBenchmark
ExcelReader.Benchmarks.CsvWriteBenchmark
ExcelReader.Benchmarks.DataReaderBenchmark
ExcelReader.Benchmarks.EncryptedWorkbookBenchmark
ExcelReader.Benchmarks.NativeRowReadBenchmark
ExcelReader.Benchmarks.NativeTypedParseBenchmark
ExcelReader.Benchmarks.ParseBenchmark
ExcelReader.Benchmarks.ReadBenchmark
ExcelReader.Benchmarks.RealDataReadBenchmark
ExcelReader.Benchmarks.RealDataTypedParseBenchmark
ExcelReader.Benchmarks.RecordWriteBenchmark
ExcelReader.Benchmarks.StringHeavyReadBenchmark
ExcelReader.Benchmarks.WriteBenchmark
ExcelReader.Benchmarks.WritePathBenchmark
ExcelReader.Benchmarks.XlsReadBenchmark
ExcelReader.Benchmarks.XlsWriteBenchmark
ExcelReader.Benchmarks.XlsxSharedStringHotPathBenchmark
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A v5 reader owned one "current sheet" that callers moved with
MoveToSheet. In v6 the reader is aworkbook that hands out sheets, and each sheet is read on its own, so the sheets of one workbook can
be read in parallel. The change runs through the C# library, the C ABI (now version 6) and the
Python, Rust and C++ bindings. This is a breaking release;
docs/guide/migrating-to-v6.mdmapsevery v5 name to its v6 replacement.
XlsxReader,XlsbReaderandXlsReaderbecomeXlsxWorkbook,XlsbWorkbookandXlsWorkbook.IExcelRowReadersplits intoIExcelWorkbookandIExcelSheet.A sheet (
XlsxSheet,XlsbSheet,XlsSheet,CsvSheet) is areadonly structmade of a workbookand an index, and it holds no resources.
MoveToSheet,SheetNameandSheets()give way toworkbook.Sheets[i],SheetAt,TryGetSheet,sheet.Nameandsheet.Visibility.Excel.Openreturns anIExcelWorkbook.ExcelParser<T>.Parse,Excel.InferSchema,ExcelDataReaderandToArrowRecordBatchtake a sheet. The CLI, benchmarks, fuzz harness andtests read through sheets.
XL_ABI_VERSIONis 6, with 50 exports. A workbook handle has no current sheet, andevery read takes a zero-based sheet index right after the handle. Rows come from a cursor handle:
xl_rows_open,xl_rows_next,xl_rows_next_view,xl_rows_read_all_blob,xl_rows_read_all_decodedandxl_rows_close. Each_exexport is folded into its base export:xl_open_file/xl_open_memorytake an options pointer,xl_parse_typed/xl_parse_arrowtake adegree of parallelism, and
xl_infer_schematakes flags. New:xl_sheet_visibility_atandxl_sheet_index. Typed readers, Arrow streams and cursors are independent of each other and keepworking after
xl_close. Handle ids are never reused, and each is checked against its type.Reading functions clear
xl_last_erroron entry, so after a non-OK status the message belongs tothat call.
workbook.sheets[i]orworkbook.sheets["Name"], which raisesKeyErroron a miss.Sheethas value equality.Workbook.close()is safe to call from severalthreads at once. Each
rows(),read_all()andread_all_columnar()call starts at the sheet'sfirst row.
workbook.sheet(i)?returns aCopySheet<'w>.RowCursorowns a native cursor.Workbook,RowCursor,TypedChunks,DecodedRowsandTableViewareSend + Sync, andWorkbookno longer needsmut. New:Sheet::infer_schema_parse_text,parse_sheet_paralleland
arrow::parse_arrow_parallel.xl::Sheet.sheet.rows()returns a move-onlyxl::RowCursor, andRowCursor::read_all_decoded()reads what the cursor has left. The comparison benchmarks buildagainst the v6 wrapper and configure under CMake 4.
bindings now run a single-threaded warm-up inside their one-time ABI check.
excelreader.mibcis regenerated for the v6 entry points. In an alternatingold/new A/B on a 65K-row XLSX from Python,
to_polarsis 3.5% faster and the row andread_allpaths are 0.9 to 1.2% faster. CSV and XLSB are flat; XLSB is left out of the profile on purpose.
Behavior changes
xl_parse_typedandxl_parse_arrowused to read sequentially. Pass1as the degree ofparallelism to keep that;
0uses every processor on a CSV.at once, and none interferes with another read.
read_all_blob/read_all_decodedon a sheet read the whole sheet; they no longer continue acursor. Use the cursor's own method for its remainder.
Test plan
dotnet test(Core, Native, Arrow, CLI)tests/ExcelReader.NativeSmoke)pytest python/testscargo testinrust/excelreadercpp/tests)