You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[C++][ORC] Writing struct<struct<…>> with a null inner struct shifts the child values #55618
Writing a nested struct to ORC loses data when the inner struct has a null but the outer struct has none. Every child value after the null row is shifted by one, and the first shifted value is written as 0 (or "" for strings).
Not affected: a top-level list or map<k, struct> with null elements, list<list<…>> and struct<list> with null lists, and any case where the outer struct also has a null in the batch. A round trip over a few thousand random nested schemas outside that shape found no other differences.
Seen on pyarrow 12.0.1 through 26.0.1 (Linux and Windows).
It looks like a struct's null mask is not passed on to its children when the enclosing struct reports no nulls, so the child gets an extra value for each null row. The writer side is WriteStructBatch in cpp/src/arrow/adapters/orc/util.cc.
Found while cross-checking the same files across several readers.
Hello,
I would like to work on this issue, if it is still open.
From what I have gathered, the presence of null values within nested structures can cause incorrect values in the ORC output, specifically affecting subsequent child values.
My plan is to first examine how the WriteStructBatch function (located in cpp/src/arrow/adapters/orc/util.cc) handles null values and child offsets, and then add regression tests covering these structures.
Before I begin, could you please let me know if anyone else is already working on this issue? Also, does my proposed approach seem reasonable to you?
if the issue is available i would be happy to be assigned to it...
Activity
emadleo13 commentedon Nov 17, 2026
Hello,
I would like to work on this issue, if it is still open.
From what I have gathered, the presence of
nullvalues within nested structures can cause incorrect values in the ORC output, specifically affecting subsequent child values.My plan is to first examine how the
WriteStructBatchfunction (located incpp/src/arrow/adapters/orc/util.cc) handlesnullvalues and child offsets, and then add regression tests covering these structures.Before I begin, could you please let me know if anyone else is already working on this issue? Also, does my proposed approach seem reasonable to you?
if the issue is available i would be happy to be assigned to it...