Skip to content

[C++][Parquet] Writer cleanup can crash after dictionary hash table resize fails with OOM #55628

Open

Description

@fanruan-tuco

In Arrow 16.1.0, a Parquet Double dictionary write can fail with OOM, then crash with SIGSEGV during writer destruction.

Allocation failure — HashTable::Upsize() detaches the old buffer before allocating its replacement:

const Entry* old_entries = entries_;
ARROW_ASSIGN_OR_RAISE(auto previous, entries_builder_.FinishWithLength(capacity_));
RETURN_NOT_OK(UpsizeBuffer(new_capacity));

If entries_builder_.Resize() inside UpsizeBuffer() fails, it returns before updating entries_.

Release — FinishWithLength() resets the builder. On the error return, local previous is destroyed, releasing the old buffer through PoolBuffer::~PoolBuffer():

pool_->Free(ptr, capacity_, alignment_);

Use after free — entries_ still points to the freed allocation. The file writer destructor calls Close(), and ColumnWriterImpl::Close() still attempts to write the dictionary:

if (has_dictionary_ && !fallback_) {
  WriteDictionaryPage();
}

This reaches WriteDict() → CopyValues() → VisitEntries(), which dereferences the dangling pointer:

const auto& entry = entries_[i];

Expected: a failed write must not cause subsequent close/destruction to access freed memory or crash the process.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions