Skip to content

[Java] Check IPC and C Data counts before using them, and release message bodies on errors #1462

Open

Description

@jbonofre

Describe the bug

Follow-up to #1421 / #1422. A few other values read from IPC and C Data input are used before they are checked, and some error paths leak the message body.

  • C Data: ArrayImporter reads n_children and n_buffers child and buffer pointers before comparing those counts with what the type expects. A huge count sizes a long[] from it, which throws OutOfMemoryError (an Error, so it is not wrapped like other import failures) or reads past the end of the pointer array.
  • IPC: MessageSerializer.deserializeRecordBatch copies variadicBufferCounts without checking them. VectorLoader casts each count to int and uses it to size an ArrayList, so a count like 1_000_000_000 allocates a huge list, and a count above INT_MAX is truncated and assigns buffers to the wrong fields.
  • IPC leaks:
    • deserializeMessageBatch throws after reading the body, without releasing it, when the message is too large, has an unsupported metadata version, or has an unexpected header type.
    • The ArrowBlock variants of deserializeRecordBatch and deserializeDictionaryBatch leak the block buffer when the read is short or the metadata is malformed.
  • JSON: JsonFileReader.read() calls setRowCount(count) before reading the columns, so the vectors are first allocated for the batch count even if the columns are then rejected.

Expected behavior

Counts are checked before they size an allocation or a read. Rejected messages release everything they allocated.

Component(s)

Java

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions