Skip to content

Match partition values to the partition spec in deserialize_data_file_from_json #3651

Open

Description

@mbutrovich

Is your feature request related to a problem or challenge?

When Manifest::parse_avro reads a DataFile, it first matches the partition record's fields to the partition type by name (project_partition_by_name). deserialize_data_file_from_json skips that step. The record conversion then builds the struct in the JSON record's field order, with one value per JSON field. A record whose fields differ in order or number from the partition spec gives a struct that doesn't line up with the spec's fields.

Describe the solution you'd like

Call project_partition_by_name on the JSON path too, so both paths size and order the partition struct by the partition spec. This came out of the #3354 work and is listed in #3262.

Willingness to contribute

I can contribute to this feature independently

Activity

  1. self-assigned this
    on Nov 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions