Skip to content

Parquet CLI drops leading --conf key=value options #3969

Open

Description

@zhangxinyao88

Create an Avro file with a field of type string and logical type uuid. Set parquet.avro.write-parquet-uuid=false in writer.properties, then run:

parquet --conf parquet.avro.write-parquet-uuid=true --config-file writer.properties convert --output output.parquet input.avro

The command can succeed while writing the UUID field as a string. The explicit property should override the configuration file and produce a Parquet UUID field backed by FIXED_LEN_BYTE_ARRAY. Hadoop's generic option parser consumes the leading --conf before the Parquet argument parser receives it.

PR #3866 includes a regression test that launches the CLI and checks the written schema. Issue #3289 tracks broader CLI test coverage.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions