Skip to content

Decouple Hive from Parquet Writer options #4

@yingsu00

Description

@yingsu00

Description

Current Parquet Writer options have Hive specific options:

static constexpr const char* kParquetHiveConnectorEnableDictionary =
"hive.parquet.writer.enable-dictionary";
static constexpr const char* kParquetHiveConnectorDictionaryPageSizeLimit =
"hive.parquet.writer.dictionary-page-size-limit";
static constexpr const char* kParquetHiveConnectorWriteBatchSize =
"hive.parquet.writer.batch-size";

These names are confusing. The writer options are not Hive-specific. File Writer are in different level from connectors in a structural hierarchy. These names need to be corrected to names without "HiveConnector"

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions