Skip to content

Some, but not all scripts report "load time" for stateless engines (time to download parquet and sync) #944

@alamb

Description

@alamb

While reviewing recent DataFusion results, I found there is now a "load time" reported for DataFusion

For example, this link

Image

Here is the source file that has 17:

It was added in 6d5ee0a which was part of a large refactoring in #860

Image

I am not sure if you intended the load_time to include the download / fsync time for the raw parquet dataset, but if you did I would request that we measure it consistently across stateless engines

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions