Skip to content

Refactor Parquet synthetic column helpers - #23759

Open
mhaseeb123 wants to merge 4 commits into
NVIDIA:mainfrom
mhaseeb123:codex/refactor-parquet-synthetic-column-helpers
Open

Refactor Parquet synthetic column helpers#23759
mhaseeb123 wants to merge 4 commits into
NVIDIA:mainfrom
mhaseeb123:codex/refactor-parquet-synthetic-column-helpers

Conversation

@mhaseeb123

@mhaseeb123 mhaseeb123 commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Description

Refactor only, no new code. This PR moves Parquet column synthesizer functions into a standalone reusable TU. Parquet readers include and call the helpers directly.

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@copy-pr-bot

copy-pr-bot Bot commented Aug 21, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@github-actions github-actions Bot added libcudf Affects libcudf (C++/CUDA) code. CMake CMake build issue labels Aug 21, 2026
@mhaseeb123
mhaseeb123 marked this pull request as ready for review August 21, 2026 19:33
@mhaseeb123
mhaseeb123 requested review from a team as code owners August 21, 2026 19:33
@mhaseeb123
mhaseeb123 requested review from bdice and vyasr August 21, 2026 19:33
@mhaseeb123 mhaseeb123 added 3 - Ready for Review Ready for review by team improvement Improvement / enhancement to an existing function non-breaking Non-breaking change 4 - Needs Review Waiting for reviewer to review or respond and removed 3 - Ready for Review Ready for review by team labels Aug 21, 2026
@coderabbitai

coderabbitai Bot commented Aug 21, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 56356ec4-9556-47f8-b4d6-3e13a5cd089b

📥 Commits

Reviewing files that changed from the base of the PR and between 7ae8669 and 6811f91.

📒 Files selected for processing (9)
  • cpp/CMakeLists.txt
  • cpp/src/io/parquet/experimental/hybrid_scan_impl.cpp
  • cpp/src/io/parquet/reader_impl.cpp
  • cpp/src/io/parquet/reader_impl.hpp
  • cpp/src/io/parquet/reader_impl_helpers.cpp
  • cpp/src/io/parquet/reader_impl_helpers.hpp
  • cpp/src/io/parquet/reader_impl_preprocess.cu
  • cpp/src/io/parquet/synthetic_column_helpers.cu
  • cpp/src/io/parquet/synthetic_column_helpers.hpp
💤 Files with no reviewable changes (3)
  • cpp/src/io/parquet/reader_impl_helpers.hpp
  • cpp/src/io/parquet/reader_impl.hpp
  • cpp/src/io/parquet/reader_impl_preprocess.cu
🚧 Files skipped from review as they are similar to previous changes (6)
  • cpp/src/io/parquet/experimental/hybrid_scan_impl.cpp
  • cpp/src/io/parquet/reader_impl_helpers.cpp
  • cpp/CMakeLists.txt
  • cpp/src/io/parquet/synthetic_column_helpers.hpp
  • cpp/src/io/parquet/reader_impl.cpp
  • cpp/src/io/parquet/synthetic_column_helpers.cu

Included review availability: Your plan provides up to 12 included reviews per hour; 9 remain after this review.


📝 Walkthrough

Summary by CodeRabbit

  • New Features
    • Improved Parquet reading support for generated row indices, source indices, and row-group indices.
    • Added support for selected row groups, multiple input sources, and empty results.
  • Bug Fixes
    • Improved index generation across concatenated Parquet inputs and row-group selections.
    • Added validation for row-group index inputs to improve reliability.
    • Improved handling of memory allocation and large-scale index generation.

Walkthrough

Changes

Parquet synthetic-column generation now uses shared CUDA helpers for row, source, and row-group indices. Reader-local implementations and declarations are removed. Reader and hybrid-scan paths pass row-group metadata to the new helpers, and CMake compiles the implementation.

Parquet synthetic columns

Layer / File(s) Summary
Define and implement synthetic-column helpers
cpp/src/io/parquet/synthetic_column_helpers.*
Adds documented helper interfaces and CUDA implementations for row-index mapping, source-index generation, and row-group indexing.
Integrate helpers into reader paths
cpp/src/io/parquet/reader_impl.*, cpp/src/io/parquet/experimental/hybrid_scan_impl.cpp, cpp/CMakeLists.txt
Updates reader call sites to use the shared helpers, removes the previous reader-local synthesis methods, and adds the CUDA implementation to the build target.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Merge Risk: ⚪ Minimal · up to 6811f

This refactor has no actionable merge-blocking risk identified at the current head and is merge-ready after normal checks and review.

Suggested reviewers: vyasr, bdice

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 16.67% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 6 functions across 5 files. (1 skipped: 1 unsupported.) Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly describes the main change: refactoring Parquet synthetic column helpers.
Description check ✅ Passed The description accurately relates the changes to moving Parquet column synthesizer functions into a reusable translation unit.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

4 - Needs Review Waiting for reviewer to review or respond CMake CMake build issue improvement Improvement / enhancement to an existing function libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant