Repository navigation
Open
Labels
Description
Here's the current Project Antalya roadmap for 2026. This year the focus is on direct writes to Iceberg, improved usability of the swarms and Hybrid table engine, and interoperability with regard to cloud providers and catalog types. All features are open source--there are no hold-backs.
Please suggest additional features and ideas in the comments to this issue. We also welcome contributions.
Swarms
- Support distributed joins in swarm queries - #1527
- Query task distribution
- Distribute
INSERT … SELECTbetween remote-storage tables across swarm workers - #782 - Distribute compaction across swarm workers
- Remote swarm initiators
- Launch swarm workers as on-demand processes - #2319
- Drain swarm workers before scale-down with
SYSTEM STOP SWARM MODE- #1014; #759 - Choose local or swarm execution automatically based on query size - #2808; #2006
- Discover available swarm tables and dispatch queries automatically - #2036
Tiered storage and exports
- Export MergeTree data to Parquet and Iceberg
- Export MergeTree parts/partitions to Parquet - replicated tables #1124; non-replicated tables #2290 (#1908)
- Accept table functions as export destinations - #1320
- Export materialized and ALIAS columns - #1324
- Configure export filename patterns - #1490
- Export MergeTree parts/partitions directly to Iceberg - #1618
- Export all partitions in one operation - #1741
- Split Parquet output across multiple files - #489
- Retain partition-export history with Iceberg metadata and data file paths - #1917, #1832; #1824, #1330
- Control export data selection and schema mapping
- Support type conversion, schema matching, and compatible partition expressions during export - #1779; #1110, #1717
- Allow extra source columns with positional matching - #2111
- Match columns by name and allow optional source columns - #2220
- Allow compatible partition expressions without repartitioning - #2074; #2352
- Filter rows during export - #1849
- Export expired partitions to Iceberg using TTL rules - #2442; #1793
Hybrid table engine
- Hybrid reads and watermark management
Catalogs and cloud access
- Integrate Iceberg REST catalogs - #71542 (upstream)
- Map catalog namespaces as databases - #1337
- Integrate the Glue catalog - reads #77257 (upstream); writes #84136 (upstream); #900
- Integrate the BigLake catalog - #97104 (upstream)
- Read GCS-backed tables through DataLakeCatalog - #93866 (upstream)
Iceberg tables and schema
- Iceberg table operations
- Read Iceberg tables with files outside the table location or on another storage backend - #1092
- Iceberg schema evolution and types
- Read evolved Iceberg schemas and add, drop, or modify columns - reads #69445 (upstream); writes #85769 (upstream); #816
- Evolve complex-type schemas - #73714 (upstream)
- Apply schema evolution through catalogs with concurrency checks - #106102 (upstream)
- Read and write the Iceberg time type - #1761
- Use declared column defaults in Iceberg v3 schema evolution (
initial-default/write-default)
- Back up and restore Iceberg tables
Iceberg v3
- Create format-version 3 tables - #104849 (upstream)
- Read Iceberg v3 schemas and data types
- Read nanosecond timestamp types - #97132 (upstream)
- Read experimental geometry/geography types - #103113 (upstream)
- Read and write the
unknowntype - #2363 - Read the Iceberg
varianttype - #2677 - Read unshredded Parquet variants as Dynamic/JSON - #2666
- Read tables with multi-column transform metadata - #2318
- Iceberg deletion-vector reads
- Write deletion vectors for v3 mutations
- Read and write row-lineage metadata
Databricks and format interoperability
- Read Databricks deletion vectors stored in
.binfiles - #2271 - Read Databricks-managed and external Delta Lake/Iceberg tables through one Unity Catalog integration - #2607
- Read Databricks Liquid-clustered tables through the native Delta Lake reader with clustering-based file skipping - #2395; #2394
- Aggregate Apache-compatible HLL states - #2398
- Import/export raw Apache DataSketches HLL states and select the HLL estimator - #2544
- Persist ClickHouse aggregate-function states in Parquet/Iceberg - #2680; #2206
CAS (content-addressed storage)
- Share MergeTree object storage across servers with deduplication, replica relinking, garbage collection, and pool inspection - #2159
- Support MergeTree storage and data movement on CAS
- Reclaim unused CAS data with parallel and distributed garbage collection
- Back up and restore CAS tables using
clickhouse-backupembedded mode. - Run CAS on Azure/POSIX storage and through encrypted disks
- Support Azure Blob Storage
- Support local/shared POSIX filesystems
- Support CAS under the encrypted-disk wrapper
Performance and observability
- Parallelize Parquet/Iceberg reads and reduce remote requests
- Overlap reads and decoding while bounding remote I/O - #2275
- Skip constant column chunks using statistics - #2266
- Batch Bloom-filter reads across row groups - #2588
- Accelerate dictionary-encoded column decoding - #2750
- Accelerate dictionary filtering - #2735
- Read Parquet in parallel with independent prefetch and decoding - #2235
- Load DataLake object metadata on demand instead of issuing eager HEAD requests - #742
- Cache REST catalog vended storage credentials - #1923
- Prune Iceberg scans and answer partition aggregates from metadata
- Compact Iceberg manifests and data files - #2193, #2420
- Monitor catalog requests and CAS health, and inspect Iceberg layouts
Security
Token authentication and authorization
- Validate tokens on the server and enforce access controls
- Support client sign-in and token acquisition
Catalog and cloud authentication
Reacted by aadant, Korg95, Alexander Gedranovich, Gavin Y, Pierre-Louis Pécheux, Archibald BB, Cristhian Garcia , Chris Battarbee, hpcnt-stewart and Eva
Reacted by James Greenhill, Korg95, Henrik Jensen, Cristhian Garcia , hpcnt-stewart and Eva
Activity
Korg95 commentedon Mar 30, 2026
Hi team, wondering which quarter are we planning to implement iceberg v3 support.
ylw510 commentedon Aug 13, 2026
@svb-alt Hi.
Does the roadmap support external contributors for the planned features?