Please leave a comment and mark on the TODO tests in this issue if you want to contribute
Backend
VL(Velox)
Bug Description
These suites were commented out in #11512 with TODO markers indicating they need to be fixed and re-enabled.
Context
The test suites below have been disabled due to various failures. Each table shows the status for both Spark 4.0 and 4.1:
- 🔴 = Suite is commented out (disabled)
- 🟢 = Suite is enabled
The failure count (when available) is shown in the status column.
org.apache.spark.sql
| Suite |
Spark 4.0 |
Spark 4.1 |
Owner |
Comments |
GlutenDataFrameSubquerySuite |
🟢 |
🟢 |
@baibaichen |
Fixed in #11727 |
GlutenExplainSuite |
🔴 (1 failures) |
🔴 (1 failures) |
@Surbhi-Vijay |
|
GlutenLogQuerySuite |
🔴 (2 failures) |
🔴 (2 failures) |
@kapilks |
|
GlutenSetCommandSuite |
🔴 (1 failures) |
🔴 (1 failures) |
@Surbhi-Vijay |
|
GlutenSingleLevelAggregateHashMapSuite |
🔴 (1 failures) |
🔴 (1 failures) |
@kapilks |
|
GlutenSparkSessionJobTaggingAndCancellationSuite |
🔴 (1 failures) |
🔴 (1 failures) |
|
|
GlutenTwoLevelAggregateHashMapSuite |
🔴 (1 failures) |
🔴 (1 failures) |
@kapilks |
|
GlutenTwoLevelAggregateHashMapWithVectorizedMapSuite |
🔴 (1 failures) |
🔴 (1 failures) |
@kapilks |
|
GlutenVariantEndToEndSuite |
🟢 |
🟢 |
@baibaichen |
Fixed in #11726 |
GlutenVariantShreddingSuite |
🟢 |
🟢 |
@baibaichen |
Fixed in #11726 |
GlutenXmlFunctionsSuite |
🟢 |
🟢 |
@baibaichen |
Fixed in #11725 |
catalyst.expressions
| Suite |
Spark 4.0 |
Spark 4.1 |
Owner |
Comments |
GlutenXmlExpressionsSuite |
🟢 |
🟢 |
@baibaichen |
Fixed in #11580 |
connector
| Suite |
Spark 4.0 |
Spark 4.1 |
Owner |
Comments |
GlutenGroupBasedUpdateTableSuite |
🔴 (1 failures) |
🔴 (1 failures) |
@Surbhi-Vijay |
|
GlutenMergeIntoDataFrameSuite |
🔴 (1 failures) |
🔴 (1 failures) |
|
|
execution
| Suite |
Spark 4.0 |
Spark 4.1 |
Owner |
Comments |
GlutenInsertSortForLimitAndOffsetSuite |
🔴 (6 failures) |
🔴 (6 failures) |
|
|
GlutenPlannerSuite |
🔴 (1 failures) |
🔴 (1 failures) |
|
|
GlutenRemoveRedundantSortsSuite |
🔴 (1 failures) |
🔴 (1 failures) |
|
|
GlutenSparkPlanSuite |
🔴 (1 failures) |
🔴 (1 failures) |
@Surbhi-Vijay |
|
GlutenWholeStageCodegenSparkSubmitSuite |
🔴 (1 failures) |
🔴 (1 failures) |
|
|
GlutenWholeStageCodegenSuite |
🔴 (24 failures) |
🔴 (24 failures) |
@Surbhi-Vijay |
|
execution.joins
| Suite |
Spark 4.0 |
Spark 4.1 |
Owner |
Comments |
GlutenSingleJoinSuite |
🟢 |
🟢 |
@baibaichen |
Fixed in #11577 |
execution.datasources.parquet
| Suite |
Spark 4.0 |
Spark 4.1 |
Owner |
Comments |
GlutenParquetTypeWideningSuite |
🔴 (74 failures) |
🔴 (74 failures) |
@baibaichen |
track in #11683 |
GlutenParquetVariantShreddingSuite |
🟢 |
🟢 |
@baibaichen |
Fixed in #11726 |
execution.datasources.text
| Suite |
Spark 4.0 |
Spark 4.1 |
Owner |
Comments |
GlutenWholeTextFileV1Suite |
🔴 (1 failures) |
🔴 (1 failures) |
|
|
GlutenWholeTextFileV2Suite |
🔴 (1 failures) |
🔴 (1 failures) |
|
|
execution.python
| Suite |
Spark 4.0 |
Spark 4.1 |
Owner |
Comments |
GlutenPythonDataSourceSuite |
🔴 (1 failures) |
🔴 |
|
|
GlutenPythonUDFSuite |
🔴 (1 failures) |
🔴 |
|
|
GlutenPythonUDTFSuite |
🔴 |
🔴 |
|
|
GlutenRowQueueSuite |
🟢 |
🔴 |
|
Disabled in Spark 4.1 |
sources
| Suite |
Spark 4.0 |
Spark 4.1 |
Owner |
Comments |
GlutenBucketedReadWithHiveSupportSuite |
🔴 (2 failures) |
🔴 (2 failures) |
@Surbhi-Vijay |
|
GlutenBucketedWriteWithHiveSupportSuite |
🔴 (1 failures) |
🔴 (2 failures) |
|
Failure count changed: 1 → 2 |
GlutenCommitFailureTestRelationSuite |
🔴 (2 failures) |
🔴 (2 failures) |
|
|
GlutenDisableUnnecessaryBucketedScanWithHiveSupportSuite |
🔴 (2 failures) |
🔴 (2 failures) |
@Surbhi-Vijay |
|
GlutenJsonHadoopFsRelationSuite |
�� (2 failures) |
🔴 (2 failures) |
|
|
GlutenParquetHadoopFsRelationSuite |
🔴 (2 failures) |
🔴 (2 failures) |
|
|
GlutenSimpleTextHadoopFsRelationSuite |
🔴 (2 failures) |
🔴 (2 failures) |
|
|
streaming
Most of failtures are same, it seems should be fixed in spark
| Suite |
Spark 4.0 |
Spark 4.1 |
Owner |
Comments |
GlutenEventTimeWatermarkSuite |
🔴 |
🔴 |
|
|
GlutenFileStreamSourceSuite |
🔴 |
🔴 |
|
|
GlutenFlatMapGroupsWithStateDistributionSuite |
🔴 |
🔴 |
|
|
GlutenFlatMapGroupsWithStateSuite |
🔴 |
🔴 |
|
|
GlutenRocksDBStateStoreFlatMapGroupsWithStateSuite |
🔴 |
🔴 |
|
|
GlutenRocksDBStateStoreStreamingAggregationSuite |
🔴 |
🔴 |
|
|
GlutenRocksDBStateStoreStreamingDeduplicationSuite |
🔴 |
🔴 |
|
|
GlutenStreamSuite |
🔴 |
🔴 |
|
|
GlutenStreamingAggregationDistributionSuite |
🔴 |
🔴 |
|
|
GlutenStreamingAggregationSuite |
🔴 |
🔴 |
|
|
GlutenStreamingDeduplicationDistributionSuite |
🔴 |
🔴 |
|
|
GlutenStreamingDeduplicationSuite |
🔴 |
🔴 |
|
|
GlutenStreamingInnerJoinSuite |
🔴 |
🔴 |
|
|
GlutenStreamingOuterJoinSuite |
🔴 |
🔴 |
|
|
GlutenStreamingSessionWindowDistributionSuite |
🔴 |
🔴 |
|
|
GlutenStreamingStateStoreFormatCompatibilitySuite |
🔴 |
🔴 |
|
|
Summary Statistics
- Total suites disabled in Spark 4.0: 48
- Total suites disabled in Spark 4.1: 51
- Unique suites across both versions: 51
This issue was generated based on commit 4e3df67.
