spark4

Here are 2 public repositories matching this topic...

alvertogit / bigdata_docker

Big Data Docker Data Science Spark Spark4 Hadoop HDFS Scala Python Artificial Intelligence Machine Learning Jupyter Lab Notebook

python docker data-science machine-learning scala big-data spark jupyter-notebook spark4 jupyter-lab

Updated Aug 16, 2026
Python

sivananda1995 / spark-skew-doctor

Star

Diagnoses Spark data skew from the job's own event log - no cluster, no JVM - and tells key skew apart from a straggler by comparing task time against bytes read. Then measures the fixes against each other on real Spark 4: across 16 cells, salting won 0. Broadcast 6.5x, AQE 1.5x, salting 0.81x.

python benchmarking big-data apache-spark distributed-computing pyspark data-engineering performance-tuning profiling shuffle query-optimization observability salting spark4 broadcast-join adaptive-query-execution data-skew spark-event-log

Updated Aug 19, 2026
Python

Improve this page

Add a description, image, and links to the spark4 topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the spark4 topic, visit your repo's landing page and select "manage topics."

Learn more