Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion docs/source/contributor-guide/memory_management.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,10 @@ share (`SharedDecode`), and `CometSparkToColumnarExec`. These are real off-heap
container RSS that neither Spark's `TaskMemoryManager` nor Comet's native memory pool sees. In
practice the volume is modest, a batch at a time per stream, but there is no ceiling and no
backpressure. A broadcast is the largest case: a task that reads one holds all of it decoded, and
the tasks of an executor that read it at the same time share one such copy. The executor's memory
the tasks of an executor that read it at the same time share one such copy. The executor keeps that
copy for `spark.comet.exec.broadcastExchange.shareDecoded.idleTimeout` after the last of those
tasks completes, so that the next wave of tasks reads it too, and caps the idle copies of all
broadcasts at `spark.comet.exec.broadcastExchange.shareDecoded.maxIdleSize`. The executor's memory
usage log reports the allocator's total, and the part of it charged to the import allocator
described below, so that the overhead can be sized for it.

Expand Down
16 changes: 11 additions & 5 deletions docs/source/user-guide/latest/tuning/operators.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,11 +63,17 @@ Extending reduction to those joins or across small batches is separate follow-up
Comet broadcasts the build side of a broadcast join as Arrow IPC batches, which each task that reads the broadcast
decodes before building its hash table. By default, the tasks of an executor that read a broadcast at the same time
share one decoded copy of it. The first of them decodes the broadcast, and the others wait for that decode and read the
same buffers. The copy is freed when the last of these tasks completes, so a task that starts after that decodes the
broadcast again. Each task still builds its own hash table.

Set `spark.comet.exec.broadcastExchange.shareDecoded.enabled=false` to have every task decode the whole broadcast for
itself.
same buffers. When the last of these tasks completes, the executor keeps the copy for
`spark.comet.exec.broadcastExchange.shareDecoded.idleTimeout`, 1 second by default, so that a task that starts by then,
such as one of the next wave of tasks of the same stage, reads it instead of decoding the broadcast again. A task that
starts after that decodes the broadcast again. Each task still builds its own hash table.

The idle copy is JVM-side Arrow memory that neither Spark nor Comet's memory pools account for. The idle copies of all
broadcasts on an executor are capped by `spark.comet.exec.broadcastExchange.shareDecoded.maxIdleSize`, 128 MiB by
default. A copy larger than the cap is freed as soon as no task holds it, and keeping a copy frees the copies that have
been idle longest until the rest fit. Raise the cap to keep larger broadcasts between waves, if the executor's memory
overhead has room for them. Set the timeout to `0` to free the copy as soon as no task holds it. Set `spark.comet.exec.broadcastExchange.shareDecoded.enabled=false` to
have every task decode the whole broadcast for itself.

#### Broadcast compression

Expand Down
36 changes: 33 additions & 3 deletions spark/src/main/scala/org/apache/comet/CometConf.scala
Original file line number Diff line number Diff line change
Expand Up @@ -696,12 +696,42 @@ object CometConf extends ShimCometConf {
.doc(
"Whether the tasks of an executor that read a Comet broadcast at the same time share " +
"one decoded copy of it. The first of them decodes the broadcast, and the others " +
"wait for it instead of decoding it themselves. The copy is freed when the last of " +
"them completes. When disabled, every task decodes the whole broadcast for itself. " +
s"$TUNING_GUIDE.")
"wait for it instead of decoding it themselves. The copy is kept for " +
s"$COMET_EXEC_CONFIG_PREFIX.broadcastExchange.shareDecoded.idleTimeout after the last " +
"of them completes, for the tasks that start by then. When disabled, every task " +
s"decodes the whole broadcast for itself. $TUNING_GUIDE.")
.booleanConf
.createWithDefault(true)

val COMET_EXEC_BROADCAST_SHARE_DECODED_IDLE_TIMEOUT: ConfigEntry[Long] =
conf(s"$COMET_EXEC_CONFIG_PREFIX.broadcastExchange.shareDecoded.idleTimeout")
.category(CATEGORY_EXEC)
.doc(
"How long an executor keeps the shared decoded copy of a Comet broadcast after the " +
"last task reading it completes, when " +
s"${COMET_EXEC_BROADCAST_SHARE_DECODED_ENABLED.key} is true. A task that starts by " +
"then, such as one of the next wave of tasks of the same stage, reads the copy " +
"instead of decoding the broadcast again. The idle copy is JVM-side Arrow memory " +
"that neither Spark nor Comet's memory pools account for, and " +
s"$COMET_EXEC_CONFIG_PREFIX.broadcastExchange.shareDecoded.maxIdleSize caps it. Set " +
s"to 0 to free the copy as soon as no task holds it. $TUNING_GUIDE.")
.timeConf(TimeUnit.MILLISECONDS)
.checkValue(_ >= 0, "The idle timeout of a shared broadcast decode must not be negative")
.createWithDefault(TimeUnit.SECONDS.toMillis(1))

val COMET_EXEC_BROADCAST_SHARE_DECODED_MAX_IDLE_SIZE: ConfigEntry[Long] =
conf(s"$COMET_EXEC_CONFIG_PREFIX.broadcastExchange.shareDecoded.maxIdleSize")
.category(CATEGORY_EXEC)
.doc(
"The most bytes of idle shared decoded broadcast copies that an executor keeps, over " +
"all broadcasts, under " +
s"${COMET_EXEC_BROADCAST_SHARE_DECODED_IDLE_TIMEOUT.key}. A copy larger than this " +
"is freed as soon as no task holds it. Keeping a copy frees the copies that have " +
s"been idle longest until the rest fit. $TUNING_GUIDE.")
.bytesConf(ByteUnit.BYTE)
.checkValue(_ >= 0, "The idle size of shared broadcast decodes must not be negative")
.createWithDefault(128L * 1024 * 1024)

val COMET_BROADCAST_PRODUCER_COALESCE_ENABLED: ConfigEntry[Boolean] =
conf("spark.comet.exec.broadcastExchange.producerCoalesce.enabled")
.category(CATEGORY_EXEC)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -230,7 +230,12 @@ case class CometBroadcastExchangeExec(
override protected def doExecuteColumnar(): RDD[ColumnarBatch] = {
val broadcasted = executeBroadcast[CometBroadcastValue]()

new CometBatchRDD(sparkContext, getNumPartitions(), broadcasted, shareDecoded)
new CometBatchRDD(
sparkContext,
getNumPartitions(),
broadcasted,
shareDecoded,
shareDecodedRetention)
}

// After https://issues.apache.org/jira/browse/SPARK-48195, Spark plan will cache created RDD.
Expand All @@ -242,12 +247,22 @@ case class CometBroadcastExchangeExec(
}

val broadcasted = executeBroadcast[CometBroadcastValue]()
new CometBatchRDD(sparkContext, numPartitions, broadcasted, shareDecoded)
new CometBatchRDD(
sparkContext,
numPartitions,
broadcasted,
shareDecoded,
shareDecodedRetention)
}

private def shareDecoded: Boolean =
CometConf.COMET_EXEC_BROADCAST_SHARE_DECODED_ENABLED.get(conf)

private def shareDecodedRetention: SharedDecode.Retention =
SharedDecode.Retention(
CometConf.COMET_EXEC_BROADCAST_SHARE_DECODED_IDLE_TIMEOUT.get(conf),
CometConf.COMET_EXEC_BROADCAST_SHARE_DECODED_MAX_IDLE_SIZE.get(conf))

override protected[sql] def doExecuteBroadcast[T](): broadcast.Broadcast[T] = {
try {
relationFuture.get(timeout, TimeUnit.SECONDS).asInstanceOf[broadcast.Broadcast[T]]
Expand Down Expand Up @@ -310,12 +325,16 @@ object CometBroadcastExchangeExec extends CometSink[BroadcastExchangeExec] {
* @param shareDecoded
* whether the tasks of an executor that read the broadcast at the same time share one decoded
* copy of it, through [[CometBroadcastValue]]
* @param retention
* how long an executor keeps that copy after the last task reading it completes, and the cap on
* the bytes of the idle copies that it keeps
*/
class CometBatchRDD(
sc: SparkContext,
val numPartitions: Int,
value: broadcast.Broadcast[CometBroadcastValue],
shareDecoded: Boolean)
shareDecoded: Boolean,
retention: SharedDecode.Retention)
extends RDD[ColumnarBatch](sc, Nil) {

override def getPartitions: Array[Partition] = (0 until numPartitions).toArray.map { i =>
Expand All @@ -326,7 +345,7 @@ class CometBatchRDD(
val partition = split.asInstanceOf[CometBatchPartition]
val source = this.getClass.getSimpleName
if (shareDecoded && context != null) {
partition.value.value.batches(context, source)
partition.value.value.batches(context, source, retention)
} else {
partition.value.value.chunks.iterator.flatMap(Utils.decodeBatches(_, source))
}
Expand Down
Loading