Skip to content

[Data] Keep a map task's buffered rows when max_errored_blocks skips its error; read past corrupt MCAP chunks - #67167

Merged
richardliaw merged 13 commits into
masterfrom
marwan/mcap-dsv2-errored-rows
Nov 14, 2026
Merged

richardliaw merged 13 commits into
masterfrom
marwan/mcap-dsv2-errored-rows

Conversation

@marwan116

@marwan116 marwan116 commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

How to review

Start with map_transformer.py: it is the shared map-task change, and it needs the most care. The MCAP part only reads past a chunk that fails.

File What changes Lines
map_transformer.py, map_operator.py, task_pool_map_operator.py, actor_pool_map_operator.py, task_context.py, output_buffer.py, util.py Emit the rows a task buffered before an error the job skips. 278
context.py, execution-configurations.md The max_errored_blocks docs. 11
mcap_reader.py, mcap_chunks.py, mcap_io.py Skip a corrupt chunk and keep reading. 344
test_map_transformer.py, test_actor_pool_map_operator.py, test_streaming_executor_errored_blocks.py The shared map-task behaviour. 394
test_mcap_reader.py, test_mcap_io.py, test_mcap.py Reading past corrupt chunks. 459

Problem.

  • When max_errored_blocks lets a job skip a failed task, the job also loses every row the task had already read but not yet emitted. A map task buffers rows until a block is full, so a task that fails part-way drops all of its buffered rows with no error.
  • In an MCAP read, one corrupt chunk failed its whole read task. The task's other files, and the rest of the damaged file, were lost.

Change.

  • A map task emits its buffered rows before it raises an error that the job will skip. If the error is one the task will retry, it raises without emitting, so the retry rebuilds the whole block and no row is read twice.
 rows:  [r1 r2 r3 r4 | r5 r6 ✗]
         block 1       buffered, then an error
 before: block 1 kept, r5 r6 lost
 after:  block 1 kept, r5 r6 emitted, then the error is skipped
  • A fused downstream batcher flushes its short last batch before a skipped error, so its rows are kept too.
  • When errors can be skipped, the MCAP reader skips a corrupt chunk and reads the rest of its file. Each chunk is compressed on its own, so one bad chunk costs only its own rows. The task then raises one error naming every chunk it skipped, so it counts once against max_errored_blocks. The reader keeps one error for all of a file's corrupt chunks, so skipped chunks add no memory.
  • When errors can be skipped, a message whose payload is not valid JSON costs only its own row. The file's one error counts such messages and names the first by row id. A wrong message encoding still fails its file.
  • A skipped message does not count against a pushed-down row limit, so limit(n) still returns n rows when the file has them.
  • A file that fails part-way keeps the rows built before the failure. Before, how many it kept depended on where a block ended.
  • Only an error that says a chunk does not decompress or parse marks it as corrupt. Any other error, such as an OSError or a bug, raises at once, so the task's retries see it.
  • A frozen RerunPolicy on TaskContext decides whether an error can still be rerun. A task can be rerun by retried_map_errors while _map_task has attempts left, or by Ray on its own retry_exceptions (for an actor task, built from actor_task_retry_on_errors). A task-pool task is also rerun on a RaySystemError. Ray reruns nothing when a task's max_retries or an actor task's max_task_retries is 0. While a rerun can follow, the task raises without a partial block. _map_task updates the policy for each attempt, so its last attempt emits its rows.
  • The policy's default is "may rerun". A TaskContext built without retry settings, as on the shuffle reduce paths, therefore never emits a partial block.
  • A chunk read back for a declaration that does not decode is skipped, so the chunk being read is not blamed for it. If a schema the task needs is declared only in that corrupt chunk, the read fails and names the chunk, rather than returning rows without a schema. A retried file read starts with no corrupt chunks recorded.
  • The max_errored_blocks docstring and the execution-configurations docs page say that rows a task emitted before a skipped error are kept.
  • With max_errored_blocks=0, nothing changes: the first failure fails the task.

Measured on a folder of recordings with two corrupt chunks and max_errored_blocks=1: the read used to return 0 rows, and now returns 59. The legacy reader returns 40.

Decisions to confirm:

  • Emit the partial block only for errors the job skips. Retried errors still rebuild the whole block.

Related issues

Additional information

PYTHONPATH=python/ray/data/tests pytest python/ray/data/tests/datasource/test_mcap.py \
  python/ray/data/tests/unit/datasource_v2/test_mcap_*.py \
  python/ray/data/tests/test_map_transformer.py \
  python/ray/data/tests/test_streaming_executor_errored_blocks.py

283 passed, 6 skipped; the shared map tests (17), errored-blocks tests (10), map operator tests (78), task pool tests (12), operator tests (8), util tests (45) and actor pool tests (60) pass. pre-commit and pyrefly clean.

AI assistance (Claude Code) was used to draft this change; I reviewed every line and ran the tests above locally.

🤖 Generated with Claude Code

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for skipping corrupt chunks in MCAP files when DataContext.max_errored_blocks is enabled, allowing other chunks and files to be read successfully. It also updates _BlockShapingIterator to emit buffered rows before raising an error when a task fails and cannot be rerun. Feedback on these changes highlights three key issues: first, stale corruption records can persist across retries because the corrupt dictionary is not cleared at the start of _read_file; second, a potential TypeError exists in _matches_retried_map_errors if patterns is falsy or None; and third, checking bool(context.actor_task_retry_on_errors) may incorrectly evaluate to True for non-matching errors if it contains a list of specific exceptions, which would prevent buffered rows from being emitted.

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

@marwan116
marwan116 force-pushed the marwan/mcap-dsv2-errored-rows branch from 63b6932 to 94e489a Compare October 19, 2026 18:57
@ray-gardener ray-gardener Bot added the data Ray Data-related issues label Oct 19, 2026

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

@marwan116
marwan116 force-pushed the marwan/mcap-dsv2-errored-rows branch from b5eb580 to 3e1b552 Compare October 29, 2026 23:38
@marwan116
marwan116 force-pushed the marwan/mcap-dsv2-errored-rows branch from 3e1b552 to 150cf06 Compare November 6, 2026 02:36

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

richardliaw pushed a commit that referenced this pull request Nov 6, 2026
…, row_id (#66654)

## Description

Part 1 of 3 of the `read_mcap` move to Datasource V2, based on master.
#67166 is stacked on it, and #67167 on #67166. The rest of the
`read_mcap` stack (#67189 onward) sits on top.

**Problem.** One task reads one whole file:

- Parallelism is capped by the file count: ten 30 GB recordings are ten
tasks.
- Every file is opened, even when its index shows it holds nothing
selected.
- Rows have no stable id, so a checkpointed pipeline cannot resume.

**Change.** `read_mcap` moves to Datasource V2, on by default as for
`read_parquet` (`DataContext.use_datasource_v2`). The legacy datasource
stays as the fallback.

```
files
  │  listing: read each file's index (a few ranged reads)
  │           one row per chunk, weighted by the memory reading it takes
  │           drop chunks that cannot match topics / time_range
  ▼
chunk rows
  │  packing: about 128 MiB per read task
  │           one big file ──► many tasks;  small files ──► one task
  │           with preserve_order: one task per file, in listing order
  ▼
read tasks: seek to their chunks, merge them by log time
```

- New code lives in `datasource_v2/formats/mcap/`. The V2 interfaces do
not change.
- Columns and types match the legacy path, with two changes.
`schema_data` is dictionary-encoded. A new `channel_metadata` column
holds each channel's metadata map as JSON text, dictionary-encoded, so a
block stores it once per channel.
- `data` holds decoded JSON only when every selected channel is JSON.
Otherwise it holds bytes. Planning types it from the selected messages
of the first chunk that holds one, as a read task types the messages it
holds.
- `row_id` is `<path>#c=<chunk offset>:<index>`, stable however the read
is split. It is on with `include_row_id=True`, or automatically under
`CheckpointConfig(id_column="row_id")`.
- New `log_time_order` argument, default `True`. Each task returns each
of its files in `log_time` order. With `preserve_order`, each file is
read by one task, so it comes out in order from start to end, and files
come out in the order they were listed. #67166 splits a large file there
too, only where its order holds.
- A file whose summary cannot be read gets a read task of its own, so
`max_errored_blocks` can skip it alone.
- The task size is `RAY_DATA_MCAP_BIN_PACKING_BYTES` (128 MiB).

**Changed after the first review round.** Each is its own commit:

- Summaries and chunks are read in a few ranged requests, not a few
bytes per GET.
- An unreadable file gets a read task of its own.
- Tables are built column by column, cut counting every column, and
chunks are weighted by the memory reading them takes.
- The read stops at the row limit.
- A file without a chunk index is sorted by log time.
- `channel_metadata` is JSON text, stored once per block.
- With `preserve_order`, each file is kept in one read task, and files
keep their listing order.
- Planning skips a chunk it cannot decode, and types `data` from the
first chunk's messages.

**Measured** on an 845 MB recording with 1,052 chunks: reading its 125
KB summary took about 20,000 GETs and now takes 3.

**Decisions to confirm:**

- Splitting is on by default. Block boundaries change; rows do not.
- With `preserve_order`, this PR reads each file in one task, as the
legacy reader does. #67166 lets it split a large file where its log-time
order holds.
- `ds.schema()` comes from the first chunk that holds a selected
message. A field that first appears later is missing from it; the
docstring says so.

## Related issues

- Related to #65640 and #65641.
- No open PR moves MCAP to Datasource V2. #66524 (ORC) and #66182
(Iceberg) are other formats moving to V2.
- The generated-ID stack (#65886 to #65888) is orthogonal.
- `log_time_order` comes from my #66653, now closed. My legacy-path PRs
#65912, #65913 and #65916 are closed in favour of this one.

## Additional information

```
PYTHONPATH=python/ray/data/tests pytest python/ray/data/tests/datasource/test_mcap.py \
  python/ray/data/tests/unit/datasource_v2/test_mcap_*.py \
  python/ray/data/tests/unit/datasource_v2/test_online_bin_packer.py
```

178 passed, 3 skipped. pre-commit and pyrefly clean.

AI assistance (Claude Code) was used to draft this change; I reviewed
every line and ran the tests above locally.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Marwan Sarieddine <sarieddine.marwan@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
@marwan116
marwan116 force-pushed the marwan/mcap-dsv2-errored-rows branch from 4a600ab to 76c7607 Compare November 11, 2026 19:43
@richardliaw
richardliaw force-pushed the marwan/mcap-dsv2-errored-rows branch from 76c7607 to f82391b Compare November 11, 2026 22:36

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

@richardliaw
richardliaw force-pushed the marwan/mcap-dsv2-errored-rows branch from f82391b to 5d0a690 Compare November 12, 2026 19:33
Base automatically changed from marwan/mcap-dsv2-io to master November 12, 2026 20:16
marwan116 and others added 11 commits November 12, 2026 12:16
Problem:
- A map task buffers its output until a block reaches its target size.
  When the task raised, its buffered rows were dropped, even when
  max_errored_blocks let the job skip the error. A read task that hit
  one corrupt file lost the rows of the files it had read before it.

Change:
- When the job skips errors (max_errored_blocks is not 0) and the task
  cannot run again, _BlockShapingIterator emits the buffered rows as a
  last block, then raises the error.
- A task that may run again raises without a partial block: on an
  error that retried_map_errors matches while _map_task has attempts
  left, on a RaySystemError, or on an error the task's
  retry_exceptions names (or one it wraps). An actor task's
  retry_exceptions come from actor_task_retry_on_errors. A rerun skips
  as many blocks as were already emitted, so a partial block would
  hide rows of the rerun.
- TaskContext carries the task's Ray retry_exceptions (False when
  max_retries is 0), and _map_task sets each attempt's retries left
  and retried_map_errors patterns on it.
- BlockOutputBuffer.has_buffered_rows says whether rows are waiting.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Marwan Sarieddine <sarieddine.marwan@gmail.com>
Problem:
- The listing cannot isolate a file whose summary reads but whose
  chunk is corrupt. That chunk failed the read task holding it: the
  reader stopped at the bad file, and the small files sharing the task
  lost their rows too.

Change:
- When max_errored_blocks lets errors be skipped, the reader skips a
  corrupt chunk and reads the rest of its file. Each chunk is
  compressed on its own and found through the chunk index, so one bad
  chunk costs only its own rows.
- A chunk read back for a declaration that does not decode is skipped,
  and the look-back goes on to the next earlier chunk, so the chunk
  being read is not blamed for it.
- A retried file read starts with no corrupt chunks recorded.
- A file that fails as a whole does not stop the task's other files.
  A file without a chunk index still fails whole, and an error reading
  the file still raises, so the read retry applies.
- Once the task's files are read, it raises one error naming every
  file and chunk that failed, so the task counts once against
  max_errored_blocks. With max_errored_blocks=0 the first failure
  still fails the task at once.
- With the map task's flush of buffered rows, the job keeps every
  other row, also when a map is fused into the read.
- New tests: a chunk whose payload does not decompress and one whose
  record does not parse, in both read orders, also when fetched in
  one ranged read with its neighbours. End to end, a file with two
  corrupt chunks loses only their rows under max_errored_blocks=1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Marwan Sarieddine <sarieddine.marwan@gmail.com>
Problem:
- An actor task counted as rerun on any error its retry_exceptions
  names, even when its actor's max_task_retries is 0. Ray then does
  not rerun it, so a skipped error dropped the task's buffered rows.

Change:
- An actor task counts as rerun only while max_task_retries allows it.
  The task option overrides the actor's; Ray defaults it to 0, as it
  stays with max_restarts=0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Marwan Sarieddine <sarieddine.marwan@gmail.com>
Problem:
- A map task counted as rerun on every RaySystemError, even with
  max_retries=0 or max_task_retries=0. Ray then does not rerun it, so
  a skipped error dropped the task's buffered rows.

Change:
- RaySystemError counts as rerun only through the task's
  retry_exceptions, which Ray Data always extends with it and which
  are empty when retries are off.
- The task-pool retry check adds RaySystemError as cached_remote_fn
  does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Marwan Sarieddine <sarieddine.marwan@gmail.com>
…error

Problem:
- A fused downstream batcher dropped the rows it held when an upstream
  stage raised an error the job skips. Map then MapBatches(batch_size=4)
  kept rows 0 to 3 of 0 to 6.
- A TaskContext built without retry settings meant "never rerun", so a
  shuffle reduce task emitted partial output it did not opt in to.
- The retry settings were sentinels on TaskContext, and the pattern
  check was a copy of iterate_with_retry's.
- The max_errored_blocks docs still said failed blocks are dropped.

Change:
- A batch transform ends its input at an error the task skips, hands
  on a short last batch, then raises the error.
- A frozen RerunPolicy replaces the TaskContext retry fields. Its
  default reruns on any error; map operators and _map_task fill it in.
- iterate_with_retry and RerunPolicy share is_retryable_error.
- _emit_rows_before_error is renamed _should_emit_buffered_rows.
- The context docstring and the docs page say which rows are kept.
- Dataset-level tests cover a skipped map_batches error and a fused
  batcher.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Marwan Sarieddine <sarieddine.marwan@gmail.com>
…once

Problem:
- A corrupt chunk read back for a declaration was always skipped. When
  it alone declared a schema, rows came out with no schema, even with
  max_errored_blocks=0.
- Any error while a chunk was read, such as the reader's own missing
  channel error, a MemoryError or a bug, was reported as a skipped
  corrupt chunk.
- A task joined every failed file into one ValueError caused by the
  first. A later OSError was hidden, so the task was skipped instead
  of rerun.

Change:
- is_corrupt_data_error says which errors mean the bytes do not decode.
  Only those are skipped, and only around decoding a chunk.
- A chunk read back for a declaration is skipped only when corrupt
  chunks are skipped. A schema still missing after the look-back
  raises, naming the corrupt chunk.
- A file's corrupt data is deferred to the task's one error. Any other
  error raises at once. _read_owned_file holds one file's read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Marwan Sarieddine <sarieddine.marwan@gmail.com>
…uplicates

Problem:
- Ten unit tests each ran one rerun policy and checked whether the
  buffered rows came out first.
- The errored-rows test ran max_errored_blocks 1 and -1, which take
  the same path.
- Two read_mcap tests also ran with a fused map, which the fused
  batcher test already covers.

Change:
- Make the ten tests one test over a small case dataclass of the
  rerun policy and whether the rows come out, with readable ids.
- Let the failing transformer helper raise a given error type, and
  drop the separate system-error transformer.
- Keep only max_errored_blocks=-1, which also catches a check that
  skips only on a positive limit.
- Drop the fused_map parameter from the two read_mcap tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Marwan Sarieddine <sarieddine.marwan@gmail.com>
Problem:
- With max_errored_blocks != 0, one message whose payload is not valid
  JSON dropped the rest of its file, rows already built included.
- The error did not say that the rest of the file was dropped.

Change:
- decode_payload raises InvalidJsonPayload, a ValueError, for a payload
  that is not valid JSON. A wrong message encoding still raises a
  ValueError that fails the file.
- When errors can be skipped, the reader skips that one message and
  keeps every other row of the file, as it does for a corrupt chunk.
- One error per file names the skipped chunks by byte offset, counts
  the skipped messages and names the first by row id, so the task
  counts once. The record keeps only the first message.
- The table builder decodes a payload before it adds anything, so a
  skipped message leaves it consistent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Marwan Sarieddine <sarieddine.marwan@gmail.com>
Problem:
- With max_errored_blocks set, a message that is not valid JSON used
  up a place in the row limit, so a limited read could return fewer
  rows than asked although later messages were valid.
- A file that failed mid-way, for example on a topic of another
  encoding, dropped the rows already built into its current table.
  What the file kept depended on the block size.

Change:
- The limit counts rows built. A skipped message frees its place, and
  no message is pulled once the limit is met.
- An ordered scan of a file without a chunk index holds every message
  when it can skip, so a skipped one cannot shorten the result.
- When errors are skipped, a file error yields the rows built before
  it, then fails the rest of the file as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Marwan Sarieddine <sarieddine.marwan@gmail.com>
Problem:
- A missing schema raised for every message, before the selection
  check ran.
- So a topic the read filters out could fail the rest of its file
  when its schema lived only in a skipped corrupt chunk. The later
  selected rows were lost and a max_errored_blocks slot was spent.

Change:
- Split the channel and schema lookups. A message whose channel the
  listing or the topic filter rejects, or whose log time is out of
  range, is skipped before its schema is looked up.
- A message that may be read still fails on a missing schema and
  names the corrupt chunk read back for it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Marwan Sarieddine <sarieddine.marwan@gmail.com>
Problem:
- A read that skips corrupt chunks keeps each chunk's error until the
  file is done. An error's traceback holds its chunk's bytes, so every
  corrupt chunk adds its size to the read's memory.

Change:
- Keep the byte offsets of a file's corrupt chunks and only the error
  at the lowest offset. The error the read raises is unchanged.
- A test reads a file whose every chunk is corrupt and bounds its peak.
- Cut four reader tests whose cases other tests already cover.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Marwan Sarieddine <sarieddine.marwan@gmail.com>
@richardliaw
richardliaw force-pushed the marwan/mcap-dsv2-errored-rows branch from 5d0a690 to ff354e8 Compare November 12, 2026 20:16
@richardliaw
richardliaw enabled stacked auto-merge (squash) November 12, 2026 20:31
@richardliaw

Copy link
Copy Markdown
Contributor

fix premerge ?

Problem:
- Master's retry wrapper now names the file in a note on the error
  instead of rewriting its message. The test that a task fails at its
  first corrupt chunk looked for the file name in str(error), so it
  fails on master.

Change:
- Look for the file name in the error's __notes__.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Marwan Sarieddine <sarieddine.marwan@gmail.com>
@github-actions
github-actions Bot disabled stacked auto-merge November 13, 2026 17:16

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit f913090. Configure here.

Problem:
- The retry wrapper adds the file name as an exception note, and
  Python 3.10 has no notes, so CI's error carries no file name at all.

Change:
- Check that the task raised the corrupt chunk's ZstdError. The rows
  check already shows the task stopped in b.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Marwan Sarieddine <sarieddine.marwan@gmail.com>
@marwan116

Copy link
Copy Markdown
Contributor Author

fix premerge ?

@richardliaw should now be fixed

@richardliaw
richardliaw merged commit eb534f2 into master Nov 14, 2026
6 checks passed
@richardliaw
richardliaw deleted the marwan/mcap-dsv2-errored-rows branch November 14, 2026 00:35
eugenepaniot added a commit to eugenepaniot/ray that referenced this pull request Nov 19, 2026
…floop-input

* upstream/master:
  [Data][3/n] update progress bar on lineage reconstruction (ray-project#67366)
  [Data] Enable the Top-N rewrite by default and expose it on DataContext (ray-project#68329)
  [Data] Adaptively skip map-side partial aggregation for high-cardinality keys (ray-project#68118)
  [Core] Spawn specific servers without preexec_fn to fix intermittent startup crashes (ray-project#67947)
  [Serve] Stop the __wrapped__ walk at non-callables (ray-project#67117)
  [Data] Deflake test_read_parquet_v2 by cutting redundant Ray Data executions (ray-project#68318)
  [Data] Retry FooterReader actor deaths with Ray Core and yield footers in listing order (ray-project#67988)
  [Data] Fix order-dependent schema unification when a list<null> arm is present (ray-project#68259)
  [Serve] Make @serve.batch work when wrapped in another decorator (ray-project#68177)
  [serve] Deflake test_metrics_3 - record the batch queue length once a batch is taken (ray-project#68302)
  [serve] Drop completed requests from the multiplexed model ID index (ray-project#68085)
  [Data] Fix flaky test_hudi: read each table once instead of sorting in Ray Data, keep size small (ray-project#68162)
  [Data] Fix flaky test_context: start 2 Ray clusters instead of 5 and keep size small (ray-project#68161)
  [Data] Fix flaky test_kafka: use recent timestamps in test_read_kafka_datetime_partial_range (ray-project#68248)
  [Data] Iceberg DSv2 [2/5]: The listing-to-reading manifest for Iceberg (ray-project#67951)
  [Train] Report training errors when the post-mortem debugger raises a BaseException (ray-project#68317)
  [RLlib] Skip the PPOConfig docstring example that crashes the rllib doctest target (ray-project#68306)
  [Train] Support post-mortem debugging for training function errors (ray-project#68296)
  [train][1/n] Give each health probe its own timeout (ray-project#68191)
  [Data] Fix unifying a fixed-shape tensor array with a lower-ndim variable-shaped one (ray-project#67266)
  [Train] Add a guide to debugging failures and hangs (ray-project#68299)
  [Data] Persist BlockMetadata.num_rows and batch ray.get in _calculate_blocks_rows (ray-project#67662)
  [Data] Vectorize LeRobot delta segment preparation and validate index contiguity (ray-project#67845)
  [Data] Avoid full-DataFrame apply in _AbstractKBinsDiscretizer._transform_pandas (ray-project#67862)
  [doc] Refresh intersphinx inventory snapshots (ray-project#68303)
  [Data] Fix SimpleImputer on categorical columns in append mode and with existing category values (ray-project#67467)
  [Data] Replace O(N) list.pop(0) with deque.popleft() in PushBasedShuffleTaskScheduler (ray-project#67243)
  [Train] Reject an elastic ScalingConfig with max_workers=0 (ray-project#68218)
  [core][move semantics][3/n]Populate task args with is_move for movable refs (ray-project#66803)
  [core][sandbox] Remove the readonly option (ray-project#68088)
  [Data](1/n) Implement the line-chunk reader for CSV and JSONL on DataSource V2 (ray-project#67907)
  [Data] Set MIMALLOC_PURGE_DELAY=0 on random_shuffle to fix the pyarrow 25 OOM regression (ray-project#68151)
  [core] Quote each `uv run` argument in the uv runtime env hook (ray-project#68156)
  [core] Don't let gRPC cancel calls that wait over 30s in the server queue (ray-project#68210)
  [Data] Fix shuffle and join on native tensor columns with pyarrow 25+ (ray-project#68072)
  [serve][llm] Remove decode load progress (ray-project#68258)
  [ci] Bump sglang to 0.5.23 in release tests (ray-project#68250)
  [Core] Build the test auth token file locally so Windows tests can read it (ray-project#68262)
  [image] Bump EFA installer to 1.51.0 in CPU base-extra images (ray-project#67730)
  [serve] Observe the result of rank reassignment reconfigures (ray-project#67923)
  [Serve][Docs] Explain imperative vs declarative deploys and in-place updates from Python (ray-project#68243)
  [Data] (Fault tolerance) Ray Data lineage reconstruction docs (ray-project#68061)
  [RLlib] Add a multi-node footsies APPO memory release test (ray-project#67595)
  [Train] Remove local mode (ray-project#68239)
  [Data] Fix filtered Lance row-count metadata (ray-project#67228)
  [Serve][2/2] Propagate TracingConfig to proxies at startup (ray-project#65437)
  [Data] Fix synthesized path filtering in DataSourceV2 (ray-project#67079)
  [RLlib] Run the BCConfig Tune example one trial at a time to unblock the rllib doctest target (ray-project#68242)
  [Data][Delta DSv2][2/7] List Delta Lake table files from the log (ray-project#67888)
  [Data] Parallelize Parquet and ORC full-schema reads (ray-project#67542)
  [Data] Random Shuffle on Shuffle V2 Map/Reduce Operators (ray-project#68004)
  [Data] Propagate metadata prefetch thread failures to executor (ray-project#68171)
  [core] Send generator consumed-progress updates from io_service_ (ray-project#67160)
  [Data] Add @ray-data-connectors as codeowner for datasource/sinks files (ray-project#68154)
  [data][llm] Keep the prompt in vLLM error rows (ray-project#67665)
  [data][actor-only][3/n] Actor-only backend: operator fusion (ray-project#67699)
  [Data] Read text files on DataSource v2 (ray-project#68063)
  [Autoscaler][KubeRay] Never patch worker group replicas below minReplicas (ray-project#67332)
  [Serve] Show progress of updating (ray-project#67107)
  [core][test] Trigger usage-stats OOM test without exhausting host memory (ray-project#68009)
  [Serve] Remove unused RAY_SERVE_HAPROXY_INGRESS_RETRY_ON and RAY_SERV… (ray-project#68175)
  [Data] Implement lineage tracker garbage collection upon marking operator durable (ray-project#67788)
  [Core] Fix recovery suppression race in ObjectRecoveryManager (ray-project#64974)
  [serve] Pass grpc.so_reuseport to gRPC as an integer (ray-project#66816)
  [Autoscaler][v2] Preserve retry accounting when trimming status history (ray-project#67360)
  [doc] Refresh intersphinx inventory snapshots (ray-project#68069)
  [Doc] Add item to AGENTS.md to update documentation when modifying API-facing code. (ray-project#68204)
  [Train] Preserve `num_boost_round=0` and avoid mutating `callbacks` in legacy LightGBM/XGBoost trainers (ray-project#68145)
  [Train] Raise in prepare_data_loader instead of dropping a custom batch_sampler (ray-project#68201)
  [air] Return None in skip_exceptions when exc or StartTraceback cause is None (ray-project#68172)
  [Train] Fix `checkpoint_manager` to support metrics only reported in a validation function (ray-project#67909)
  [Train] Count elastic workers in resource units for fractional resources (ray-project#68139)
  [Data] Refactor Disk shuffle map to use map Operator (ray-project#67130)
  [core][move semantics][2/n] Change state to moved on consumption and add enforcements (ray-project#66767)
  [Data] Propagate labels to object-store SHUFFLE_V2 map/reduce tasks (ray-project#67135)
  feat(core): add Cambricon MLU resource support (ray-project#66368)
  [Data] Make `zero_copy_batch=False` copy default batches (ray-project#67154)
  [data] Fix Parquet filename suffix detection (ray-project#67156)
  [data] Fix docstring/behavior mismatches in read_json and Dataset.sort (ray-project#66927)
  [Data] Run video_object_detection.ray on T4s with more memory (ray-project#68149)
  [core][sandbox] Let a node override network="public" sandboxes' default resolvers (ray-project#67691)
  [Data] read_mcap: resolve each sample's steps and lay out its row (ray-project#67925)
  [Data] read_mcap: sample read inputs: TopicSpec, steps, match and samples checks (ray-project#67924)
  [Data] Keep a map task's buffered rows when max_errored_blocks skips its error; read past corrupt MCAP chunks (ray-project#67167)
  [core][move semantics][1/n] Adding consume_once flag in actor methods (ray-project#66620)
  [Data] Size the schema pin and the columns Ray adds in the Parquet decoded-size estimate [3/6] (ray-project#67838)
  [Serve][LLM][1/3] Token based autoscaling: track autoscaling workload in KV router (ray-project#67937)
  [Data] Add native direct reads for nonpartitioned Hive tables (ray-project#67114)
  [Train] Count elastic workers in resource units for fractional resources (ray-project#68139)
  [Data] Add `condition` to `Dataset.join` and extract equi-join keys from it (ray-project#67590)
  [data] Add Top-N execution (ray-project#66931)
  [build] Bazel 8.8.1: bump protobuf to v29.6, rules_java to 8.12.0 and gRPC to 1.69.0 (ray-project#66483)
  [build] Use Arena::Create and pin Java 8 bytecode ahead of protobuf 29 / rules_java 8 (ray-project#67720)
  [core][sandbox] Don't join a sandbox's cgroup on every exec (ray-project#67690)
  [Data] Run `training_ingest_regression_test` on Instances that can be Provisioned (ray-project#68110)
  [Data] Run Sort on Disk-based Shuffle under SHUFFLE_V2 (ray-project#67850)
  [core] Deflake tests that race on timing (ray-project#67717)
  [ci] Bump pypa manylinux2014 base image to 2026.10.22-0 for CPython 3.15 (ray-project#67285)
  [Data] Make Mcap Data Column Test Insensitive to Struct Field Order (ray-project#68109)
  [Data] read_mcap V2: cut large files where their log-time order holds (ray-project#67166)
  [core][sandbox] Commit sandbox as new registry image (ray-project#67972)
  [Data] read_mcap V2: read neighbouring chunks in one ranged request (ray-project#67930)
  [train][4/n] Wire WorkerProbes through the HealthManager (ray-project#67915)
  [train][3/n] Add WorkerProbe results and health reports to the HealthState (ray-project#67879)
  [Data] (Data fault tolerance ray-project#6/6) Add a manual release test for lineage reconstruction (ray-project#67519)
  [train][6/n] Act on health decisions in the Ray Train controller (ray-project#67856)
  [Train] Do not start an elastic worker group with zero workers (ray-project#68101)
  [Train] Keep the last checkpoint after a metrics-only report in local mode (ray-project#68070)
  [Data][2/n] Migrate GPU Shuffle to Unified Phase Metrics (ray-project#67359)
  [Data][Chkpt] Fix `ArrowInvalid` when loading large generated-ID checkpoints (ray-project#68065)
  [core][test] Create auth token in test_redis_wrong_password (ray-project#68014)
  [data][actor-only][2/n] Actor-only backend: promote task pools to actor pools (ray-project#67698)
  [Data] Fix typing mismatches in Dataset public API (ray-project#62077)
  [Data] read_mcap V2: carry read tasks' summary records in the manifest (ray-project#67929)
  [Data] Distinguish generated ID collisions with synthesized columns (ray-project#67164)
  [Data] Fix incorrect scale-up in rate-based autoscaler for custom resources (ray-project#66647)
  [Data][Delta DSv2][1/7] Shared Parquet and partitioning fixes for the Delta reader (ray-project#67887)
  [Data] Eliminate redundant aggregation shuffle (ray-project#67198)
  [Data] Raise minimum supported pyarrow to 23.0.0 and test up to 26.0.1 (ray-project#68005)
  [Train] Use the default validation timeout when a report's ValidationTaskConfig has none (ray-project#68045)
  [doc][train] Use the global batch size in the TensorFlow Train examples (ray-project#68006)
  [TPU] Default multi-slice MASTER_ADDR to slice 0 host 0 in SlicePlacementGroup (ray-project#67878)
  [Data] Estimate a Parquet row group's decoded Arrow bytes from its footer [2/6] (ray-project#67837)
  [Data] Predicate pushdown through joins: split conjuncts, derive one-sided OR weakenings (ray-project#67122)
  [Data] Add partition_filters to read_delta and take file sizes from the log (ray-project#67900)
  [Data] Stop writing Unity Catalog vended credentials to the environment (ray-project#67899)
  [Data] Raise an error for Delta tables with column mapping in write_delta (ray-project#67898)
  [Core] Add initial foundational configurations and flags for fork server (ray-project#67356)
  [core][sandbox] Image layers caching (ray-project#67874)
  [Doc][Train] Add device stream batch prefetching section to accelerator guide (ray-project#67779)
  [Train] Use user-defined train function name in training thread name and error stack traces (ray-project#64292)
  [serve][docs] Remove beta label from custom autoscaling policies (ray-project#68046)
  [core] Don't treat a gRPC client with calls in flight as idle (ray-project#67718)
  [train] Fix checkpoint sorting to support nested metrics (ray-project#59130)
  [train][5/n] Run ControllerProbes on the controller's node monitor (ray-project#67671)
  [Train] Rank a checkpoint without the score attribute as worst in min order (ray-project#67901)
  [Core][RuntimeEnv] Stream runtime environment package downloads (ray-project#67184)
  [core] Deflake `_run_hanging_command` tests (ray-project#67461)
  [Data] Add Gravitino Iceberg REST catalog provider (ray-project#67007)
  [Data] Fix mixed-root partition filtering in Parquet and ORC V2 (ray-project#67067)
  [core] Keep absl logging at ERROR unless GRPC_VERBOSITY is set (ray-project#67719)
  [Data][1/n]Add Shuffle Metrics Unification Instrumentation (ray-project#66968)
  [ci][release] Stop publishing Python 3.10 wheels and images (ray-project#67722)
  [serve] Copy the ASGI lifespan state into each request scope (ray-project#67844)
  [Serve][LLM][4/5] Support scale-to-zero application routing (ray-project#67087)
  [core][sandbox] Let GPU sandboxes request NVIDIA profiling capabilities (ray-project#67974)
  [Serve][LLM] Exclude LLMRouter from dashboard QPS (ray-project#67873)
  [Data] Read the Parquet SizeStatistics PyArrow doesn't expose [1/6] (ray-project#67836)
  Add perf metrics for 2.60.0 (ray-project#67727)
  [Observability / Train] Add Grafana Annotation support with Ray Train usage (ray-project#66958)

Signed-off-by: Eugene Paniot <e.paniot@gmail.com>

# Conflicts:
#	python/ray/data/_internal/planner/plan_udf_map_op.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

data Ray Data-related issues go add ONLY when ready to merge, run all tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants