[Bug]: position delete mask wraps row ordinals past MaxInt64 instead of erroring #30431

Branch Name

main

Commit ID

b24d1b3

Other Environment Information

• Hardware parameters: Apple Silicon (arm64), standalone unit test (no mo-service needed)
• OS type: Darwin 24.5.0 arm64, go1.27.1
• Others: package pkg/sql/colexec/icebergdelete, production caller pkg/sql/colexec/external/iceberg_delete_apply.go:154

Actual Behavior

ApplyState.ApplyPositionMask (pkg/sql/colexec/icebergdelete/delete_apply.go:244) iterates batch rows as rowOrdinal := startRowOrdinal + int64(i) with no overflow check, and the exported helper RowOrdinals (delete_apply.go:281) does the same. With startRowOrdinal = math.MaxInt64-1 and 3 rows:

ordinals=[9223372036854775806 9223372036854775807 -9223372036854775808]
mask over wrapped range returned keep=[false false true] without error

The third row is probed as -9223372036854775808, which can never match the position index (PositionIndex.Add rejects negative ordinals), so the mask silently keeps going instead of reporting corrupt input. On the scan path (applyIcebergDeletes calls this for every batch with IcebergBatchStartRowOrdinal), wrapped ordinals mean position deletes are silently skipped for the affected rows (extra rows returned) rather than failing fast with a clear error. This is the read-path counterpart of the write-path synthesis overflow already reported separately (that one is in pkg/sql/iceberg/dml_matched_batch.go); the root cause here is independent code in the delete-apply package with its own fix location.

Expected Behavior

ApplyPositionMask (and RowOrdinals) must detect that startRowOrdinal + rowCount - 1 overflows int64 and return an error (consistent with the package's fail-fast stance elsewhere, e.g. CheckMemoryLimit, and with the writer-side Pos < 0 rejection in pkg/iceberg/dml/delete_writer.go:328), instead of probing wrapped negative ordinals.

Steps to Reproduce

Run the regression test added in this workspace (file pkg/sql/colexec/icebergdelete/loom_test_position_mask_overflow_test.go):

export CGO_CFLAGS="-I$PWD/cgo -I$PWD/thirdparties/install/include"
export CGO_LDFLAGS="-Wl,-rpath,$PWD/thirdparties/install/lib -Wl,-rpath,$PWD/cgo -L$PWD/thirdparties/install/lib -L$PWD/cgo -lmo -lusearch_c -lm"
go test -tags matrixone_test -short -count=1 ./pkg/sql/colexec/icebergdelete/ -run 'TestLoomPositionMaskRejectsRowOrdinalOverflow' -v

The test asserts RowOrdinals(math.MaxInt64-1, 3) is monotonic/non-negative and that ApplyPositionMask over a range crossing MaxInt64 returns an error. It currently fails on both counts. All 8 pre-existing tests in the package pass.

Additional information

The ordinal chain feeding this function also uses plain addition without validation: parquetRowGroupStartOrdinals (pkg/sql/colexec/external/parquet.go:156, next += rowGroup.NumRows()), batchStartOrdinal := r.h.rowOrdinalBase + r.h.offset (pkg/sql/colexec/external/reader_parquet.go:76). With honest metadata the values stay small, so this only triggers on corrupt/lying row-group counts — hence minor severity — but when it triggers, the failure mode is silent wrong results (skipped deletes) instead of an error. Suggested fix: overflow-check startRowOrdinal + int64(rowCount-1) up front in ApplyPositionMask (and in RowOrdinals).

Note: gh CLI authentication was unavailable in this test environment, so issue deduplication could only be done from local knowledge; no duplicate found locally.

Found by Loom bug hunt (module: custom-e69df17185; tests: unit, white-box, end-to-end)