Skip to content

test(operators): several columns, names, mappings and inputs, in every order, role and arrangement (ITL-1236, part 1) - #442

Merged
eywalker merged 7 commits into
mainfrom
eywalker/itl-1236-tests-audit-every-argument-that-takes
Nov 18, 2026
Merged

eywalker merged 7 commits into
mainfrom
eywalker/itl-1236-tests-audit-every-argument-that-takes

Conversation

@eywalker

@eywalker eywalker commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Merge order: #442, then #443, then #444. All three target main; each contains the previous one's commits. After one is merged, main is merged into the next (never a rebase), so its diff shrinks to its own files.

Part 1 of ITL-1236 (operators). Related: ITL-1241, ITL-1242, ITL-1243, ITL-1247, ITL-1249 (defects and a question found here, tested as xfail(strict=True)), ITL-1226 (PR #434, GroupBy, not duplicated here).

Why

GroupBy was tested with one tag and one data column, which hid a result that depended on the order of the input's tag columns (ITL-1226). This PR audits every argument that takes several items and closes the gaps for the operators.

What

  • Inventory, docs/dev/specs/itl-1236-several-item-test-audit.md: one row per argument or input that takes several items (operators, sources, function pods, binding, ColumnConfig), with the existing test that covers each of (a) several items, (b) several orders, (c) each role, (d) 2+ tags and 2+ data in several arrangements, (e) sync and async, and the ranked gaps.
  • Shared inputs, tests/test_core/operators/several_items.py: MAIN (tags t1, t2, t3; data d1, d2, d3) and join partners P, Q, R. Each can be built in any tag order, in three table layouts, with plain tags, data-tags or a mix.
  • Tests, tests/test_core/operators/test_several_items.py, public interface only (op(stream), keys, iter_data, hashes), every case on the sync and the async path:
    • Every operator gives the same rows, row hashes, system tags, source info, content and pipeline hashes for every arrangement of its input (27 arrangements).
    • The order of a set argument changes nothing, not even the column order or the operator's identity: select and drop names (tag, data, both), DropNulls names, MapTags / MapData / MapColumns mappings (swaps included), PolarsFilter constraints. PolarsFilter predicates in every order keep the same rows.
    • Join over 2, 3 and 4 inputs: every permutation gives the same rows, hashes, system-tag names and column order. MergeJoin both ways with three colliding columns. SemiJoin matches on every shared tag.
    • Names of several roles in one call (plain tag, data-tag, data) follow the documented role rules.
    • AddTags: the order of its names is the order of the new tags and part of its identity (as documented).
  • Rule: CLAUDE.md and .zed/rules, "Testing conventions".

Defects found (not fixed here)

Test (xfail(strict=True)) Issue
test_merge_join_of_tied_values_does_not_depend_on_the_order_of_its_inputs: tied values keep their source info in argument order ITL-1241
test_map_tags_swaps_data_tags_as_it_swaps_plain_tags: a swap of data-tags is refused ITL-1242
test_merge_join_of_a_nullable_column_holding_nulls: TypeError on a null in a merged column ITL-1243
test_unary_result_keys_equal_its_row_keys_in_every_arrangement, test_*_async_column_order_equals_sync*: on an input whose table layout differs from its tag order, a result's keys disagree with its rows, and async rows' column order with sync rows' (no hash, no stored table: an operator node stores one order on both orchestrators, tested) ITL-1249
test_polars_filter_predicate_order_is_not_identity: the order of the predicates is in the identity (rule undecided; the test states the proposal) ITL-1247

No library code changes; no golden file changes; no changelog fragment (tests and developer docs only, per changelog.d/README.md).

Follow-up PRs on ITL-1236: sources, function pods and binding.

Posted by a Claude Code agent on behalf of @eywalker.

🤖 Generated with Claude Code

…y order, role and arrangement (ITL-1236)

A shared family of inputs with three tags and three data columns, in several tag
orders, table layouts and role settings (plain, data-tags, mixed), and tests of every
operator over it, sync and async: the result does not depend on the arrangement of the
input; the order of a set argument (names, rename mappings, constraints) changes
nothing; Join over 2, 3 and 4 inputs and MergeJoin are commutative; SemiJoin matches
on every shared tag. Defects found are xfail(strict=True): ITL-1241 (MergeJoin ties),
ITL-1242 (MapTags swap of data-tags), MergeJoin null crash (issue pending).

The inventory of every several-item argument, with the evidence and the ranked gaps,
is docs/dev/specs/itl-1236-several-item-test-audit.md. CLAUDE.md and .zed/rules state
the rule in "Testing conventions".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@eywalker
eywalker requested a balanced review from Copilot November 18, 2026 05:12
@coderabbitai

coderabbitai Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

  • Run on-demand review

This review includes 5 billable files and costs up to $1.25.

  • Ask an admin to make reviews automatic

Open in CodeRabbit

Reviews can continue after your included limit without a manual trigger. An admin must approve usage-based billing.

Or wait 11 minutes for your next included review.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available. Your 115 included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

Learn how review limits work.

Review configuration:

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 4da4268 and e1800eb.

Files selected for processing: 5

  • .zed/rules
  • CLAUDE.md
  • docs/dev/specs/itl-1236-several-item-test-audit.md
  • tests/test_core/operators/several_items.py
  • tests/test_core/operators/test_several_items.py
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The join tests leave coverage gaps: async results lack sync-equivalence assertions, restricted-input behavior is tested only synchronously, and MergeJoin omits reordered and interleaved column layouts.

3 open issues
What changed in this PR

This first part of ITL-1236 expands operator regression coverage for multiple columns, argument orders, column roles and input layouts, complementing the GroupBy work in PR #434. It changes tests and developer guidance only, not library code or golden hashes.

Changes:

  • Adds reusable multi-column inputs and sync/async operator test matrices.
  • Records coverage gaps and known defects, with strict expected-failure tests.
  • Adds matching several-item testing guidance to both agent instruction files.
File Description
tests/​test_core/​operators/​test_several_items.py Adds operator arrangement, argument-order, role and multi-input tests.
tests/​test_core/​operators/​several_items.py Provides shared sources, layouts, fingerprints and async execution helpers.
docs/​dev/​specs/​itl-1236-several-item-test-audit.md Inventories coverage, known defects and remaining gaps.
CLAUDE.md Adds several-item testing requirements.
.zed/​rules Mirrors the testing requirements for Zed.

Review effort: Balanced

…se; MergeJoin and the restricted SemiJoin in every layout (ITL-1236)

Addresses the Copilot review of #442.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…; predicates in every order and role, sync and async (ITL-1236)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The new tests can miss incorrect predicate-combination semantics and sync/async column-order mismatches because they lack expected-row and order-sensitive assertions.

1 open issue
1 previously missed issue

In code that hasn't changed since last review

Medium severity Assert expected rows for combined predicate semantics

tests/​test_core/​operators/​test_several_items.py:281

This only checks agreement between permutations, so an implementation that combines multiple predicates with OR instead of AND still passes: all six permutations would return every MAIN row, whereas the intended conjunction keeps only the row with d1 == 50. The other new predicate cases likewise compare arrangements or sync/async results without an expected-row assertion. Please assert the expected surviving row here so the several-predicate coverage also detects incorrect combination semantics.

3 resolved since last review

Review effort: Balanced

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…arrangement; defect 7 as xfail (ITL-1236)

Addresses the second Copilot review of #442: sync and async rows are compared with
their column order, system-tag order and Arrow schemas within each arrangement, and
a result's keys with its rows' keys. Both fail on inputs whose table layout differs
from the tag order (reported, issue pending); those combinations are
xfail(strict=True), the others are asserted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…de stores one column order on both orchestrators (ITL-1236)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The tests leave order independence during actual tag selection unverified, while broad xfail sweeps can mask regressions in previously working arrangements.

2 open issues
1 previously missed issue

In code that hasn't changed since last review

Medium severity Exercise partial tag selection across all permutations

tests/​test_core/​operators/​test_several_items.py:295

This case selects all three tags, so every permutation takes the passthrough branch in SelectTagColumns.unary_static_process, and the async path has no tags to drop. The partial selections elsewhere use only one argument order, leaving order independence during actual selection untested. Select two of the three tags here so the permutation matrix exercises selection on both paths and in every role.

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed because mixed-role data-tag handling lacks test coverage across argument orders and input arrangements on both synchronous and asynchronous paths.

2 open issues

Review effort: Balanced

@codecov

codecov Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
see 5 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…ere ITL-1249 reproduces (ITL-1236)

Addresses the third Copilot review of #442.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are reported for the expanded operator regression coverage and developer testing guidance.

2 open issues

Review effort: Balanced

@eywalker
eywalker merged commit d81eec8 into main Nov 18, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants