Skip to content

test(data_tags, streams, sources): several nullable data-tags, list-valued data-tags, stream methods and column groups (ITL-1236, part 3) - #444

Open
eywalker wants to merge 21 commits into
mainfrom
eywalker/itl-1236-part3-methods-data-tags
Open

eywalker wants to merge 21 commits into
mainfrom
eywalker/itl-1236-part3-methods-data-tags

Conversation

@eywalker

@eywalker eywalker commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Merge order: #442, then #443, then #444. All three target main; each contains the previous one's commits. After one is merged, main is merged into the next (never a rebase), so its diff shrinks to its own files.

Part 3 (last) of ITL-1236. Stacked on #443, which is stacked on #442 (merged in, no rebase). Until those merge, review only the files below.

Closes ITL-1236. Related: ITL-1241, ITL-1242, ITL-1243, ITL-1244, ITL-1245, ITL-1246, ITL-1247 (defects and undecided rules the audit found, each with an xfail(strict=True) test in #442 or #443).

What (tests only, public interface; every case as documented, no defect found)

  • tests/test_core/data_tags/test_several_items.py
    • DropNulls naming plain tags, data-tags holding nulls and data, in every order: one row set and schema; each named column non-nullable, a data-tag in both roles; async equals sync.
    • Index(..., out=) over a list-valued data-tag (after Batch): read as a data column, the data-tag kept; async equals sync.
  • tests/test_core/streams/test_several_items_methods.py
    • Every stream method given several items (select_*, drop_*, positional select_columns / drop_columns, map_tags, map_data, drop_nulls, polars_filter with several predicates, several constraints, and constraint_map with keywords, add_tags, tag_by, join, semi_join) equals its operator: rows, hashes and column order, for plain tags, data-tags and a mix.
    • Every combination of ColumnConfig groups gives the all_info columns of those groups in the same order (source, join, batch; every role and layout).
  • tests/test_core/sources/test_several_items.py: a DictSource data_schema in another field order is the same source.
  • The inventory (docs/dev/specs/itl-1236-several-item-test-audit.md) records part 3 and what remains (a MapTags golden entry, out of scope; the defect issues).

No library code changes; no golden changes; no changelog fragment (tests and developer docs only).

Posted by a Claude Code agent on behalf of @eywalker.

🤖 Generated with Claude Code

Edgar Y. Walker and others added 9 commits November 17, 2026 21:12
…y order, role and arrangement (ITL-1236)

A shared family of inputs with three tags and three data columns, in several tag
orders, table layouts and role settings (plain, data-tags, mixed), and tests of every
operator over it, sync and async: the result does not depend on the arrangement of the
input; the order of a set argument (names, rename mappings, constraints) changes
nothing; Join over 2, 3 and 4 inputs and MergeJoin are commutative; SemiJoin matches
on every shared tag. Defects found are xfail(strict=True): ITL-1241 (MergeJoin ties),
ITL-1242 (MapTags swap of data-tags), MergeJoin null crash (issue pending).

The inventory of every several-item argument, with the evidence and the ranked gaps,
is docs/dev/specs/itl-1236-several-item-test-audit.md. CLAUDE.md and .zed/rules state
the rule in "Testing conventions".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… bound sources, in every order and role (ITL-1236)

Sources: tag order and layout enter no hash; tag_only_columns order changes nothing;
record ids follow the key order (content hash moves, pipeline hash and source id do
not); tags_as_data order is the new columns' order only; CachedSource reads back every
role and arrangement. Function pods: every order of three inputs gives the same rows,
hashes and columns, and a job over another order finds every result stored, sync and
async; parameters match by name; a function reads several data-tags in any
arrangement. Bind: two sources in one call, one refused leaves both unbound; a source in
another arrangement binds without moving anything.

Defects found are xfail(strict=True) with the issue pending: the fall-through tags'
order in a function pod's identity (defect 6), swapped output_keys keep the uri
(defect 4), a mixed-role source in another order breaks the bound node's table
(defect 5).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…se; MergeJoin and the restricted SemiJoin in every layout (ITL-1236)

Addresses the Copilot review of #442.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…' into eywalker/itl-1236-part2-sources-pods-bind
…; predicates in every order and role, sync and async (ITL-1236)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…' into eywalker/itl-1236-part2-sources-pods-bind

# Conflicts:
#	docs/dev/specs/itl-1236-several-item-test-audit.md
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…' into eywalker/itl-1236-part2-sources-pods-bind
…alued data-tags, stream methods and column groups (ITL-1236)

DropNulls naming plain tags, data-tags holding nulls and data in every order; Index
with out= over a list-valued data-tag; a DictSource schema in another field order;
every stream method given several items equals its operator (polars_filter keywords
win over constraint_map); every combination of ColumnConfig groups keeps the all_info
column order. Sync and async compared where an async path exists. All as documented.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@eywalker
eywalker requested a balanced review from Copilot November 18, 2026 05:25
@coderabbitai

coderabbitai Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

  • Run on-demand review

This review includes 10 billable files and costs up to $2.50.

  • Ask an admin to make reviews automatic

Open in CodeRabbit

Reviews can continue after your included limit without a manual trigger. An admin must approve usage-based billing.

Or wait 8 minutes for your next included review.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available. Your 115 included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

Learn how review limits work.

Review configuration:

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 4da4268 and 88f810b.

Files selected for processing: 10

  • .zed/rules
  • CLAUDE.md
  • docs/dev/specs/itl-1236-several-item-test-audit.md
  • tests/test_core/data_tags/test_several_items.py
  • tests/test_core/function_pod/test_several_items.py
  • tests/test_core/operators/several_items.py
  • tests/test_core/operators/test_several_items.py
  • tests/test_core/sources/test_several_items.py
  • tests/test_core/streams/test_several_items_methods.py
  • tests/test_pipeline/test_bind_several_items.py
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

…arrangement; defect 7 as xfail (ITL-1236)

Addresses the second Copilot review of #442: sync and async rows are compared with
their column order, system-tag order and Arrow schemas within each arrangement, and
a result's keys with its rows' keys. Both fail on inputs whose table layout differs
from the tag order (reported, issue pending); those combinations are
xfail(strict=True), the others are asserted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The tests leave key behaviors unverified: requested column-group membership, complete schema narrowing, and preservation of the indexed column’s data role and values.

3 open issues
What changed in this PR

Completes the final part of ITL-1236’s several-item test audit, building on stacked PRs #442 and #443. This adds test coverage and developer documentation without changing library code or golden values.

Changes:

  • Tests nullable and list-valued data-tags, stream-method equivalence, and column-group combinations.
  • Adds DictSource schema-field-order coverage.
  • Updates the audit inventory with completed coverage and deferred issues.
File Description
tests/​test_pipeline/​test_bind_several_items.py Stacked coverage for multiple bindings and source arrangements.
tests/​test_core/​streams/​test_several_items_methods.py Tests stream methods and column-group ordering.
tests/​test_core/​sources/​test_several_items.py Covers source ordering, caching, and schema-field order.
tests/​test_core/​operators/​test_several_items.py Stacked operator order, role, and execution-path coverage.
tests/​test_core/​operators/​several_items.py Shared source fixtures and comparison helpers.
tests/​test_core/​function_pod/​test_several_items.py Stacked multi-input, cache-reuse, and data-tag coverage.
tests/​test_core/​data_tags/​test_several_items.py Tests nullable data-tags and indexing after batching.
docs/​dev/​specs/​itl-1236-several-item-test-audit.md Records audit coverage and remaining work.
CLAUDE.md Adds the several-item testing convention.
.zed/​rules Mirrors the testing convention for Zed.

Review effort: Balanced

Edgar Y. Walker and others added 6 commits November 17, 2026 21:31
…' into eywalker/itl-1236-part2-sources-pods-bind

# Conflicts:
#	docs/dev/specs/itl-1236-several-item-test-audit.md
#	tests/test_core/operators/several_items.py
…th bound sources (ITL-1236)

Addresses the Copilot review of #443.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…de stores one column order on both orchestrators (ITL-1236)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…' into eywalker/itl-1236-part2-sources-pods-bind

# Conflicts:
#	docs/dev/specs/itl-1236-several-item-test-audit.md

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Self-referential expected-column assertions and an unchecked second nullable data-tag allow incorrect combined-group handling and incomplete type narrowing to pass the tests.

3 open issues

Review effort: Balanced

@codecov

codecov Bot commented Nov 18, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
see 4 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Edgar Y. Walker and others added 4 commits November 17, 2026 21:50
…ere ITL-1249 reproduces (ITL-1236)

Addresses the third Copilot review of #442.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…' into eywalker/itl-1236-part2-sources-pods-bind
…ements, cache-only readback, bind on both orchestrators (ITL-1236)

Addresses the second Copilot review of #443.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ma; Index keeps the data-tag's lists; column groups checked against their own columns (ITL-1236)

Addresses the Copilot review of #444.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are identified for the several-item test audit.

No open issues 🎉

3 resolved since last review

Review effort: Balanced

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants