Skip to content

fix(function_pod): as_table clears a stale cache before using it (ITL-1045) - #358

Merged
eywalker merged 1 commit into
mainfrom
eywalker/itl-1045-as_table-builds-rows-and-content-hashes-in
Nov 13, 2026
Merged

eywalker merged 1 commit into
mainfrom
eywalker/itl-1045-as_table-builds-rows-and-content-hashes-in

Conversation

@eywalker

Copy link
Copy Markdown
Contributor

Fixes ITL-1045

What

FunctionPodStream.as_table (_CachedFunctionPodStream inherits it) and FunctionNodeBase.as_table used their cached table without a staleness check and built the content hashes in a second iter_data() pass, which is the pass that clears a stale cache. With a non-deterministic function, an upstream clear_cache() between as_table() and as_table(columns={"content_hash": True}) gave the OLD rows with the hashes of the NEW rows. as_table now clears a stale cache first (the same two lines in both), so the table and the hashes come from the same rows and the call returns the current rows. It builds on #355 (staleness is transitive, a pod stream stamps only when complete); nothing here stamps at the start of a read.

Evidence

  • Failing-first tests (tests/test_core/function_pod/test_as_table_stale_between_passes.py, public interface, tags not in order, a call-counter function): pod stream and pod-cached stream failed before the fix (0 of 3 rows carried their own hash), pass after. head() and iter_data() after the same sequence were already correct (tests added, pass).
  • Job node: no public sequence made it wrong. After upstream_node.clear_cache() the job node is stale, but its values are re-read from the store, so the rows are the same and the hashes aligned (checked on the unfixed code). It was latent; the shared base-class fix covers it, and a test pins it.
  • No hash, golden entry, stored table, public API or dependency change; golden tests pass. Targeted tests locally on 3.12 (function_pod, streams, golden hashes, design issues, changelog: 2695 passed); the full suite on 3.11, 3.12 and 3.13 is verified by CI. ruff, format, basedpyright exit 0.

🤖 Generated with Claude Code

…-1045)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@linear-code

linear-code Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
ITL-1045 as_table builds rows and content hashes in two passes: a stale upstream between them gives old rows with new rows' hashes

Overview

Found on November 12, 2026 by the code sweep of ITL-1034 and reproduced by that worker on current main (after the ITL-1029 fix). Not verified by the coordinator. It is the same class of defect as ITL-1029: a column built in one pass attached to a table built in another.

FunctionPodStream.as_table (core/function_pod.py, about lines 1160 to 1194) builds the table in one iter_data pass and the content hashes in a second, with no staleness check between them. If the upstream is reset between a cached as_table() and as_table(content_hash=True), the hash pass clears the cache and recomputes, and the old cached table gets the new rows' hashes.

s1 = stamp.pod(src)        # stamp is not deterministic (a counter, a timestamp, a random draw)
s2 = plus_one.pod(s1)
s2.as_table()              # z = [2, 103, 204]
s1.clear_cache()           # public; s2.is_stale -> True
t = s2.as_table(columns={"content_hash": True})   # z still [2, 103, 204]
# no row's _content_hash is the hash of its own z (0 of 3); a fresh read gives z = [302, 403, 504]

With a deterministic function the recomputed values are the same and the table stays aligned, so it needs an upstream whose values changed. FunctionNodeBase.as_table has the same structure; for a job node the worker found no public way to make the node stale between the two calls, so there it is latent.

Goals & Success Criteria

  • as_table never returns rows from one computation with hashes from another: a stale cache is cleared before the cached table is used, or table and hashes are built from one list of pairs. For FunctionPodStream, _CachedFunctionPodStream and FunctionNodeBase.
  • The same call also returns the current rows, not the stale cached ones, when the stream is stale.
  • A failing-first test through the public interface with a non-deterministic function and clear_cache().
  • No hash, golden entry, stored table or public API changes.

Scope & Boundaries

In scope:

  • The as_table paths named above and their tests.

Out of scope:

  • The row-alignment suite itself (ITL-1034), which adds "upstream changed between reads" as a dimension afterwards.

Dependencies

Review in Linear

@coderabbitai

coderabbitai Bot commented Nov 13, 2026

Copy link
Copy Markdown

Warning

Review limit reached

The included review limit has been reached and this organization has disabled usage-based review continuation. Wait for reviews to reset or ask a billing admin to change After included review limits.

Next included review available in 57 seconds.

Check out review usage here.

View limit details

Limit details: You’ve used all 10 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 54829e93-28d7-4461-b829-c6bef9e43ae2
Commits

Reviewing files that changed from the base of the PR and between 89160c4 and 989359c.

Files selected for processing: 6

  • changelog.d/ITL-1045.md
  • design_issues/ITL-1045-FP1.md
  • src/orcapod/core/function_pod.py
  • src/orcapod/core/nodes/function_node.py
  • superpowers/plans/2026-11-13-itl-1045-as-table-two-passes.md
  • tests/test_core/function_pod/test_as_table_stale_between_passes.py
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the change that clears stale caches before table assembly.

No open issues 🎉

What changed in this PR

Fixes ITL-1045 by clearing stale caches before as_table() uses them, keeping returned rows aligned with their content hashes. This builds on #355’s transitive staleness handling.

Changes:

  • Adds upfront staleness checks to function pod streams and function nodes; cached pod streams inherit the fix.
  • Adds regression coverage for stale reads, content-hash alignment, and related public read methods.
  • Records the fix in the design issue, implementation plan, and changelog without changing hashes, stored layouts, or public APIs.
File Description
tests/​test_core/​function_pod/​test_as_table_stale_between_passes.py Adds regression tests for stale reads and row/hash alignment.
superpowers/​plans/​2026-11-13-itl-1045-as-table-two-passes.md Documents the fix strategy and test scope.
src/​orcapod/​core/​nodes/​function_node.py Clears stale node caches before table assembly.
src/​orcapod/​core/​function_pod.py Clears stale pod-stream caches before table assembly.
design_issues/​ITL-1045-FP1.md Records the defect and its resolution.
changelog.d/​ITL-1045.md Adds the release note for the stale-cache fix.

Review effort: Balanced

@eywalker
eywalker merged commit ad88dc4 into main Nov 13, 2026
22 checks passed
@eywalker
eywalker deleted the eywalker/itl-1045-as_table-builds-rows-and-content-hashes-in branch November 13, 2026 16:32
eywalker pushed a commit that referenced this pull request Nov 13, 2026
…nfiguration (ITL-1034)

Merge main at 5c7a058 (#351 ITL-1050, #354 ITL-1053, #356 ITL-1060, #358
ITL-1045, #360 ITL-1065, #361). With --runxfail only the ITL-1066 cases fail,
so every ITL-1050, ITL-1060, ITL-1065 and ITL-1045 mark is removed.

head(n, cfg) now takes sort_by_tags and all_info: node, operator and stream
heads are checked under every configuration, must be sorted when asked and
must equal as_table(cfg).slice(0, n). Previews before a full read sort the
rows the stream holds; the check allows any order among tied tags.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants