Skip to content

Compile nested dataclass schemas along with the outer one - #482

Open
farmio wants to merge 2 commits into
frenck:mainfrom
farmio:compile-nested-dataclasses
Open

farmio wants to merge 2 commits into
frenck:mainfrom
farmio:compile-nested-dataclasses

Conversation

@farmio

@farmio farmio commented Oct 19, 2026

Copy link
Copy Markdown
Collaborator

Breaking change

Proposed change

A nested dataclass field was built as a plain Schema around All(inner, _Constructor). A plain Schema sees only the All, so it never armed under any policy, and DataclassSchema.compile() generated the outer mapping alone: every level below it stayed interpreted, which is why compiling a dataclass tree barely moved its numbers.

The builder now makes a fused dataclass Schema subclass for every level, carrying the compile logic DataclassSchema had. It arms and generates like any schema, and compiling it compiles the nested dataclass schemas under its fields first (through a list, a dict, Maybe, All, or a union), so a tree generates all the way down. The outer compile flag is threaded into every nested schema, so compile=False keeps the whole tree interpreted. A Schema the user placed in an Annotated keeps its own flag and is not entered. A second compile() is a no-op instead of stacking validators.

A schema on a recursion cycle must not compile: a generated validator bails to its interpreted twin on a failure, and the twin re-enters the generated code below it, so every compiled level on a cycle doubles a failing validation once per turn, exponential in the data's depth. Taking a recursive reference now marks every schema from its target down to the field taking it, and a marked schema never arms or generates; the schema above the cycle still does. A regression test pins the deep failure to linear time.

On a three-level API response the compiled path drops from about 22µs to about 10µs per document. A nested-dataclass CodSpeed pair tracks it.

Type of change

  • Dependency or tooling upgrade
  • Bugfix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Deprecation (replaces or removes a feature, with a migration path)
  • Breaking change (a fix or feature that changes existing behavior)
  • Code quality, refactor, or test-only change
  • Documentation only

Additional information

  • This PR fixes or closes issue: fixes #
  • This PR is related to:
  • Link to a separate documentation pull request:

Checklist

  • I fully understand the code in this pull request and can explain every line, including any AI-assisted changes.
  • The change is covered by tests, and uv run --no-sync just test passes locally. A pull request cannot be merged unless CI is green.
  • uv run --no-sync just lint passes.
  • uv run --no-sync just typecheck passes.
  • No commented-out or dead code is left in the pull request.

A nested dataclass field was built as a plain Schema around All(inner,
_Constructor). A plain Schema sees only the All, so it never armed under
any policy, and DataclassSchema.compile() generated the outer mapping
alone: every level below it stayed interpreted, which is why compiling a
dataclass tree barely moved its numbers.

The builder now makes a fused dataclass Schema subclass for every level,
carrying the compile logic DataclassSchema had. It arms and generates like
any schema, and compiling it compiles the nested dataclass schemas under
its fields first (through a list, a dict, Maybe, All, or a union), so a
tree generates all the way down. The outer compile flag is threaded into
every nested schema, so compile=False keeps the whole tree interpreted. A
Schema the user placed in an Annotated keeps its own flag and is not
entered. A second compile() is a no-op instead of stacking validators.

A schema on a recursion cycle must not compile: a generated validator
bails to its interpreted twin on a failure, and the twin re-enters the
generated code below it, so every compiled level on a cycle doubles a
failing validation once per turn, exponential in the data's depth. Taking
a recursive reference now marks every schema from its target down to the
field taking it, and a marked schema never arms or generates; the schema
above the cycle still does. A regression test pins the deep failure to
linear time.

On a three-level API response the compiled path drops from about 22µs to
about 10µs per document. A nested-dataclass CodSpeed pair tracks it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ANcsDvtNPxCRTk33uagPak
Copilot AI balanced review requested due to automatic review settings October 19, 2026 18:42

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@coderabbitai

coderabbitai Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 9 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 4e502c4a-8dc0-4531-a34f-989fd1868026
📥 Commits

Reviewing files that changed from the base of the PR and between 81bde4d and 8c09928.

📒 Files selected for processing: 3
  • docs/src/content/docs/guides/compiled-schemas.md
  • src/probatio/dataclass_schema.py
  • tests/test_codegen.py
📝 Walkthrough

Nested schema construction now propagates compile settings through dataclass and TypedDict annotations. Fused dataclass schemas compile nested builder-created schemas while keeping recursion-cycle members interpreted. Tests, benchmarks, and documentation cover nested compilation behavior.

Changes: Nested schema compilation

Layer / File(s) Summary
Compile policy in schema construction
src/probatio/dataclass_schema.py, src/probatio/decorator.py, tests/__snapshots__/test_public_surface.ambr
Annotation conversion passes compile settings into nested dataclass and TypedDict schemas. Recursive references are marked, and schema-builder signatures and snapshots include the private _compile parameter.
Fused dataclass compilation
src/probatio/dataclass_schema.py
DataclassSchema uses a fused validator and constructor schema. Compilation handles nested builder-created schemas, preserves existing generated validators, and uses interpreted validators for cycle members and unsupported cases.
Validation and usage coverage
tests/test_codegen.py, bench/test_benchmarks.py, docs/src/content/docs/guides/compiled-schemas.md
Tests cover nested compilation policies, repeated compilation, recursion, and validation outcomes. Benchmarks compare interpreted and compiled validation; the guide describes nested compilation and recursion-cycle limitations.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~45 minutes

Merge Risk: 🟡 Moderate · up to 81bde

Deep invalid recursive TypedDict data may take excessive time to reject. Keep cycle members interpreted before merging.

Pre-merge checks: Passed 5
Check name Status Explanation
Title check Passed The title clearly and concisely describes the main change: compiling nested dataclass schemas with the outer schema.
Description check Passed The description directly explains nested dataclass compilation, compile policy propagation, recursive-cycle handling, tests, and performance impact. It is directly related to the changeset.
Docstring Coverage Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 38 functions across 4 files. (2 skipped: 2…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 38 functions across 4 files. (2 skipped: 2 unsupported.)

✨ Finishing Touches: 💡 1
🛠️ Fix failing CI checks: 💡
  • Commit to this branch
  • Create a new PR
✨ Simplify code
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codspeed

codspeed Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 23 untouched benchmarks
🆕 2 new benchmarks

Performance Changes

Benchmark BASE HEAD Efficiency
🆕 test_validate_dataclass_tree N/A 163.4 µs N/A
🆕 test_validate_dataclass_tree_compiled N/A 109.8 µs N/A

Comparing farmio:compile-nested-dataclasses (8c09928) with main (596d7a6)

Open in CodSpeed

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration
  • Configuration used: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 302dbc38-a26c-4815-964f-8ff1f8fd8d0e
📥 Commits

Reviewing files that changed from the base of the PR and between 596d7a6 and 81bde4d.

📒 Files selected for processing: 6
  • bench/test_benchmarks.py
  • docs/src/content/docs/guides/compiled-schemas.md
  • src/probatio/dataclass_schema.py
  • src/probatio/decorator.py
  • tests/__snapshots__/test_public_surface.ambr
  • tests/test_codegen.py

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

📜 Review details
⏰ Context from checks skipped due to timeout: 1
  • GitHub Check: Python 3.12
⚠️ CI failures not shown inline: 2

GitHub Actions: PR Labels / 0_Verify.txt: Compile nested dataclass schemas along with the outer one

Conclusion: failure

View job details

##[group]Run ludeeus/action-require-labels@be19cd7f04c6fad46a85336f9e9cebdf961807bb
 with:
   labels: breaking-change, bugfix, documentation, enhancement, refactor, performance, new-feature, maintenance, ci, dependencies
 ##[endgroup]
 ##[error] No labels defined on the pull request.

GitHub Actions: PR Labels / Verify: Compile nested dataclass schemas along with the outer one

Conclusion: failure

View job details

##[group]Run ludeeus/action-require-labels@be19cd7f04c6fad46a85336f9e9cebdf961807bb
 with:
   labels: breaking-change, bugfix, documentation, enhancement, refactor, performance, new-feature, maintenance, ci, dependencies
 ##[endgroup]
 ##[error] No labels defined on the pull request.
🧰 Additional context used
📚 Code guidelines: 1
AGENTS.md — auto-discovered
📓 Path-based instructions: 1
Source excerpt: This project is public and read closely.

📄 CodeRabbit inference engine (AGENTS.md)

Files:

  • docs/src/content/docs/guides/compiled-schemas.md
🔇 Additional comments: 6
src/probatio/dataclass_schema.py: 1

117-125: LGTM!

Also applies to: 444-463, 872-878, 966-1019, 1194-1273

src/probatio/decorator.py: 1

151-153: LGTM!

Also applies to: 187-189

tests/__snapshots__/test_public_surface.ambr: 1

1837-1837: LGTM!

Also applies to: 1847-1847

tests/test_codegen.py: 1

752-946: LGTM!

bench/test_benchmarks.py: 1

370-413: LGTM!

Also applies to: 421-421

docs/src/content/docs/guides/compiled-schemas.md: 1

66-69: LGTM!

Also applies to: 129-131

Comment thread src/probatio/dataclass_schema.py Outdated
The nested schema for a TypedDict on a recursion cycle could enter the
generated path: under the ON policy already, and now also through the
compile=True flag the outer schema threads down. That is the same cascade
the dataclass rule guards against, a failing validation exponential in the
data's depth (about 13 seconds at 20 levels). The inner schema is now built
with compile=False when its reference closes a cycle, so it never arms
under any policy; the top-level TypedDictSchema above the cycle still
compiles. A regression test pins a failure 24 levels deep to linear time.

Raised by the CodeRabbit review on the pull request.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ANcsDvtNPxCRTk33uagPak
Copilot AI balanced review requested due to automatic review settings October 19, 2026 19:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants