Skip to content

[ty] Separate specialized class objects from runtime generic aliases - #29403

Open
charliermarsh wants to merge 5 commits into
mainfrom
charlie/ty-class-specialization-representation
Open

charliermarsh wants to merge 5 commits into
mainfrom
charlie/ty-class-specialization-representation

Conversation

@charliermarsh

@charliermarsh charliermarsh commented Oct 15, 2026 •

Copy link
Copy Markdown
Member

Summary

This PR adds a Type variant to distinguish runtime generic aliases from class objects that carry specialization information:

from typing import final

@final
class Box[T]: ...

box = Box[int]()
alias = Box[int]  # Type::GenericAlias
cls = type(box)   # Type::SpecializedClass

isinstance(box, cls)    # True at runtime.
isinstance(box, alias)  # Raises TypeError at runtime.

Both alias and cls carry the specialization Box[int] in our type model. But alias is an alias object, while cls is the actual Box class. The int on cls records static information about the instance it came from; it does not mean Python created a separate class for Box[int].

Previously, we represented both values with Type::GenericAlias. We need to distinguish them to reject aliases where Python requires a class, such as the second argument to isinstance. (If we instead represent cls with ClassLiteral, we lose its specialization.)

This PR is entirely behavior-preserving, but paves the way for #29405, which uses it to reject aliases where a class is required.

@charliermarsh charliermarsh added internal An internal refactor or improvement (not for changelogs) ty The ty type checker labels Oct 15, 2026
@astral-sh-bot

astral-sh-bot Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Memory usage report

Memory usage unchanged ✅

@astral-sh-bot

astral-sh-bot Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Typing conformance results

No changes detected ✅

Current numbers
The percentage of diagnostics emitted that were expected errors held steady at 98.24%. The percentage of expected errors that received a diagnostic held steady at 98.24%. The number of fully passing files held steady at 134/146.

@astral-sh-bot

astral-sh-bot Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

ecosystem-analyzer results

No diagnostic changes detected ✅

Full report with detailed diff (timing results)

@charliermarsh charliermarsh changed the title [ty] Name class specialization data independently of runtime aliases [ty] Name specialized class types and their specialization data Oct 15, 2026
@charliermarsh charliermarsh changed the title [ty] Name specialized class types and their specialization data [ty] Separate specialized class objects from runtime generic aliases Oct 15, 2026
@charliermarsh
charliermarsh removed this pull request from stack #29404 October 15, 2026 03:00
@charliermarsh
charliermarsh marked this pull request as ready for review October 15, 2026 10:59
@charliermarsh
charliermarsh requested review from a team as code owners October 15, 2026 10:59
@charliermarsh
charliermarsh requested review from carljm and removed request for a team October 15, 2026 10:59
@charliermarsh
charliermarsh removed this pull request from stack #29407 October 15, 2026 11:21
@charliermarsh
charliermarsh removed this pull request from stack #29418 October 15, 2026 12:52
@charliermarsh
charliermarsh force-pushed the charlie/ty-class-specialization-representation branch from 396841a to d16e3f8 Compare October 15, 2026 12:52
@charliermarsh
charliermarsh changed the base branch from main to charlie/ty-generic-alias-conformance-pin October 15, 2026 12:52
@charliermarsh
charliermarsh force-pushed the charlie/ty-class-specialization-representation branch from d16e3f8 to 97b100b Compare October 15, 2026 12:55
Base automatically changed from charlie/ty-generic-alias-conformance-pin to main October 15, 2026 13:02
@AlexWaygood
AlexWaygood force-pushed the charlie/ty-class-specialization-representation branch from 97b100b to 90f17cd Compare October 15, 2026 13:03
}

match (source, target) {
(Type::GenericAlias(alias), _) => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doing this for redundancy/equivalence relations makes a union of a class object and its runtime alias collapse to whichever one was added first, if they have the same specialization - unions should be independent of order.

This is probably acceptable temporarily until #29405, though. I don't think we really want to land this PR without #29405 regardless.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that's correct. These should land together! I just think it's easier review this way.

Comment thread crates/ty_ide/src/semantic_tokens.rs Resolved
use crate::Db;
use crate::types::{ClassSpecialization, Type};

/// Recover the NumPy scalar class objects that its type definitions describe as generic aliases.

@carljm carljm Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: this is a pretty opaque/compressed description of the situation. I'd expand it slightly, e.g. "Numpy's type stubs pretend that some runtime class objects are generic aliases; identify them and return a class-object type for them instead. TODO We may be able to remove this special-casing once all supported numpy versions have numpy/numpy#33052".

Also we should make sure this special-casing doesn't have any negative effect (I think ideally no effect at all) if we are dealing with a numpy version that has that PR included.

Comment on lines +136 to 138
Type::SpecializedClass(alias) | Type::GenericAlias(alias) => {
builder.into_diagnostic(format_args!(
"`types.GenericAlias` instance `{}` is not a valid class",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the very least, this error message no longer looks correct. And I think the code change might not be correct, either? But it looks like this is fixed later in the stack.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that's right.

if alias.origin(db).is_tuple(db) =>
{
Some(Cow::Owned(TupleSpec::homogeneous(todo_type!(
"*tuple[] annotations"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is right -- I think this should defer to normal class-object iterability?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this is maybe another fix that belongs in the second PR, given the intended split? But (at least as of last night) I don't think it was fixed in the second PR.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, hmm, it's now fixed in #29403 (this PR), so not entirely behavior-preserving.

// It's not valid to use a generic alias as the second argument to `isinstance()` or `issubclass()`,
// e.g. `isinstance(x, list[int])` fails at runtime.
Type::GenericAlias(_) => None,
// We do not currently narrow against specialized class objects.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But we probably could, and treat them as their unspecialized class? Could be worth a TODO at least. (The weird case with the numpy types not being exact could be an issue here, since negative narrowing on e.g. np.int64 can't exclude all signedinteger.)

Comment thread crates/ty_python_semantic/src/types.rs Outdated
Comment on lines +2044 to +2045
/// A class object carrying specialization information. The class may be a subclass of the
/// specialization's origin, so the origin does not necessarily identify the runtime class.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think "may be a subclass" is the semantics we want for this type -- it should be exact. SubclassOf should be used for the inexact case. And I don't think the inexact semantics are currently consistently applied in this stack, either: subtype checking seems to accept subclasses, but disjointness checking treats differing origins as disjoint.

Codex is suggesting to me that the "may be a subclass" here really just comes from the numpy special case, where e.g. np.int64 is a subclass of signedinteger at runtime. I think we should standardize on SpecializedClass being more like ClassLiteral and not admitting subclasses, and find a different way to avoid problems from the numpy case (though without looking into it more I'm not sure what that way should be.)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok so I probably should have taken the time to look into this deeper and recommend a specific direction, because the updated approach of using SubclassOf for the numpy types and then adding a bunch of machinery to make them still usable in type annotations, seems way too invasive to me for this already-kind-of-obsolete compatibility hack.

I think a better direction might be to just let these numpy types continue to be represented as GenericAlias, as the (old) stubs say they are, and instead try to add targeted exceptions to suppress specific false positives that are actually problematic. I think "assignability to type types" might actually be the only place we need that exception?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I reverted this, so the NumPy types use GenericAlias, and we have assignability exceptions in #29405. I was liberal with TODOs there to mark each one...

@charliermarsh
charliermarsh requested a review from carljm October 21, 2026 15:44
@charliermarsh
charliermarsh force-pushed the charlie/ty-class-specialization-representation branch from f206764 to 28aa15c Compare October 21, 2026 15:52
Comment thread crates/ty_python_semantic/src/types.rs Outdated
Comment on lines +2044 to +2045
/// A class object carrying specialization information. The class may be a subclass of the
/// specialization's origin, so the origin does not necessarily identify the runtime class.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok so I probably should have taken the time to look into this deeper and recommend a specific direction, because the updated approach of using SubclassOf for the numpy types and then adding a bunch of machinery to make them still usable in type annotations, seems way too invasive to me for this already-kind-of-obsolete compatibility hack.

I think a better direction might be to just let these numpy types continue to be represented as GenericAlias, as the (old) stubs say they are, and instead try to add targeted exceptions to suppress specific false positives that are actually problematic. I think "assignability to type types" might actually be the only place we need that exception?

@carljm
carljm self-requested a review October 21, 2026 22:08
@charliermarsh
charliermarsh force-pushed the charlie/ty-class-specialization-representation branch from 5a05782 to f96180b Compare October 24, 2026 15:16
@astral-automations-bot

astral-automations-bot Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Typing conformance results

No changes detected ✅

Current numbers
The percentage of diagnostics emitted that were expected errors held steady at 98.60%. The percentage of expected errors that received a diagnostic held steady at 98.86%. The number of fully passing files held steady at 137/146.

Results for 3ba5fa8 · Workflow run

@astral-automations-bot

astral-automations-bot Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Memory usage report

Memory usage unchanged ✅


Results for 3ba5fa8 · Workflow run

@astral-automations-bot

astral-automations-bot Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

ecosystem-analyzer results

No diagnostic changes detected ✅

Flaky changes detected. This PR summary excludes flaky changes; see the HTML report for details.

Full report with detailed diff · Timing results


Results for 3ba5fa8 · Workflow run

@charliermarsh
charliermarsh force-pushed the charlie/ty-class-specialization-representation branch from f96180b to 3ba5fa8 Compare November 5, 2026 16:33
@codspeed

codspeed Bot commented Nov 5, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 5.96%

1 warning

⚠️ 1 benchmark with a significant change ran on different hardware

Its base and head results were measured on machines with a different CPU, core count or memory, so the change might not come from the code. Run base and head on the same runner type.

Learn more about Macro Runners

See affected benchmarks

⚡ 1 improved benchmark
✅ 233 untouched benchmarks
⏩ 90 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
⚡ CPU Simulation ty_micro[typed_dict_get_large_literal_union] 372.9 ms 351.9 ms +5.96%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing charlie/ty-class-specialization-representation (3ba5fa8) with main (c86652a)

Open in CodSpeed

Footnotes

  1. 90 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

This branch was previously deployed

1 inactive deployment
automations — 3ba5fa8c Deployed Nov 5, 2026 by charliermarsh via security-review / security review #97373
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

internal An internal refactor or improvement (not for changelogs) ty The ty type checker

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants