Repository navigation
[ty] Separate specialized class objects from runtime generic aliases - #29403
charliermarsh wants to merge 5 commits into
Conversation
Memory usage reportMemory usage unchanged ✅ |
Typing conformance resultsNo changes detected ✅Current numbersThe percentage of diagnostics emitted that were expected errors held steady at 98.24%. The percentage of expected errors that received a diagnostic held steady at 98.24%. The number of fully passing files held steady at 134/146. |
|
396841a to
d16e3f8
Compare
d16e3f8 to
97b100b
Compare
97b100b to
90f17cd
Compare
| } | ||
|
|
||
| match (source, target) { | ||
| (Type::GenericAlias(alias), _) => { |
There was a problem hiding this comment.
Doing this for redundancy/equivalence relations makes a union of a class object and its runtime alias collapse to whichever one was added first, if they have the same specialization - unions should be independent of order.
This is probably acceptable temporarily until #29405, though. I don't think we really want to land this PR without #29405 regardless.
There was a problem hiding this comment.
Yeah that's correct. These should land together! I just think it's easier review this way.
| use crate::Db; | ||
| use crate::types::{ClassSpecialization, Type}; | ||
|
|
||
| /// Recover the NumPy scalar class objects that its type definitions describe as generic aliases. |
There was a problem hiding this comment.
Nit: this is a pretty opaque/compressed description of the situation. I'd expand it slightly, e.g. "Numpy's type stubs pretend that some runtime class objects are generic aliases; identify them and return a class-object type for them instead. TODO We may be able to remove this special-casing once all supported numpy versions have numpy/numpy#33052".
Also we should make sure this special-casing doesn't have any negative effect (I think ideally no effect at all) if we are dealing with a numpy version that has that PR included.
| Type::SpecializedClass(alias) | Type::GenericAlias(alias) => { | ||
| builder.into_diagnostic(format_args!( | ||
| "`types.GenericAlias` instance `{}` is not a valid class", |
There was a problem hiding this comment.
At the very least, this error message no longer looks correct. And I think the code change might not be correct, either? But it looks like this is fixed later in the stack.
There was a problem hiding this comment.
Yeah that's right.
| if alias.origin(db).is_tuple(db) => | ||
| { | ||
| Some(Cow::Owned(TupleSpec::homogeneous(todo_type!( | ||
| "*tuple[] annotations" |
There was a problem hiding this comment.
I don't think this is right -- I think this should defer to normal class-object iterability?
There was a problem hiding this comment.
I guess this is maybe another fix that belongs in the second PR, given the intended split? But (at least as of last night) I don't think it was fixed in the second PR.
There was a problem hiding this comment.
Yeah, hmm, it's now fixed in #29403 (this PR), so not entirely behavior-preserving.
| // It's not valid to use a generic alias as the second argument to `isinstance()` or `issubclass()`, | ||
| // e.g. `isinstance(x, list[int])` fails at runtime. | ||
| Type::GenericAlias(_) => None, | ||
| // We do not currently narrow against specialized class objects. |
There was a problem hiding this comment.
But we probably could, and treat them as their unspecialized class? Could be worth a TODO at least. (The weird case with the numpy types not being exact could be an issue here, since negative narrowing on e.g. np.int64 can't exclude all signedinteger.)
| /// A class object carrying specialization information. The class may be a subclass of the | ||
| /// specialization's origin, so the origin does not necessarily identify the runtime class. |
There was a problem hiding this comment.
I don't think "may be a subclass" is the semantics we want for this type -- it should be exact. SubclassOf should be used for the inexact case. And I don't think the inexact semantics are currently consistently applied in this stack, either: subtype checking seems to accept subclasses, but disjointness checking treats differing origins as disjoint.
Codex is suggesting to me that the "may be a subclass" here really just comes from the numpy special case, where e.g. np.int64 is a subclass of signedinteger at runtime. I think we should standardize on SpecializedClass being more like ClassLiteral and not admitting subclasses, and find a different way to avoid problems from the numpy case (though without looking into it more I'm not sure what that way should be.)
There was a problem hiding this comment.
Ok so I probably should have taken the time to look into this deeper and recommend a specific direction, because the updated approach of using SubclassOf for the numpy types and then adding a bunch of machinery to make them still usable in type annotations, seems way too invasive to me for this already-kind-of-obsolete compatibility hack.
I think a better direction might be to just let these numpy types continue to be represented as GenericAlias, as the (old) stubs say they are, and instead try to add targeted exceptions to suppress specific false positives that are actually problematic. I think "assignability to type types" might actually be the only place we need that exception?
There was a problem hiding this comment.
Ok, I reverted this, so the NumPy types use GenericAlias, and we have assignability exceptions in #29405. I was liberal with TODOs there to mark each one...
f206764 to
28aa15c
Compare
| /// A class object carrying specialization information. The class may be a subclass of the | ||
| /// specialization's origin, so the origin does not necessarily identify the runtime class. |
There was a problem hiding this comment.
Ok so I probably should have taken the time to look into this deeper and recommend a specific direction, because the updated approach of using SubclassOf for the numpy types and then adding a bunch of machinery to make them still usable in type annotations, seems way too invasive to me for this already-kind-of-obsolete compatibility hack.
I think a better direction might be to just let these numpy types continue to be represented as GenericAlias, as the (old) stubs say they are, and instead try to add targeted exceptions to suppress specific false positives that are actually problematic. I think "assignability to type types" might actually be the only place we need that exception?
5a05782 to
f96180b
Compare
Typing conformance resultsNo changes detected ✅Current numbersThe percentage of diagnostics emitted that were expected errors held steady at 98.60%. The percentage of expected errors that received a diagnostic held steady at 98.86%. The number of fully passing files held steady at 137/146. Results for 3ba5fa8 · Workflow run |
Memory usage reportMemory usage unchanged ✅ Results for 3ba5fa8 · Workflow run |
|
f96180b to
3ba5fa8
Compare
Merging this PR will improve performance by 5.96%1 warning
Performance Changes
Tip Curious why performance improved? Comment Comparing Footnotes
|
Summary
This PR adds a
Typevariant to distinguish runtime generic aliases from class objects that carry specialization information:Both
aliasandclscarry the specializationBox[int]in our type model. Butaliasis an alias object, whileclsis the actualBoxclass. Theintonclsrecords static information about the instance it came from; it does not mean Python created a separate class forBox[int].Previously, we represented both values with
Type::GenericAlias. We need to distinguish them to reject aliases where Python requires a class, such as the second argument toisinstance. (If we instead representclswithClassLiteral, we lose its specialization.)This PR is entirely behavior-preserving, but paves the way for #29405, which uses it to reject aliases where a class is required.