Repository navigation
Fix exponential union growth when narrowing generic TypedDicts - #11850
Karan Rajeshbhai Mungara (withkarann) wants to merge 4 commits into
Conversation
|
@microsoft-github-policy-service agree |
🔴 Pyright CLI QA —
|
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
Yes, this is intended. It is the join rule from #11745: when two narrowed forms of the same TypedDict do not contain each other, the join falls back to the declared TypedDict. Before this PR, the check that applies that rule skipped any TypedDict with type arguments. It tested for no type arguments, while its comment says "different type arguments". Generic TypedDicts now get the same result as non-generic ones. The same code with a non-generic from typing import TypedDict
class Record(TypedDict, total=False):
common: int
left: int
right: int
def joined(flag: bool) -> int:
record: Record
if flag:
record = {"common": 1, "left": 2}
else:
record = {"common": 3, "right": 4}
return record["common"] # reportTypedDictNotRequiredAccess on mainKeeping both narrowed forms, as main does for generic TypedDicts, is what makes the union grow. Each
Generatorimport sys
n = int(sys.argv[1])
print("from typing import TypedDict\n")
print("class TD[T](TypedDict, total=False):")
for i in range(n):
print(f" a{i}: T\n b{i}: T")
print("\ndef f(v: int, c: bool) -> TD[int]:\n td: TD[int] = {}")
for i in range(n):
print(f" if c:\n td[\"a{i}\"] = v\n else:\n td[\"b{i}\"] = v")
print(" return td")Production CLI bundles, Apple M5 Pro, Node 24.19, one run each. Keeping only the keys that both sides provide would avoid this error for both kinds of TypedDict. But Another option is to merge generic copies only when one contains the other. That still fixes #11228, where each |
|
Copilot resolve the merge conflicts in this pull request |
Narrowed copies of a generic TypedDict were never merged at control flow join points, so each narrowing doubled the size of the union. Merge them when their type arguments match, as is already done for non-generic TypedDicts. Fixes microsoft#11228.
9652e47 to
5f82eec
Compare
|
Bill Schnurr (@bschnurr) Rebased on main and resolved the conflict. The only conflict was the sample file name: main added its own |
|
/benchmark |
|
According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅ |
|
Compared candidate View the full release history charts with the base and PR results. Execution time historyPeak memory historyType checker benchmark🟢 No performance regressions detected. Regression threshold:
Pyright stats
|
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Replayed |
|
🤖 AI-assisted review posted on behalf of Bill Schnurr (@bschnurr) The new same-type-arguments check can merge callables that differ in whether an argument is required. That lets a callback be replaced with an incompatible one: from typing import Generic, NotRequired, TypeVar, TypedDict
T = TypeVar("T")
class Box(TypedDict, Generic[T]):
callback: T
left: NotRequired[int]
right: NotRequired[int]
def wrap(callback: T) -> Box[T]:
return {"callback": callback}
def required(x: int) -> int:
return x
def optional(x: int = 0) -> int:
return x
def test(flag: bool) -> int:
safe = wrap(optional)
if flag:
value = wrap(required)
value["left"] = 1
else:
value = safe
value["right"] = 1
value["callback"] = required # Should be an error
return safe["callback"]()
print(test(False)) # TypeError: required() missing argument 'x'On base Could the join distinguish callable type arguments with different call behavior, with regression tests for both branch orders? The focused TypedDict and callable-identity tests passed locally. |
Classification B: the new join used an insufficiently strict type comparison. Honor callable behavior and related type metadata to preserve distinct type arguments, including nested forms, without reducing type precision. No typeshed change is involved. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
AI-assisted follow-up: pushed Validation: the core build, all 8 evaluator suites (1,462 tests), ESLint, and Prettier passed. A separate merge with The 12-key growth probe took a median 0.332 s of reported analysis time on both the original head and the repair (five paired runs with the same range-compatible dependencies). This is only a synthetic case, not a representative performance or memory benchmark. A separate pre-existing general union fast path can still collapse callable defaults when the TypedDict presence states are identical; this occurs on the pre-PR base too and is outside this repair. The PR's previously discussed common-key-presence precision tradeoff is unchanged. |
Summary
Fixes #11228.
When control flow joins two narrowed copies of the same TypedDict,
_addTypeIfUniquemerges them into one union entry. It only did this when neither copy had type arguments. Copies of a generic TypedDict such asTD[int]were never merged, so every conditional key assignment doubled the union: 14if ...: td["k"] = vblocks build a union of 2^14 entries.This change merges them when the type arguments are the same, which is what the comment above the check describes. Copies with different type arguments, such as
TD[int] | TD[str], are still kept apart.It also removes false
assert_typeerrors after such narrowing (expected "TD1[int]" but received "TD1[int]"), and the join behavior from #11745 now applies to generic TypedDicts too.Validation
typedDict31.py(testTypedDict31). On main it reports 4 errors: 3 falseassert_typemismatches and the expected one. With this change only the expected one remains.pnpm testinpackages/pyright-internal: all pass (one call hierarchy case inlanguageServer.test.tstimed out once under load and passes on rerun).pnpm run check: clean.Timing with the production CLI bundle, Apple M5 Pro, macOS 26.5, Node 24.19, median of 3 runs (one run for main at 16 keys):
Generator
python gen.py 14 > keys14.py && pyright keys14.pyNo change on ordinary code (CPU time, paired runs of the production bundles): pytest -1.8% +/- 2.5% over 15 runs, Home Assistant mean +0.7% / median -3.4% over 5 runs.
Note on
build/perfCompare.py: it runsdist/pyright.jsdirectly, soglobal.__rootDirectoryis not set and the bundle does not find typeshed (Import "sys" could not be resolved). I used its method but started each bundle through anindex.jswrapper likepackages/pyright/index.js.