You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Degraded performance in versions >=1.1.412, worse with multiple threads #11751
Describe the bug
We have a monorepo where we use pyright in CI to ensure proper typing. Up to version 1.1.411 using 8 threads it would take ~1 minute to run pyright on our codebase (~5K python files). Starting in 1.1.412 it takes ~22m.
Code or Screenshots
Testing with following script:
With THREADS=1, it goes from ~3.5 minutes to 45 minutes:
Version
Real
User
Sys
pyright 1.1.410
3m37.992s
5m41.581s
0m12.607s
pyright 1.1.411
3m38.553s
5m39.328s
0m11.544s
pyright 1.1.412
15m53.537s
20m19.392s
0m19.188s
pyright 1.1.413
55m29.642s
60m10.076s
0m37.799s
pyright 1.1.414
44m14.298s
24m42.466s
0m59.330s
With THREADS=8, it goes from ~1 to ~22 minutes:
Version
Real
User
Sys
pyright 1.1.410
1m4.128s
9m0.630s
0m57.164s
pyright 1.1.411
1m0.468s
8m42.691s
0m44.016s
pyright 1.1.412
22m10.466s
31m45.013s
0m55.151s
pyright 1.1.413
22m27.549s
32m52.530s
1m16.357s
pyright 1.1.414
22m14.179s
32m37.119s
1m31.261s
As an added insight, with versions >=412, when using multiple threads the first N-1 workers finish as quickly as in versions <=411, and the last worker lingers at 100% CPU for the remainder of the time.
VS Code extension or command-line
Using pyright in command line, multiple versions.
OK, I found a file that when added to the exclude list, makes the pyright runtime go back down to ~1m30s. I can't share the code, but I relied on an LLM to help me debug it and create a shareable repro. Here's the outcome.
The problem turned out to be two comprehensions in the same function reusing the same walrus target name, in sibling arguments of a nested constructor call — Outer(a=Inner(...), b=Inner(...)). A walrus inside a comprehension binds in the enclosing function scope, so those aren't two independent variables; they're one variable written from two argument positions of the same call.
The file in our codebase is 297 lines and takes 101.6s. Renaming the two walrus targets apart — semantically a no-op — takes it to 1.2s. In the reduced repro below, the two halves are identical except for that rename: 55.3s vs 3.1s.
Each of these ingredients is required; remove any one and the file checks in under a second:
the callee's __init__ is synthesized rather than hand-written (@dataclass, attrs, NamedTuple and TypedDict all reproduce; a hand-written __init__ with the identical signature does not)
the call nests two levels
the parameter's type has several __init__ overloads
two or more sibling arguments bind the same walrus name
Separately: I suspect #11759 is the same root cause rather than a distinct TypedDict-sizing issue. A TypedDict callee in this repro behaves exactly like a @dataclass one, its constructor being likewise synthesized. Might be worth looking at them together.
This is the script to showcase the problem:
"""Self-contained repro for pyright argument-inference blowup (pyright #11751).stdlib only -- no third-party imports. Tested with pyright 1.1.414 on macOS. $ time pyright pyright_perf_repro.py # both halves, ~59 sTo time the two halves separately (the file is split on its marker comments): $ sed '/^# ---8<--- FAST/,$d' pyright_perf_repro.py > slow_only.py $ sed '/^# ---8<--- SLOW/,/^# ---8<--- FAST/d' pyright_perf_repro.py > fast_only.py $ time pyright slow_only.py $ time pyright fast_only.pyTHE PATTERN (all four ingredients are required): 1. A callee whose __init__ is SYNTHESIZED rather than hand-written (@dataclass, attrs, NamedTuple, or TypedDict -- all four reproduce). 2. Nested construction: Outer(s0=Inner(...), s1=Inner(...)). 3. A parameter whose type is a generic class with MANY overloaded __init__ signatures (here `fd`, modelled on frozendict / Mapping-likes). 4. Two or more sibling arguments each containing a comprehension with a walrus (:=) THAT BINDS THE SAME NAME.Ingredient 4 is the trigger. PEP 572 makes a walrus inside a comprehension bindin the *enclosing function* scope, so `c` here is one variable in `f` assignedfrom two different argument positions. Evaluating `c` in the second argumentforces code-flow analysis back through the first argument's assignment -- andbecause argument expressions are re-evaluated speculatively once per overload,the cost of each sibling multiplies by the cost of all siblings to its left.MEASURED (pyright 1.1.414, 6 overloads on `fd`, @dataclass): siblings sharing one walrus name time --------------------------------- -------- 1 0.5 s 2 58 s 2, but each walrus given its own distinct name (c0 / c1) 3.1 s <-- the fix overloads on `fd` (2 siblings) time --------------------------------- -------- 3 0.8 s 4 3.7 s 5 30.5 s 6 55 s With only 4 overloads, going 1 -> 2 -> 3 siblings gives 0.5 s -> 3.8 s -> >20 min.Replacing @dataclass with a hand-written __init__ of the *identical* signaturedrops the 58 s case to 0.4 s. Hoisting each comprehension into a local variablebefore the call also drops it to ~0.9 s. Neither changes runtime semantics.This is very likely the same root cause as #11759 (large TypedDicts): a TypedDictconstructor is also a synthesized call target, and the blowup below reproduceswith `class Inner(TypedDict)` just as it does with @dataclass."""fromcollections.abcimportIterable, Iterator, MappingfromtypingimportAny, TypeVar, overloadimportdataclassesK=TypeVar("K")
V=TypeVar("V")
classfd(Mapping[K, V]):
"""A frozendict-shaped Mapping: several self-specializing __init__ overloads."""@overloaddef__init__(self): ...
@overloaddef__init__(self: "fd[str, V]", **kwargs: V): ... # pyright: ignore@overloaddef__init__(self: "fd[K, V]", mapping: Mapping[K, V], /): ... # pyright: ignore@overloaddef__init__(self: "fd[str, V]", mapping: Mapping[str, V], /, **kwargs: V): ... # pyright: ignore@overloaddef__init__(self: "fd[K, V]", iterable: Iterable[tuple[K, V]], /): ... # pyright: ignore@overloaddef__init__(self: "fd[str, V]", iterable: Iterable[tuple[str, V]], /, **kwargs: V): ... # pyright: ignoredef__init__(self, *args: Any, **kwargs: Any) ->None: ...
def__getitem__(self, key: K) ->V: ...
def__len__(self) ->int: ...
def__iter__(self) ->Iterator[K]: ...
COLS= ["a", "b", "c"]
@dataclasses.dataclass(frozen=True, kw_only=True)classInner:
name: strmeta: fd[str, Any] =dataclasses.field(default_factory=fd)
@dataclasses.dataclass(frozen=True, kw_only=True)classOuter:
name: strs0: Inners1: Inner# ---8<--- SLOW ---defslow(items: list[tuple[Any, Any]]) ->None:
"""~58 s. Both comprehensions bind the SAME walrus name `c`."""Outer(
name="x",
s0=Inner(name="i", meta=fd({c: vfork, vinitemsif (c:=str(k).removesuffix("_s0")) inCOLS})),
s1=Inner(name="i", meta=fd({c: vfork, vinitemsif (c:=str(k).removesuffix("_s1")) inCOLS})),
)
# ---8<--- FAST ---deffast(items: list[tuple[Any, Any]]) ->None:
"""~3 s. Identical except each walrus binds its own name. Same runtime semantics."""Outer(
name="x",
s0=Inner(name="i", meta=fd({c0: vfork, vinitemsif (c0:=str(k).removesuffix("_s0")) inCOLS})),
s1=Inner(name="i", meta=fd({c1: vfork, vinitemsif (c1:=str(k).removesuffix("_s1")) inCOLS})),
)
Save as pyright_perf_repro.py, and run with either slow() or fast() using 1.1.414:
sed '/^# ---8<--- FAST/,$d' pyright_perf_repro.py > slow_only.py
sed '/^# ---8<--- SLOW/,/^# ---8<--- FAST/d' pyright_perf_repro.py > fast_only.py
time pyright fast_only.py # ~3stime pyright slow_only.py # ~57s
Activity
rchiodo commentedon Sep 14, 2026
There's likely a regression but we'd need the source code to identify the issue. Can you share your source or an example the shows the problem?
rgoya commentedon Sep 14, 2026
I can try to scan independent subfolders with
411and412and see if I can narrow down a specific location where the time increases.Is there any logging that
pyrightcan output? I tried your recommendation in a different issue of adding:to my configuration; but that didn't seem to do anything (even tried lowering it to 10).
rchiodo commentedon Sep 14, 2026
Yeah those two settings should output anything that's taking a long time. Command line pyright might need the '--verbose' flag though
rgoya commentedon Sep 15, 2026
OK, I found a file that when added to the
excludelist, makes thepyrightruntime go back down to ~1m30s. I can't share the code, but I relied on an LLM to help me debug it and create a shareable repro. Here's the outcome.The problem turned out to be two comprehensions in the same function reusing the same walrus target name, in sibling arguments of a nested constructor call —
Outer(a=Inner(...), b=Inner(...)). A walrus inside a comprehension binds in the enclosing function scope, so those aren't two independent variables; they're one variable written from two argument positions of the same call.The file in our codebase is 297 lines and takes 101.6s. Renaming the two walrus targets apart — semantically a no-op — takes it to 1.2s. In the reduced repro below, the two halves are identical except for that rename: 55.3s vs 3.1s.
Each of these ingredients is required; remove any one and the file checks in under a second:
__init__is synthesized rather than hand-written (@dataclass,attrs,NamedTupleandTypedDictall reproduce; a hand-written__init__with the identical signature does not)__init__overloadsSeparately: I suspect #11759 is the same root cause rather than a distinct TypedDict-sizing issue. A
TypedDictcallee in this repro behaves exactly like a@dataclassone, its constructor being likewise synthesized. Might be worth looking at them together.This is the script to showcase the problem:
Save as
pyright_perf_repro.py, and run with eitherslow()orfast()using1.1.414:rchiodo commentedon Sep 15, 2026
Thanks a lot. That should really help figure out the issue.
rchiodo commentedon Sep 15, 2026
Okay looks like this was caused by #11601, which I reverted because it also caused other problems. So this should be fixed in our main branch.
rgoya commentedon Sep 15, 2026
Awesome, thank you. Out of curiosity, did you look into whether this also solves #11759?
rchiodo commentedon Sep 15, 2026
No sorry, totally separate issue