Skip to content

Degraded performance in versions >=1.1.412, worse with multiple threads #11751

Closed

Description

@rgoya

Describe the bug
We have a monorepo where we use pyright in CI to ensure proper typing. Up to version 1.1.411 using 8 threads it would take ~1 minute to run pyright on our codebase (~5K python files). Starting in 1.1.412 it takes ~22m.

Code or Screenshots
Testing with following script:

#!/usr/bin/env bash
for THREADS in 1 8
do
    for VERSION in 410 411 412 413 414
    do
        pnpm add -D "pyright@1.1.$VERSION" --no-optional 2>&1
        ./node_modules/.bin/pyright --version 1>&2
        time ./node_modules/.bin/pyright --threads $THREADS src/ 1>pyright.t$THREADS.v1.1.$VERSION.out 2>pyright.t$THREADS.v1.1.$VERSION.err
    done
done

With THREADS=1, it goes from ~3.5 minutes to 45 minutes:

Version Real User Sys
pyright 1.1.410 3m37.992s 5m41.581s 0m12.607s
pyright 1.1.411 3m38.553s 5m39.328s 0m11.544s
pyright 1.1.412 15m53.537s 20m19.392s 0m19.188s
pyright 1.1.413 55m29.642s 60m10.076s 0m37.799s
pyright 1.1.414 44m14.298s 24m42.466s 0m59.330s

With THREADS=8, it goes from ~1 to ~22 minutes:

Version Real User Sys
pyright 1.1.410 1m4.128s 9m0.630s 0m57.164s
pyright 1.1.411 1m0.468s 8m42.691s 0m44.016s
pyright 1.1.412 22m10.466s 31m45.013s 0m55.151s
pyright 1.1.413 22m27.549s 32m52.530s 1m16.357s
pyright 1.1.414 22m14.179s 32m37.119s 1m31.261s

As an added insight, with versions >=412, when using multiple threads the first N-1 workers finish as quickly as in versions <=411, and the last worker lingers at 100% CPU for the remainder of the time.

VS Code extension or command-line
Using pyright in command line, multiple versions.

We have the following in our pyproject.toml:

[tool.pyright]
exclude = [
  "**/.*",
  "**/__pycache__",
  "**/node_modules",
]
typeCheckingMode = "basic"
strictParameterNoneValue = false
reportUnusedExpression = false
reportPrivateImportUsage = false
reportUnnecessaryTypeIgnoreComment = false
stubPath = "<path_to>/stubs"

Activity

  1. rchiodo commented on Sep 14, 2026

    @rchiodo
    Collaborator

    There's likely a regression but we'd need the source code to identify the issue. Can you share your source or an example the shows the problem?

  2. rgoya commented on Sep 14, 2026

    @rgoya
    Author

    I can try to scan independent subfolders with 411 and 412 and see if I can narrow down a specific location where the time increases.

    Is there any logging that pyright can output? I tried your recommendation in a different issue of adding:

    logTypeEvaluationTime = true
    typeEvaluationTimeThreshold = 500
    

    to my configuration; but that didn't seem to do anything (even tried lowering it to 10).

  3. rchiodo commented on Sep 14, 2026

    @rchiodo
    Collaborator

    Yeah those two settings should output anything that's taking a long time. Command line pyright might need the '--verbose' flag though

  4. rgoya commented on Sep 15, 2026

    @rgoya
    Author

    OK, I found a file that when added to the exclude list, makes the pyright runtime go back down to ~1m30s. I can't share the code, but I relied on an LLM to help me debug it and create a shareable repro. Here's the outcome.

    The problem turned out to be two comprehensions in the same function reusing the same walrus target name, in sibling arguments of a nested constructor call — Outer(a=Inner(...), b=Inner(...)). A walrus inside a comprehension binds in the enclosing function scope, so those aren't two independent variables; they're one variable written from two argument positions of the same call.

    The file in our codebase is 297 lines and takes 101.6s. Renaming the two walrus targets apart — semantically a no-op — takes it to 1.2s. In the reduced repro below, the two halves are identical except for that rename: 55.3s vs 3.1s.

    Each of these ingredients is required; remove any one and the file checks in under a second:

    • the callee's __init__ is synthesized rather than hand-written (@dataclass, attrs, NamedTuple and TypedDict all reproduce; a hand-written __init__ with the identical signature does not)
    • the call nests two levels
    • the parameter's type has several __init__ overloads
    • two or more sibling arguments bind the same walrus name

    Separately: I suspect #11759 is the same root cause rather than a distinct TypedDict-sizing issue. A TypedDict callee in this repro behaves exactly like a @dataclass one, its constructor being likewise synthesized. Might be worth looking at them together.

    This is the script to showcase the problem:

    """Self-contained repro for pyright argument-inference blowup (pyright #11751).
    
    stdlib only -- no third-party imports. Tested with pyright 1.1.414 on macOS.
    
        $ time pyright pyright_perf_repro.py          # both halves, ~59 s
    
    To time the two halves separately (the file is split on its marker comments):
    
        $ sed '/^# ---8<--- FAST/,$d'                  pyright_perf_repro.py > slow_only.py
        $ sed '/^# ---8<--- SLOW/,/^# ---8<--- FAST/d'  pyright_perf_repro.py > fast_only.py
        $ time pyright slow_only.py
        $ time pyright fast_only.py
    
    THE PATTERN (all four ingredients are required):
    
      1. A callee whose __init__ is SYNTHESIZED rather than hand-written
         (@dataclass, attrs, NamedTuple, or TypedDict -- all four reproduce).
      2. Nested construction: Outer(s0=Inner(...), s1=Inner(...)).
      3. A parameter whose type is a generic class with MANY overloaded
         __init__ signatures (here `fd`, modelled on frozendict / Mapping-likes).
      4. Two or more sibling arguments each containing a comprehension with a
         walrus (:=) THAT BINDS THE SAME NAME.
    
    Ingredient 4 is the trigger. PEP 572 makes a walrus inside a comprehension bind
    in the *enclosing function* scope, so `c` here is one variable in `f` assigned
    from two different argument positions. Evaluating `c` in the second argument
    forces code-flow analysis back through the first argument's assignment -- and
    because argument expressions are re-evaluated speculatively once per overload,
    the cost of each sibling multiplies by the cost of all siblings to its left.
    
    MEASURED (pyright 1.1.414, 6 overloads on `fd`, @dataclass):
    
      siblings sharing one walrus name    time
      ---------------------------------   --------
      1                                   0.5 s
      2                                   58   s
      2, but each walrus given its own
        distinct name (c0 / c1)           3.1 s     <-- the fix
    
      overloads on `fd` (2 siblings)      time
      ---------------------------------   --------
      3                                   0.8 s
      4                                   3.7 s
      5                                   30.5 s
      6                                   55   s
    
      With only 4 overloads, going 1 -> 2 -> 3 siblings gives 0.5 s -> 3.8 s -> >20 min.
    
    Replacing @dataclass with a hand-written __init__ of the *identical* signature
    drops the 58 s case to 0.4 s. Hoisting each comprehension into a local variable
    before the call also drops it to ~0.9 s. Neither changes runtime semantics.
    
    This is very likely the same root cause as #11759 (large TypedDicts): a TypedDict
    constructor is also a synthesized call target, and the blowup below reproduces
    with `class Inner(TypedDict)` just as it does with @dataclass.
    """
    
    from collections.abc import Iterable, Iterator, Mapping
    from typing import Any, TypeVar, overload
    import dataclasses
    
    K = TypeVar("K")
    V = TypeVar("V")
    
    
    class fd(Mapping[K, V]):
        """A frozendict-shaped Mapping: several self-specializing __init__ overloads."""
    
        @overload
        def __init__(self): ...
        @overload
        def __init__(self: "fd[str, V]", **kwargs: V): ...  # pyright: ignore
        @overload
        def __init__(self: "fd[K, V]", mapping: Mapping[K, V], /): ...  # pyright: ignore
        @overload
        def __init__(self: "fd[str, V]", mapping: Mapping[str, V], /, **kwargs: V): ...  # pyright: ignore
        @overload
        def __init__(self: "fd[K, V]", iterable: Iterable[tuple[K, V]], /): ...  # pyright: ignore
        @overload
        def __init__(self: "fd[str, V]", iterable: Iterable[tuple[str, V]], /, **kwargs: V): ...  # pyright: ignore
        def __init__(self, *args: Any, **kwargs: Any) -> None: ...
    
        def __getitem__(self, key: K) -> V: ...
        def __len__(self) -> int: ...
        def __iter__(self) -> Iterator[K]: ...
    
    
    COLS = ["a", "b", "c"]
    
    
    @dataclasses.dataclass(frozen=True, kw_only=True)
    class Inner:
        name: str
        meta: fd[str, Any] = dataclasses.field(default_factory=fd)
    
    
    @dataclasses.dataclass(frozen=True, kw_only=True)
    class Outer:
        name: str
        s0: Inner
        s1: Inner
    
    
    # ---8<--- SLOW ---
    def slow(items: list[tuple[Any, Any]]) -> None:
        """~58 s. Both comprehensions bind the SAME walrus name `c`."""
        Outer(
            name="x",
            s0=Inner(name="i", meta=fd({c: v for k, v in items if (c := str(k).removesuffix("_s0")) in COLS})),
            s1=Inner(name="i", meta=fd({c: v for k, v in items if (c := str(k).removesuffix("_s1")) in COLS})),
        )
    
    
    # ---8<--- FAST ---
    def fast(items: list[tuple[Any, Any]]) -> None:
        """~3 s. Identical except each walrus binds its own name. Same runtime semantics."""
        Outer(
            name="x",
            s0=Inner(name="i", meta=fd({c0: v for k, v in items if (c0 := str(k).removesuffix("_s0")) in COLS})),
            s1=Inner(name="i", meta=fd({c1: v for k, v in items if (c1 := str(k).removesuffix("_s1")) in COLS})),
        )

    Save as pyright_perf_repro.py, and run with either slow() or fast() using 1.1.414:

    sed '/^# ---8<--- FAST/,$d' pyright_perf_repro.py > slow_only.py
    sed '/^# ---8<--- SLOW/,/^# ---8<--- FAST/d' pyright_perf_repro.py > fast_only.py
    time pyright fast_only.py   #  ~3s
    time pyright slow_only.py   #  ~57s
  5. rchiodo commented on Sep 15, 2026

    @rchiodo
    Collaborator

    Thanks a lot. That should really help figure out the issue.

  6. rchiodo commented on Sep 15, 2026

    @rchiodo
    Collaborator

    Okay looks like this was caused by #11601, which I reverted because it also caused other problems. So this should be fixed in our main branch.

  7. rgoya commented on Sep 15, 2026

    @rgoya
    Author

    Awesome, thank you. Out of curiosity, did you look into whether this also solves #11759?

  8. rchiodo commented on Sep 15, 2026

    @rchiodo
    Collaborator

    No sorry, totally separate issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

addressed in next versionIssue is fixed and will appear in next published versionbugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions