Skip to content

Short-circuit isTypeVarSame subtype checks - #11681

Open
Bill Schnurr (bschnurr) wants to merge 7 commits into
microsoft:mainfrom
bschnurr:perf/type-utils-short-circuit
Open

Bill Schnurr (bschnurr) wants to merge 7 commits into
microsoft:mainfrom
bschnurr:perf/type-utils-short-circuit

Conversation

@bschnurr

@bschnurr Bill Schnurr (bschnurr) commented Aug 26, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Fixes allSubtypes so union checks return callback results.
  • Delegates directly to Array.every for allocation-free early termination.
  • Uses allSubtypes in isTypeVarSame to stop at the first incompatible subtype instead of walking the remaining union entries with no-op callbacks.
  • Preserves someSubtypes and adds direct unit coverage for both helpers.

This is the safe typeUtils portion split from #11419. It intentionally excludes the literal, tuple, derivesFromAnyOrUnknown, and NoReturn early exits that produced mypy_primer differences.

Validation

  • pnpm exec jest typeUtils.test --forceExit
  • pnpm run build
  • pnpm run check:prettier
  • pnpm run check:eslint
  • full hosted validation: 16 checks passed
  • exact-head mypy_primer rerun: all eight diff artifacts empty

@bschnurr

Copy link
Copy Markdown
Member Author

/benchmark

@rchiodo

Rich Chiodo (rchiodo) commented Aug 26, 2026 •

Copy link
Copy Markdown
Collaborator

🔒 Automated review in progress — Rich Chiodo (@rchiodo) is auto-reviewing this PR.

@github-actions

This comment has been minimized.

@rchiodo

Copy link
Copy Markdown
Collaborator

Verification: The relevant tests could not be fully run in the isolated environment; this review is not fully verified.

@rchiodo Rich Chiodo (rchiodo) left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved via Review Center.

@rchiodo Rich Chiodo (rchiodo) added the review-auto:approved Automated review: no blocking findings (approval posted). label Aug 26, 2026
@github-actions

This comment has been minimized.

@bschnurr
Bill Schnurr (bschnurr) force-pushed the perf/type-utils-short-circuit branch from 72e74f2 to 2901d5f Compare August 26, 2026 18:36
@bschnurr Bill Schnurr (bschnurr) changed the title Fix and short-circuit subtype helper paths Short-circuit subtype helper paths Aug 26, 2026
@rchiodo

Copy link
Copy Markdown
Collaborator

Verification: The relevant tests could not be fully run in the isolated environment; this review is not fully verified.

@github-actions

This comment has been minimized.

@bschnurr
Bill Schnurr (bschnurr) force-pushed the perf/type-utils-short-circuit branch from 2901d5f to ce7460c Compare August 26, 2026 19:30
@bschnurr Bill Schnurr (bschnurr) changed the title Short-circuit subtype helper paths Fix allSubtypes callback handling Aug 26, 2026
@rchiodo

Copy link
Copy Markdown
Collaborator

Verification: The relevant tests could not be fully run in the isolated environment; this review is not fully verified.

@github-actions

This comment has been minimized.

@bschnurr

Copy link
Copy Markdown
Member Author

/benchmark

Fix allSubtypes callback handling and delegate union traversal directly to Array.every. Use it in isTypeVarSame to stop the subtype scan at the first incompatibility instead of invoking no-op callbacks for the remainder.

Add direct allSubtypes and someSubtypes coverage.
@bschnurr
Bill Schnurr (bschnurr) force-pushed the perf/type-utils-short-circuit branch from ce7460c to c49ce2f Compare August 26, 2026 23:47
@bschnurr Bill Schnurr (bschnurr) changed the title Fix allSubtypes callback handling Short-circuit isTypeVarSame subtype checks Aug 26, 2026
@rchiodo

Copy link
Copy Markdown
Collaborator

Verification: The relevant tests could not be fully run in the isolated environment; this review is not fully verified.

Result: could-not-verify

Summary: Verification could not run because no trusted `microsoft/pyright` sandbox image was configured and local execution was unauthorized. The PR adds focused coverage for union short-circuiting and non-union callbacks in `allSubtypes` and `someSubtypes`. No meaningful test execution occurred, so the verdict is **could-not-verify**.

Test runs: 1 not run

  • ⚠️ Not run | typeUtils unit tests | pnpm exec jest typeUtils.test --runInBand
⚠️ typeUtils unit tests diagnostic output
Container verification could not start: no trusted sandbox image is configured for microsoft/pyright; local execution was not authorized for this PR HEAD.

@github-actions

This comment has been minimized.

sourceNode === source && targetNode === target ? targetNode : undefined;
}

test('AllSubtypes', () => {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Issue · Please address or respond

Move both new tests above the existing file-scoped helper section. Appending tests after createFunction, createClass, and replacePair violates docs/pylancewiki/review/tests-helper-placement.md, which requires helpers at the end of their scope.

return classType;
}

function createUnion(...subtypes: UnionableType[]) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Issue · Please address or respond

Declare the rest parameter as ...subtypes: readonly UnionableType[]. The helper only reads this array and passes its elements individually downstream, so the mutable parameter violates docs/pylancewiki/review/readonly-by-default.md without a demonstrated mutability requirement.

@rchiodo

Copy link
Copy Markdown
Collaborator

Result: ⚠️ partially-verified

Verification details

Verification: Isolated verification observed failures that were not classified as caused by this PR: Dependency and test discovery.

Summary: Targeted Jest runs passed all 50 selected tests, including both PR-added helper tests and existing TypeVar/union regressions. Offline dependency installation and the internal TypeScript build also passed. Discovery failed only because the container snapshot lacks Git metadata. Hosted validation and mypy_primer were not reproduced, so verification is partial.

Test runs: 4 passed, 1 failed

  • ❌ Failed | unrelated to this PR | Dependency and test discovery | printf 'Sandbox profile: %s\n' "$AUTOMATION_SANDBOX_PROFILE"; git rev-parse HEAD; git status --short; node --version; pnpm --version; test -f pnpm-lock.yaml && printf 'pnpm lockfile present\n'; if test -d node_modules; then printf 'root node_modules present\n'; else printf 'root node_modules missing\n'; fi; if test -f packages/pyright-internal/node_modules/jest/bin/jest.js; then printf 'package-local Jest present\n'; else printf 'package-local Jest missing\n'; fi; cat packages/pyright-internal/jest.config.js; cat packages/pyright-internal/tsconfig.jest.json; git log -2 --oneline
  • ✅ Passed | Type utilities and type equality | node ./node_modules/jest/bin/jest.js src/tests/typeUtils.test.ts src/tests/typeEquality.test.ts --runInBand --forceExit
  • ✅ Passed | TypeVar and union analyzer regressions | node ./node_modules/jest/bin/jest.js src/tests/typeEvaluator2.test.ts src/tests/typeEvaluator4.test.ts src/tests/typeEvaluator7.test.ts --testNamePattern='^(ConstrainedTypeVar[0-9]+|TypeVar[0-9]+|Unions[0-9]+)$' --runInBand --forceExit
  • ✅ Passed | Offline dependency bootstrap | pnpm --filter pyright-internal install --offline --frozen-lockfile
  • ✅ Passed | Internal TypeScript build | pnpm run build
❌ Dependency and test discovery diagnostic output
Sandbox profile: typescript
fatal: not a git repository (or any parent up to mount point /)
v24.19.0
11.21.0
pnpm lockfile present
root node_modules missing
package-local Jest missing
Jest configuration files were readable.
[container exit=128]

@rchiodo Rich Chiodo (rchiodo) added review-auto:changes-requested Automated review: posted blocking findings to address. and removed review-auto:approved Automated review: no blocking findings (approval posted). labels Oct 8, 2026
@bschnurr

Copy link
Copy Markdown
Member Author

/benchmark

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

This comment has been minimized.

@rchiodo

Rich Chiodo (rchiodo) commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

Reviewed commit: 82c1872fc0bee8a7f44582a3c30a7167f2a44133

Result: 🔴 could-not-verify

Verification details

Verification: Automated verification is best effort. The PR author remains responsible for testing.

Summary: [unavailable] Container verification could not start and local execution was not authorized for this PR HEAD: C:\Users\rchiodo\AppData\Local\Programs\Podman\podman.EXE --connection pyrx-automation failed: stderr: Error: unable to start container "ec9cfc6b96f446f26c42dac78714f03f9b93a218557bcfdcb8597246a434459a": crun: open `memory.max` for writing: No such file or directory: OCI runtime attempted to invoke a command that was not found

Test runs: none recorded.

@rchiodo Rich Chiodo (rchiodo) added review-auto:approved Automated review: no blocking findings (approval posted). and removed review-auto:changes-requested Automated review: posted blocking findings to address. labels Nov 16, 2026
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

@rchiodo Rich Chiodo (rchiodo) left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a nice refactor. One thing with allSubtypes: it passes the callback straight to Array.every, which calls it with (subtype, index, array). someSubtypes and the non-union path only pass the subtype, so callbacks with optional positional arguments can behave differently.

For example, take a union ordered as a literal followed by a Container specialized with that literal. containsLiteralType gives [true, false] when called on each member with the default includeTypeArgs, but allSubtypes(union, containsLiteralType) returns true. The second call gets index 1 as includeTypeArgs and checks the type arguments. Wrapping it as allSubtypes(union, subtype => containsLiteralType(subtype)) returns false. Reverse the members and both forms return false. Passing the function directly type-checks, which makes this easy to miss.

Could we wrap the callback passed to every, like subtype => callback(subtype), so it only gets the subtype? A focused test would be useful too.

The base already returned false for every non-empty union, and there's no current in-repo production caller using a multi-parameter callback. External callers weren't checked. So this looks like a latent issue in the helper fix, not a demonstrated Python regression.

Two smaller optional things: could the existing createClass in typeUtils.test.ts cover these tests instead of adding createClassType? It gives the same results for these tests, though the helpers have different metadata and MROs.

Would a direct unit test for isTypeVarSame be useful too? There isn't one now, and the ten cases checked matched base and head. That's just a coverage suggestion, not an observed behavior change.

Class B: Pyright helper contract. Pass only the subtype to each predicate so optional arguments retain their defaults. Preserve short-circuit behavior and literal-detection precision.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bschnurr

Copy link
Copy Markdown
Member Author

Pushed 7603cc8 and merged current main in 6191331. allSubtypes now passes only the subtype to its predicate, preserving optional-argument defaults and short-circuiting. The new containsLiteralType regression fails on the previous PR head and passes with the fix, in both subtype orders.

The core build, 26 utility/equality tests, 38 TypeVar/union tests, 128 mypy-primer workflow tests, ESLint, and Prettier passed on the merged branch. python build\benchmark\test_compare_benchmarks.py has one failing YAML guard: the expected action pins differ from the weekly workflow. The same failure occurs on current main (7e29199f), with unchanged workflow and benchmark-test files.

The wrapper removes the direct Array.every delegation described in the PR summary; please update that wording. The follow-up has not been performance-benchmarked, and the previous benchmark does not cover this head.

@github-actions

Copy link
Copy Markdown
Contributor

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

@bschnurr

Copy link
Copy Markdown
Member Author

/benchmark

@github-actions

Copy link
Copy Markdown
Contributor

Compared candidate 49635c481aac614165a08a4943232f24d96d0a64 against its first parent 7e29199f7b3c5bddeda6956588a15552398a66ad.

Release history charts were not rendered for this run. See the workflow run for details.

Type checker benchmark

🟢 No performance regressions detected.

Regression threshold: 20.0%
Statistic: median
Variance guard: >1.0s time and >100.0 MB memory

Package Checker Files checked Time Time delta Peak memory Memory delta Status
ansible pyright 583 20.881s -0.2% 1080.6 MB +0.1% 🟢 Pass
click pyright 17 2.495s +4.5% 357.0 MB -0.5% 🟢 Pass
homeassistant pyright 9850 143.576s -2.2% 5989.4 MB +0.1% 🟢 Pass
numpy pyright 356 38.971s +0.2% 1215.3 MB -0.5% 🟢 Pass
pandas pyright 1459 111.488s +1.8% 3496.2 MB +0.6% 🟢 Pass
pytest pyright 243 13.494s +1.1% 862.4 MB +0.1% 🟢 Pass
requests pyright 19 1.592s +3.3% 348.7 MB -0.3% 🟢 Pass
torch pyright 2400 131.834s +3.4% 4666.1 MB -6.2% 🟢 Pass
transformers pyright 2901 100.727s +6.5% 4868.2 MB -0.1% 🟢 Pass

Pyright stats

Package Parsed/bound Checked Find Read Tokenize Parse Imports Bind Check Cycles
ansible 988 583 0.047s 0.077s 0.390s 0.757s 0.210s 0.937s 17.693s 0.000s
click 111 17 0.000s 0.013s 0.093s 0.167s 0.030s 0.220s 1.683s 0.000s
homeassistant 11154 9850 0.697s 0.700s 2.807s 5.097s 1.933s 6.523s 124.753s 0.000s
numpy 606 356 0.010s 0.057s 0.357s 0.660s 0.253s 0.857s 36.287s 0.000s
pandas 1899 1459 0.050s 0.203s 0.957s 2.007s 0.267s 2.470s 103.927s 0.000s
pytest 545 243 0.020s 0.050s 0.383s 0.530s 0.103s 0.717s 11.313s 0.000s
requests 167 19 0.000s 0.010s 0.070s 0.137s 0.037s 0.177s 0.930s 0.000s
torch 3078 2400 0.140s 0.390s 1.813s 3.197s 0.573s 4.620s 120.030s 0.000s
transformers 3536 2901 0.123s 0.417s 1.713s 2.843s 0.787s 7.200s 89.080s 0.000s

Classification: C (test maintenance). No type precision changes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

@github-actions

Copy link
Copy Markdown
Contributor

Compared candidate 423cf98d30f24597d20eee36a15f1aa988cbb224 against its first parent 2981f9f1ce41be5c15667da1f7a3cfda4f245089.

Release history charts were not rendered for this run. See the workflow run for details.

Type checker benchmark

🟢 No performance regressions detected.

Regression threshold: 20.0%
Statistic: median
Variance guard: >1.0s time and >100.0 MB memory

Package Checker Files checked Time Time delta Peak memory Memory delta Status
ansible pyright 583 20.338s -0.2% 1085.8 MB +0.3% 🟢 Pass
click pyright 17 3.065s -0.4% 358.1 MB -0.1% 🟢 Pass
homeassistant pyright 9850 115.207s +0.5% 5966.4 MB -0.2% 🟢 Pass
numpy pyright 356 21.012s +2.9% 1356.9 MB +1.2% 🟢 Pass
pandas pyright 1459 139.450s -0.9% 3918.6 MB +9.2% 🟢 Pass
pytest pyright 243 14.286s +1.3% 860.6 MB +0.2% 🟢 Pass
requests pyright 19 1.556s -2.8% 345.8 MB -1.0% 🟢 Pass
torch pyright 2400 156.855s +2.6% 4656.5 MB -0.9% 🟢 Pass
transformers pyright 2901 69.736s -1.6% 5199.3 MB +7.2% 🟢 Pass

Pyright stats

Package Parsed/bound Checked Find Read Tokenize Parse Imports Bind Check Cycles
ansible 988 583 0.040s 0.090s 0.413s 0.643s 0.203s 0.910s 17.403s 0.000s
click 111 17 0.000s 0.017s 0.110s 0.197s 0.037s 0.260s 2.127s 0.000s
homeassistant 11154 9850 0.527s 0.647s 2.417s 4.220s 1.610s 5.223s 97.313s 0.000s
numpy 606 356 0.010s 0.043s 0.217s 0.387s 0.140s 0.543s 19.167s 0.000s
pandas 1899 1459 0.070s 0.207s 1.220s 2.527s 0.317s 2.657s 130.840s 0.000s
pytest 545 243 0.020s 0.060s 0.410s 0.557s 0.110s 0.807s 11.843s 0.000s
requests 167 19 0.000s 0.010s 0.067s 0.130s 0.037s 0.177s 0.917s 0.000s
torch 3078 2400 0.183s 0.383s 1.993s 3.817s 0.630s 5.010s 141.590s 0.000s
transformers 3536 2901 0.087s 0.270s 1.293s 2.260s 0.617s 5.790s 59.440s 0.000s

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review-auto:approved Automated review: no blocking findings (approval posted).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants