Skip to content

Remove reportErrors check in relateVariances#55222

Draft
jakebailey wants to merge 16 commits intomicrosoft:mainfrom
jakebailey:fix-55217
Draft

Remove reportErrors check in relateVariances#55222
jakebailey wants to merge 16 commits intomicrosoft:mainfrom
jakebailey:fix-55217

Conversation

@jakebailey
Copy link
Member

@jakebailey jakebailey commented Jul 31, 2023 •
edited
Loading

Fixes #55217
Fixes #61524

As noted by @Andarist, this check is what causes things to differ between errors and non-error modes in relation, causing oddities when the same relation is observed in both ways but in different orders.

No doubt this is a bad idea but I wanted to run the test suite to see what happens.

Bunch of WIP code is left in the PR, I'll remove/refactor it if this code actually looks acceptable.

@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Jul 31, 2023
@jakebailey
Copy link
Member Author

jakebailey commented Jul 31, 2023

@typescript-bot test this
@typescript-bot test top100
@typescript-bot user test this
@typescript-bot user test tsserver
@typescript-bot test tsserver top100
@typescript-bot run dt
@typescript-bot perf test this
@typescript-bot pack this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 31, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the diff-based user code test suite on this PR at df695d9. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 31, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the diff-based user code test suite (tsserver) on this PR at df695d9. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 31, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the tarball bundle task on this PR at df695d9. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 31, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the extended test suite on this PR at df695d9. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 31, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the parallelized Definitely Typed test suite on this PR at df695d9. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 31, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the perf test suite on this PR at df695d9. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 31, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the diff-based top-repos suite on this PR at df695d9. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 31, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the diff-based top-repos suite (tsserver) on this PR at df695d9. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 31, 2023 •
edited
Loading

Hey @jakebailey, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/156090/artifacts?artifactName=tgz&fileId=4E1314C8F2DBC21591658BB650878A546D6F6E3C21856E1045D90F437FA08F0902&fileName=/typescript-5.2.0-insiders.20230731.tgz"
    }
}

and then running npm install.

There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/pr-build@5.2.0-pr-55222-9".;

jakebailey
jakebailey commented Jul 31, 2023
View reviewed changes
jakebailey
jakebailey commented Jul 31, 2023
View reviewed changes
src/compiler/checker.ts Outdated
@@ -21951,7 +21956,7 @@ export function createTypeChecker(host: TypeCheckerHost): TypeChecker {
// reveal the reason).
// We can switch on `reportErrors` here, since varianceCheckFailed guarantees we return `False`,
Copy link
Member Author

@jakebailey jakebailey Jul 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This mentions the code I deleted, so I'd have to delete this if this fix is right.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 1, 2023

@jakebailey Here are the results of running the user test suite comparing main and refs/pull/55222/merge:

Everything looks good!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 1, 2023

@jakebailey Here are the results of running the user test suite comparing main and refs/pull/55222/merge:

There were infrastructure failures potentially unrelated to your change:

  • 1 instance of "Unknown failure"
  • 1 instance of "Package install failed"

Otherwise...

Something interesting changed - please have a look.

Details

rxjs-src

/mnt/ts_downloads/rxjs-src/build.sh

  • [NEW] error TS2428: All declarations of 'WeakMap' must have identical type parameters.
    • /home/vsts/work/1/s/typescript-55222/lib/lib.es2015.collection.d.ts(63,11)
    • /home/vsts/work/1/s/typescript-55222/lib/lib.es2015.iterable.d.ts(162,11)
    • /home/vsts/work/1/s/typescript-55222/lib/lib.es2015.symbol.wellknown.d.ts(140,11)
    • /home/vsts/work/1/s/typescript-55222/lib/lib.es2015.collection.d.ts(63,11)
    • /home/vsts/work/1/s/typescript-55222/lib/lib.es2015.iterable.d.ts(162,11)
    • /home/vsts/work/1/s/typescript-55222/lib/lib.es2015.symbol.wellknown.d.ts(140,11)
    • /home/vsts/work/1/s/typescript-55222/lib/lib.es2015.collection.d.ts(63,11)
    • /home/vsts/work/1/s/typescript-55222/lib/lib.es2015.iterable.d.ts(162,11)
    • /home/vsts/work/1/s/typescript-55222/lib/lib.es2015.symbol.wellknown.d.ts(140,11)
    • /home/vsts/work/1/s/typescript-55222/lib/lib.es2015.collection.d.ts(63,11)
    • /home/vsts/work/1/s/typescript-55222/lib/lib.es2015.iterable.d.ts(162,11)
    • /home/vsts/work/1/s/typescript-55222/lib/lib.es2015.symbol.wellknown.d.ts(140,11)
    • /home/vsts/work/1/s/typescript-55222/lib/lib.es2015.collection.d.ts(63,11)
    • /home/vsts/work/1/s/typescript-55222/lib/lib.es2015.iterable.d.ts(162,11)
    • /home/vsts/work/1/s/typescript-55222/lib/lib.es2015.symbol.wellknown.d.ts(140,11)
  • [MISSING] error TS2428: All declarations of 'WeakMap' must have identical type parameters.
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.collection.d.ts(63,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.iterable.d.ts(162,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.symbol.wellknown.d.ts(140,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.collection.d.ts(63,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.iterable.d.ts(162,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.symbol.wellknown.d.ts(140,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.collection.d.ts(63,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.iterable.d.ts(162,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.symbol.wellknown.d.ts(140,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.collection.d.ts(63,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.iterable.d.ts(162,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.symbol.wellknown.d.ts(140,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.collection.d.ts(63,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.iterable.d.ts(162,11)
    • /home/vsts/work/1/s/typescript-main/lib/lib.es2015.symbol.wellknown.d.ts(140,11)

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 1, 2023

@jakebailey
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - main..55222
Metric main 55222 Delta Best Worst p-value
Angular - node (v18.10.0, x64)
Memory used 368,785k (± 0.01%) 368,794k (± 0.01%) ~ 368,756k 368,821k p=0.689 n=6
Parse Time 3.40s (± 0.55%) 3.40s (± 0.42%) ~ 3.38s 3.42s p=0.685 n=6
Bind Time 1.12s (± 0.92%) 1.12s (± 0.73%) ~ 1.11s 1.13s p=0.932 n=6
Check Time 8.95s (± 0.34%) 9.00s (± 0.44%) +0.05s (+ 0.60%) 8.93s 9.04s p=0.045 n=6
Emit Time 7.57s (± 1.03%) 7.58s (± 0.76%) ~ 7.50s 7.65s p=1.000 n=6
Total Time 21.05s (± 0.38%) 21.10s (± 0.49%) ~ 20.95s 21.22s p=0.421 n=6
Compiler-Unions - node (v18.10.0, x64)
Memory used 193,051k (± 1.18%) 194,922k (± 1.56%) ~ 192,120k 197,736k p=0.128 n=6
Parse Time 1.51s (± 0.65%) 1.50s (± 0.98%) ~ 1.48s 1.52s p=0.247 n=6
Bind Time 0.78s (± 0.66%) 0.77s (± 0.67%) ~ 0.77s 0.78s p=0.311 n=6
Check Time 9.47s (± 0.42%) 9.46s (± 0.30%) ~ 9.42s 9.50s p=1.000 n=6
Emit Time 2.76s (± 0.38%) 2.72s (± 1.02%) ~ 2.70s 2.77s p=0.052 n=6
Total Time 14.51s (± 0.35%) 14.46s (± 0.18%) ~ 14.42s 14.48s p=0.053 n=6
Monaco - node (v18.10.0, x64)
Memory used 347,755k (± 0.01%) 347,772k (± 0.01%) ~ 347,741k 347,813k p=0.471 n=6
Parse Time 2.62s (± 1.05%) 2.63s (± 1.19%) ~ 2.58s 2.66s p=0.418 n=6
Bind Time 1.01s (± 0.74%) 1.01s (± 0.51%) ~ 1.01s 1.02s p=0.784 n=6
Check Time 7.29s (± 0.38%) 7.29s (± 0.64%) ~ 7.21s 7.35s p=0.936 n=6
Emit Time 4.24s (± 0.90%) 4.27s (± 0.88%) ~ 4.22s 4.33s p=0.260 n=6
Total Time 15.16s (± 0.51%) 15.21s (± 0.65%) ~ 15.03s 15.31s p=0.261 n=6
TFS - node (v18.10.0, x64)
Memory used 301,794k (± 0.01%) 301,786k (± 0.01%) ~ 301,751k 301,818k p=0.574 n=6
Parse Time 2.08s (± 1.23%) 2.07s (± 1.77%) ~ 2.04s 2.13s p=0.871 n=6
Bind Time 1.13s (± 1.03%) 1.12s (± 0.36%) -0.01s (- 1.18%) 1.11s 1.12s p=0.025 n=6
Check Time 6.67s (± 0.53%) 6.66s (± 0.50%) ~ 6.61s 6.70s p=0.518 n=6
Emit Time 3.86s (± 0.95%) 3.89s (± 0.48%) ~ 3.86s 3.91s p=0.259 n=6
Total Time 13.75s (± 0.50%) 13.74s (± 0.44%) ~ 13.65s 13.84s p=1.000 n=6
material-ui - node (v18.10.0, x64)
Memory used 482,550k (± 0.01%) 482,585k (± 0.02%) ~ 482,533k 482,774k p=0.230 n=6
Parse Time 3.11s (± 1.18%) 3.09s (± 2.29%) ~ 2.95s 3.14s p=0.933 n=6
Bind Time 0.92s (± 2.44%) 0.94s (± 5.00%) ~ 0.91s 1.03s p=1.000 n=6
Check Time 17.32s (± 0.73%) 17.33s (± 0.78%) ~ 17.09s 17.49s p=0.470 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 21.36s (± 0.54%) 21.37s (± 0.63%) ~ 21.14s 21.55s p=0.334 n=6
xstate - node (v18.10.0, x64)
Memory used 563,842k (± 0.02%) 565,624k (± 0.01%) +1,782k (+ 0.32%) 565,514k 565,730k p=0.005 n=6
Parse Time 3.85s (± 0.61%) 3.85s (± 0.35%) ~ 3.83s 3.87s p=0.806 n=6
Bind Time 1.64s (± 1.24%) 1.64s (± 0.63%) ~ 1.62s 1.65s p=0.935 n=6
Check Time 2.81s (± 1.00%) 3.03s (± 0.61%) +0.22s (+ 7.64%) 3.01s 3.05s p=0.005 n=6
Emit Time 0.08s (± 0.00%) 0.08s (± 6.44%) ~ 0.08s 0.09s p=0.071 n=6
Total Time 8.39s (± 0.27%) 8.61s (± 0.24%) +0.22s (+ 2.64%) 8.58s 8.64s p=0.005 n=6
Angular - node (v16.17.1, x64)
Memory used 368,218k (± 0.00%) 368,222k (± 0.01%) ~ 368,193k 368,246k p=0.575 n=6
Parse Time 3.57s (± 0.39%) 3.55s (± 0.18%) ~ 3.54s 3.56s p=0.059 n=6
Bind Time 1.19s (± 0.63%) 1.19s (± 0.43%) ~ 1.18s 1.19s p=0.784 n=6
Check Time 9.76s (± 0.72%) 9.77s (± 0.54%) ~ 9.68s 9.83s p=1.000 n=6
Emit Time 8.04s (± 0.80%) 8.01s (± 0.39%) ~ 7.96s 8.04s p=0.228 n=6
Total Time 22.55s (± 0.58%) 22.51s (± 0.31%) ~ 22.37s 22.56s p=0.374 n=6
Compiler-Unions - node (v16.17.1, x64)
Memory used 193,823k (± 0.02%) 194,372k (± 0.67%) ~ 193,796k 197,022k p=0.261 n=6
Parse Time 1.59s (± 0.96%) 1.59s (± 0.62%) ~ 1.58s 1.61s p=0.557 n=6
Bind Time 0.82s (± 0.92%) 0.82s (± 0.50%) ~ 0.82s 0.83s p=1.000 n=6
Check Time 10.16s (± 0.48%) 10.17s (± 0.58%) ~ 10.09s 10.25s p=0.810 n=6
Emit Time 2.98s (± 0.87%) 3.01s (± 0.91%) ~ 2.97s 3.05s p=0.106 n=6
Total Time 15.56s (± 0.35%) 15.60s (± 0.24%) ~ 15.55s 15.64s p=0.149 n=6
Monaco - node (v16.17.1, x64)
Memory used 347,078k (± 0.01%) 347,085k (± 0.01%) ~ 347,028k 347,116k p=0.630 n=6
Parse Time 2.77s (± 0.44%) 2.76s (± 0.37%) ~ 2.75s 2.78s p=0.675 n=6
Bind Time 1.08s (± 0.48%) 1.08s (± 0.70%) ~ 1.07s 1.09s p=0.784 n=6
Check Time 8.04s (± 0.61%) 8.06s (± 0.33%) ~ 8.03s 8.10s p=0.295 n=6
Emit Time 4.48s (± 0.56%) 4.47s (± 0.60%) ~ 4.44s 4.51s p=0.227 n=6
Total Time 16.36s (± 0.36%) 16.37s (± 0.38%) ~ 16.30s 16.48s p=1.000 n=6
TFS - node (v16.17.1, x64)
Memory used 301,118k (± 0.01%) 301,109k (± 0.01%) ~ 301,082k 301,135k p=0.377 n=6
Parse Time 2.20s (± 0.38%) 2.19s (± 0.24%) ~ 2.18s 2.19s p=0.073 n=6
Bind Time 1.22s (± 1.70%) 1.21s (± 1.60%) ~ 1.18s 1.23s p=0.625 n=6
Check Time 7.34s (± 0.31%) 7.33s (± 0.71%) ~ 7.24s 7.38s p=1.000 n=6
Emit Time 4.30s (± 1.17%) 4.33s (± 0.97%) ~ 4.29s 4.39s p=0.146 n=6
Total Time 15.06s (± 0.25%) 15.06s (± 0.57%) ~ 14.95s 15.17s p=0.748 n=6
material-ui - node (v16.17.1, x64)
Memory used 481,882k (± 0.01%) 481,860k (± 0.01%) ~ 481,811k 481,954k p=0.297 n=6
Parse Time 3.26s (± 0.72%) 3.23s (± 0.26%) ~ 3.23s 3.25s p=0.122 n=6
Bind Time 0.96s (± 0.85%) 0.95s (± 0.57%) ~ 0.95s 0.96s p=0.859 n=6
Check Time 18.34s (± 0.39%) 18.34s (± 0.74%) ~ 18.21s 18.57s p=0.687 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.55s (± 0.36%) 22.53s (± 0.63%) ~ 22.41s 22.78s p=0.422 n=6
xstate - node (v16.17.1, x64)
Memory used 561,343k (± 0.01%) 563,061k (± 0.01%) +1,718k (+ 0.31%) 562,967k 563,164k p=0.005 n=6
Parse Time 4.02s (± 0.30%) 4.02s (± 0.43%) ~ 4.00s 4.05s p=0.871 n=6
Bind Time 1.66s (± 5.88%) 1.77s (± 1.62%) +0.11s (+ 6.82%) 1.73s 1.81s p=0.024 n=6
Check Time 3.09s (± 1.60%) 3.28s (± 1.26%) +0.19s (+ 6.09%) 3.23s 3.34s p=0.005 n=6
Emit Time 0.09s (± 5.53%) 0.09s (± 0.00%) ~ 0.09s 0.09s p=0.174 n=6
Total Time 8.87s (± 0.58%) 9.17s (± 0.82%) +0.29s (+ 3.33%) 9.06s 9.28s p=0.005 n=6
Angular - node (v14.21.3, x64)
Memory used 362,126k (± 0.01%) 362,093k (± 0.01%) ~ 362,054k 362,155k p=0.128 n=6
Parse Time 3.71s (± 0.14%) 3.73s (± 0.37%) +0.02s (+ 0.54%) 3.71s 3.74s p=0.020 n=6
Bind Time 1.22s (± 0.00%) 1.22s (± 0.62%) ~ 1.21s 1.23s p=0.598 n=6
Check Time 10.13s (± 0.37%) 10.15s (± 0.56%) ~ 10.06s 10.22s p=0.377 n=6
Emit Time 8.35s (± 0.56%) 8.36s (± 0.55%) ~ 8.30s 8.41s p=0.748 n=6
Total Time 23.41s (± 0.27%) 23.46s (± 0.32%) ~ 23.33s 23.53s p=0.336 n=6
Compiler-Unions - node (v14.21.3, x64)
Memory used 189,118k (± 0.01%) 189,094k (± 0.01%) ~ 189,060k 189,128k p=0.092 n=6
Parse Time 1.62s (± 0.72%) 1.61s (± 0.32%) ~ 1.61s 1.62s p=0.191 n=6
Bind Time 0.85s (± 0.61%) 0.85s (± 0.61%) ~ 0.84s 0.85s p=1.000 n=6
Check Time 10.32s (± 0.70%) 10.33s (± 0.44%) ~ 10.28s 10.39s p=0.574 n=6
Emit Time 3.11s (± 0.55%) 3.12s (± 0.33%) ~ 3.10s 3.13s p=0.741 n=6
Total Time 15.90s (± 0.49%) 15.91s (± 0.21%) ~ 15.86s 15.95s p=0.520 n=6
Monaco - node (v14.21.3, x64)
Memory used 342,072k (± 0.01%) 342,077k (± 0.01%) ~ 342,045k 342,108k p=0.936 n=6
Parse Time 2.80s (± 0.87%) 2.80s (± 0.76%) ~ 2.78s 2.84s p=0.684 n=6
Bind Time 1.09s (± 0.50%) 1.09s (± 0.37%) ~ 1.09s 1.10s p=0.282 n=6
Check Time 8.30s (± 0.26%) 8.31s (± 0.43%) ~ 8.26s 8.36s p=0.687 n=6
Emit Time 4.67s (± 0.87%) 4.66s (± 0.57%) ~ 4.63s 4.69s p=0.570 n=6
Total Time 16.86s (± 0.40%) 16.87s (± 0.23%) ~ 16.81s 16.91s p=0.809 n=6
TFS - node (v14.21.3, x64)
Memory used 296,199k (± 0.01%) 296,199k (± 0.01%) ~ 296,181k 296,222k p=0.936 n=6
Parse Time 2.42s (± 0.43%) 2.42s (± 0.80%) ~ 2.39s 2.44s p=1.000 n=6
Bind Time 1.08s (± 0.91%) 1.08s (± 0.59%) ~ 1.07s 1.09s p=0.733 n=6
Check Time 7.66s (± 0.42%) 7.67s (± 0.48%) ~ 7.61s 7.70s p=0.413 n=6
Emit Time 4.27s (± 0.69%) 4.28s (± 0.84%) ~ 4.23s 4.32s p=0.871 n=6
Total Time 15.44s (± 0.22%) 15.46s (± 0.40%) ~ 15.38s 15.54s p=0.629 n=6
material-ui - node (v14.21.3, x64)
Memory used 477,350k (± 0.00%) 477,350k (± 0.00%) ~ 477,319k 477,365k p=0.748 n=6
Parse Time 3.31s (± 0.31%) 3.29s (± 0.31%) ~ 3.28s 3.31s p=0.067 n=6
Bind Time 0.99s (± 0.41%) 0.99s (± 0.76%) ~ 0.98s 1.00s p=0.389 n=6
Check Time 19.15s (± 0.40%) 19.21s (± 0.56%) ~ 19.08s 19.34s p=0.471 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 23.45s (± 0.30%) 23.49s (± 0.48%) ~ 23.37s 23.64s p=0.630 n=6
xstate - node (v14.21.3, x64)
Memory used 550,168k (± 0.00%) 551,869k (± 0.01%) +1,701k (+ 0.31%) 551,833k 551,905k p=0.005 n=6
Parse Time 4.22s (± 0.61%) 4.20s (± 0.51%) ~ 4.18s 4.24s p=0.258 n=6
Bind Time 1.69s (± 0.30%) 1.68s (± 1.74%) ~ 1.62s 1.70s p=0.242 n=6
Check Time 3.13s (± 0.48%) 3.33s (± 0.84%) +0.21s (+ 6.56%) 3.29s 3.37s p=0.005 n=6
Emit Time 0.10s (± 5.76%) 0.09s (± 0.00%) ~ 0.09s 0.09s p=0.071 n=6
Total Time 9.13s (± 0.35%) 9.31s (± 0.62%) +0.18s (+ 1.97%) 9.20s 9.36s p=0.005 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-148-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.21.3, x64)
Scenarios
  • Angular - node (v18.10.0, x64)
  • Angular - node (v16.17.1, x64)
  • Angular - node (v14.21.3, x64)
  • Compiler-Unions - node (v18.10.0, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Compiler-Unions - node (v14.21.3, x64)
  • Monaco - node (v18.10.0, x64)
  • Monaco - node (v16.17.1, x64)
  • Monaco - node (v14.21.3, x64)
  • TFS - node (v18.10.0, x64)
  • TFS - node (v16.17.1, x64)
  • TFS - node (v14.21.3, x64)
  • material-ui - node (v18.10.0, x64)
  • material-ui - node (v16.17.1, x64)
  • material-ui - node (v14.21.3, x64)
  • xstate - node (v18.10.0, x64)
  • xstate - node (v16.17.1, x64)
  • xstate - node (v14.21.3, x64)
Benchmark Name Iterations
Current 55222 6
Baseline main 6

TSServer

Comparison Report - main..55222
Metric main 55222 Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 2,525ms (± 1.15%) 2,537ms (± 0.76%) ~ 2,511ms 2,559ms p=0.630 n=6
Req 2 - geterr 5,384ms (± 0.47%) 5,403ms (± 0.56%) ~ 5,365ms 5,435ms p=0.336 n=6
Req 3 - references 342ms (± 0.86%) 340ms (± 1.19%) ~ 335ms 345ms p=0.226 n=6
Req 4 - navto 289ms (± 0.34%) 291ms (± 0.95%) ~ 287ms 295ms p=0.141 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 76ms (± 1.08%) 76ms (± 3.99%) ~ 74ms 82ms p=0.451 n=6
CompilerTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 2,618ms (± 0.49%) 2,636ms (± 1.01%) ~ 2,604ms 2,670ms p=0.336 n=6
Req 2 - geterr 4,136ms (± 0.41%) 4,153ms (± 0.50%) ~ 4,118ms 4,177ms p=0.092 n=6
Req 3 - references 350ms (± 1.01%) 351ms (± 0.36%) ~ 349ms 353ms p=0.141 n=6
Req 4 - navto 289ms (± 0.36%) 289ms (± 0.48%) ~ 286ms 290ms p=0.867 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 79ms (± 6.58%) 78ms (± 1.32%) ~ 77ms 80ms p=1.000 n=6
xstateTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 3,088ms (± 0.53%) 3,081ms (± 0.52%) ~ 3,066ms 3,107ms p=0.520 n=6
Req 2 - geterr 1,602ms (± 0.57%) 1,598ms (± 0.86%) ~ 1,579ms 1,614ms p=0.748 n=6
Req 3 - references 114ms (± 1.32%) 115ms (± 2.01%) ~ 112ms 118ms p=0.217 n=6
Req 4 - navto 369ms (± 0.37%) 368ms (± 0.34%) ~ 366ms 369ms p=0.096 n=6
Req 5 - completionInfo count 2,872 (± 0.00%) 2,872 (± 0.00%) ~ 2,872 2,872 p=1.000 n=6
Req 5 - completionInfo 383ms (± 0.98%) 375ms (± 1.01%) -8ms (- 1.96%) 372ms 380ms p=0.013 n=6
Compiler-UnionsTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,629ms (± 0.40%) 2,615ms (± 0.80%) ~ 2,591ms 2,645ms p=0.229 n=6
Req 2 - geterr 6,081ms (± 0.38%) 6,084ms (± 0.67%) ~ 6,004ms 6,116ms p=0.521 n=6
Req 3 - references 355ms (± 0.37%) 356ms (± 0.66%) ~ 353ms 360ms p=0.217 n=6
Req 4 - navto 284ms (± 0.41%) 284ms (± 0.84%) ~ 281ms 288ms p=0.934 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 83ms (± 7.49%) 81ms (± 1.22%) ~ 80ms 82ms p=0.673 n=6
CompilerTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,803ms (± 0.52%) 2,798ms (± 0.89%) ~ 2,760ms 2,824ms p=0.936 n=6
Req 2 - geterr 4,689ms (± 0.44%) 4,699ms (± 0.36%) ~ 4,673ms 4,723ms p=0.470 n=6
Req 3 - references 364ms (± 0.23%) 363ms (± 0.58%) ~ 361ms 366ms p=0.505 n=6
Req 4 - navto 281ms (± 0.95%) 281ms (± 0.84%) ~ 278ms 284ms p=1.000 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 77ms (± 1.09%) 76ms (± 1.75%) ~ 75ms 78ms p=0.342 n=6
xstateTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 3,230ms (± 0.30%) 3,221ms (± 0.24%) ~ 3,211ms 3,229ms p=0.261 n=6
Req 2 - geterr 1,733ms (± 1.20%) 1,757ms (± 1.34%) ~ 1,728ms 1,788ms p=0.093 n=6
Req 3 - references 130ms (± 7.76%) 126ms (± 6.65%) ~ 121ms 143ms p=0.216 n=6
Req 4 - navto 352ms (± 0.49%) 351ms (± 0.28%) ~ 350ms 352ms p=0.510 n=6
Req 5 - completionInfo count 2,872 (± 0.00%) 2,872 (± 0.00%) ~ 2,872 2,872 p=1.000 n=6
Req 5 - completionInfo 429ms (± 0.71%) 423ms (± 1.47%) ~ 414ms 431ms p=0.092 n=6
Compiler-UnionsTSServer - node (v14.21.3, x64)
Req 1 - updateOpen 2,771ms (± 0.36%) 2,767ms (± 0.33%) ~ 2,750ms 2,774ms p=0.688 n=6
Req 2 - geterr 6,220ms (± 0.51%) 6,188ms (± 0.65%) ~ 6,122ms 6,247ms p=0.298 n=6
Req 3 - references 361ms (± 0.51%) 364ms (± 0.51%) ~ 362ms 367ms p=0.061 n=6
Req 4 - navto 291ms (± 0.92%) 291ms (± 0.78%) ~ 289ms 295ms p=1.000 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 102ms (± 8.45%) 102ms (± 9.41%) ~ 91ms 111ms p=1.000 n=6
CompilerTSServer - node (v14.21.3, x64)
Req 1 - updateOpen 2,936ms (± 0.28%) 2,931ms (± 0.49%) ~ 2,906ms 2,943ms p=0.520 n=6
Req 2 - geterr 4,583ms (± 0.44%) 4,573ms (± 0.48%) ~ 4,535ms 4,594ms p=0.575 n=6
Req 3 - references 370ms (± 0.76%) 369ms (± 0.68%) ~ 366ms 372ms p=0.629 n=6
Req 4 - navto 299ms (± 0.46%) 298ms (± 0.25%) ~ 297ms 299ms p=0.111 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 86ms (± 8.01%) 87ms (± 7.63%) ~ 82ms 100ms p=0.366 n=6
xstateTSServer - node (v14.21.3, x64)
Req 1 - updateOpen 3,498ms (± 1.79%) 3,514ms (± 0.78%) ~ 3,480ms 3,538ms p=1.000 n=6
Req 2 - geterr 1,863ms (± 0.80%) 1,899ms (± 0.82%) +36ms (+ 1.91%) 1,876ms 1,919ms p=0.008 n=6
Req 3 - references 139ms (± 8.58%) 135ms (± 6.70%) ~ 130ms 153ms p=0.623 n=6
Req 4 - navto 391ms (± 0.77%) 386ms (± 0.16%) -5ms (- 1.15%) 385ms 387ms p=0.005 n=6
Req 5 - completionInfo count 2,872 (± 0.00%) 2,872 (± 0.00%) ~ 2,872 2,872 p=1.000 n=6
Req 5 - completionInfo 420ms (± 1.35%) 420ms (± 0.77%) ~ 415ms 424ms p=1.000 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-148-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.21.3, x64)
Scenarios
  • Compiler-UnionsTSServer - node (v18.10.0, x64)
  • Compiler-UnionsTSServer - node (v16.17.1, x64)
  • Compiler-UnionsTSServer - node (v14.21.3, x64)
  • CompilerTSServer - node (v18.10.0, x64)
  • CompilerTSServer - node (v16.17.1, x64)
  • CompilerTSServer - node (v14.21.3, x64)
  • xstateTSServer - node (v18.10.0, x64)
  • xstateTSServer - node (v16.17.1, x64)
  • xstateTSServer - node (v14.21.3, x64)
Benchmark Name Iterations
Current 55222 6
Baseline main 6

Startup

Comparison Report - main..55222
Metric main 55222 Delta Best Worst p-value
tsc-startup - node (v16.17.1, x64)
Execution time 143.04ms (± 0.25%) 142.52ms (± 0.18%) -0.52ms (- 0.37%) 141.86ms 145.01ms p=0.000 n=600
tsserver-startup - node (v16.17.1, x64)
Execution time 222.23ms (± 0.17%) 221.80ms (± 0.18%) -0.43ms (- 0.19%) 220.94ms 229.73ms p=0.000 n=600
tsserverlibrary-startup - node (v16.17.1, x64)
Execution time 223.71ms (± 0.17%) 223.26ms (± 0.17%) -0.45ms (- 0.20%) 222.31ms 228.31ms p=0.000 n=600
typescript-startup - node (v16.17.1, x64)
Execution time 206.38ms (± 0.24%) 205.27ms (± 0.16%) -1.10ms (- 0.53%) 204.37ms 209.04ms p=0.000 n=600
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-148-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • tsc-startup - node (v16.17.1, x64)
  • tsserver-startup - node (v16.17.1, x64)
  • tsserverlibrary-startup - node (v16.17.1, x64)
  • typescript-startup - node (v16.17.1, x64)
Benchmark Name Iterations
Current 55222 6
Baseline main 6

Developer Information:

Download Benchmark

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 1, 2023

@jakebailey Here are the results of running the top-repos suite comparing main and refs/pull/55222/merge:

Everything looks good!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 1, 2023

Hey @jakebailey, it looks like the DT test run failed. Please check the log for more details.
You can check the log here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 1, 2023

@jakebailey Here are the results of running the top-repos suite comparing main and refs/pull/55222/merge:

Everything looks good!

@jakebailey
Copy link
Member Author

jakebailey commented Aug 1, 2023

7% check degradation in xstate, ouch

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 1, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the abridged perf test suite on this PR at 5902018. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 1, 2023

@jakebailey
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..55222

Metric main 55222 Delta Best Worst p-value
Angular - node (v16.17.1, x64)
Memory used 368,218k (± 0.00%) 368,212k (± 0.00%) ~ 368,196k 368,238k p=0.298 n=6
Parse Time 3.57s (± 0.39%) 3.58s (± 0.63%) ~ 3.54s 3.60s p=0.370 n=6
Bind Time 1.19s (± 0.63%) 1.19s (± 0.34%) ~ 1.19s 1.20s p=0.389 n=6
Check Time 9.76s (± 0.72%) 9.73s (± 0.32%) ~ 9.68s 9.77s p=0.294 n=6
Emit Time 8.04s (± 0.80%) 7.99s (± 0.62%) ~ 7.92s 8.04s p=0.107 n=6
Total Time 22.55s (± 0.58%) 22.48s (± 0.36%) ~ 22.40s 22.58s p=0.520 n=6
Compiler-Unions - node (v16.17.1, x64)
Memory used 193,823k (± 0.02%) 193,817k (± 0.01%) ~ 193,777k 193,851k p=0.936 n=6
Parse Time 1.59s (± 0.96%) 1.59s (± 0.86%) ~ 1.56s 1.60s p=0.867 n=6
Bind Time 0.82s (± 0.92%) 0.82s (± 0.99%) ~ 0.82s 0.84s p=1.000 n=6
Check Time 10.16s (± 0.48%) 10.14s (± 0.76%) ~ 10.04s 10.23s p=0.628 n=6
Emit Time 2.98s (± 0.87%) 2.97s (± 0.45%) ~ 2.95s 2.98s p=0.565 n=6
Total Time 15.56s (± 0.35%) 15.52s (± 0.54%) ~ 15.39s 15.60s p=0.423 n=6
Monaco - node (v16.17.1, x64)
Memory used 347,078k (± 0.01%) 347,080k (± 0.01%) ~ 347,050k 347,102k p=0.688 n=6
Parse Time 2.77s (± 0.44%) 2.77s (± 0.63%) ~ 2.75s 2.79s p=0.369 n=6
Bind Time 1.08s (± 0.48%) 1.08s (± 0.59%) ~ 1.07s 1.09s p=0.386 n=6
Check Time 8.04s (± 0.61%) 8.01s (± 0.37%) ~ 7.96s 8.05s p=0.291 n=6
Emit Time 4.48s (± 0.56%) 4.45s (± 0.59%) ~ 4.42s 4.49s p=0.063 n=6
Total Time 16.36s (± 0.36%) 16.31s (± 0.38%) ~ 16.25s 16.42s p=0.128 n=6
TFS - node (v16.17.1, x64)
Memory used 301,118k (± 0.01%) 301,102k (± 0.01%) ~ 301,080k 301,137k p=0.199 n=6
Parse Time 2.20s (± 0.38%) 2.21s (± 0.62%) +0.02s (+ 0.84%) 2.20s 2.23s p=0.021 n=6
Bind Time 1.22s (± 1.70%) 1.22s (± 0.84%) ~ 1.21s 1.24s p=0.459 n=6
Check Time 7.34s (± 0.31%) 7.33s (± 0.22%) ~ 7.31s 7.35s p=0.252 n=6
Emit Time 4.30s (± 1.17%) 4.29s (± 0.41%) ~ 4.27s 4.32s p=0.745 n=6
Total Time 15.06s (± 0.25%) 15.06s (± 0.30%) ~ 15.00s 15.11s p=0.871 n=6
material-ui - node (v16.17.1, x64)
Memory used 481,882k (± 0.01%) 481,875k (± 0.01%) ~ 481,830k 481,901k p=0.748 n=6
Parse Time 3.26s (± 0.72%) 3.27s (± 0.45%) ~ 3.25s 3.29s p=0.287 n=6
Bind Time 0.96s (± 0.85%) 0.95s (± 0.54%) ~ 0.95s 0.96s p=0.523 n=6
Check Time 18.34s (± 0.39%) 18.35s (± 0.65%) ~ 18.20s 18.52s p=0.936 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.55s (± 0.36%) 22.58s (± 0.51%) ~ 22.45s 22.75s p=0.809 n=6
xstate - node (v16.17.1, x64)
Memory used 561,343k (± 0.01%) 563,176k (± 0.04%) +1,833k (+ 0.33%) 562,997k 563,455k p=0.005 n=6
Parse Time 4.02s (± 0.30%) 4.02s (± 0.30%) ~ 4.01s 4.04s p=1.000 n=6
Bind Time 1.66s (± 5.88%) 1.72s (± 5.38%) ~ 1.54s 1.80s p=0.370 n=6
Check Time 3.09s (± 1.60%) 3.31s (± 2.99%) +0.22s (+ 7.06%) 3.25s 3.51s p=0.005 n=6
Emit Time 0.09s (± 5.53%) 0.09s (± 0.00%) ~ 0.09s 0.09s p=0.174 n=6
Total Time 8.87s (± 0.58%) 9.15s (± 0.21%) +0.28s (+ 3.16%) 9.12s 9.17s p=0.005 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-148-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • Angular - node (v16.17.1, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Monaco - node (v16.17.1, x64)
  • TFS - node (v16.17.1, x64)
  • material-ui - node (v16.17.1, x64)
  • xstate - node (v16.17.1, x64)
Benchmark Name Iterations
Current 55222 6
Baseline main 6

Developer Information:

Download Benchmark

@jakebailey
Copy link
Member Author

jakebailey commented Aug 1, 2023

This PR failed RWC but the diffs aren't correctly being pushed. Need to figure that out.

@jakebailey
Copy link
Member Author

jakebailey commented Aug 15, 2023

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 15, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the tsc-only perf test suite on this PR at 5902018. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 15, 2023

@jakebailey
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v16.17.1, x64)
Memory used 300,307k (± 0.01%) 300,336k (± 0.00%) +30k (+ 0.01%) 300,312k 300,349k p=0.020 n=6
Parse Time 3.02s (± 0.13%) 3.03s (± 0.13%) +0.01s (+ 0.22%) 3.02s 3.03s p=0.034 n=6
Bind Time 0.93s (± 0.00%) 0.93s (± 0.00%) ~ 0.93s 0.93s p=1.000 n=6
Check Time 9.46s (± 0.26%) 9.45s (± 0.20%) ~ 9.43s 9.47s p=0.188 n=6
Emit Time 7.62s (± 0.29%) 7.63s (± 0.41%) ~ 7.59s 7.68s p=1.000 n=6
Total Time 21.04s (± 0.12%) 21.03s (± 0.15%) ~ 21.00s 21.07s p=0.935 n=6
Compiler-Unions - node (v16.17.1, x64)
Memory used 193,822k (± 0.01%) 193,830k (± 0.01%) ~ 193,797k 193,859k p=0.810 n=6
Parse Time 1.57s (± 1.05%) 1.56s (± 1.05%) ~ 1.55s 1.59s p=0.441 n=6
Bind Time 0.79s (± 0.00%) 0.79s (± 0.00%) ~ 0.79s 0.79s p=1.000 n=6
Check Time 9.90s (± 0.21%) 9.86s (± 0.17%) -0.04s (- 0.37%) 9.84s 9.88s p=0.012 n=6
Emit Time 2.75s (± 0.19%) 2.74s (± 0.38%) ~ 2.73s 2.76s p=0.437 n=6
Total Time 15.01s (± 0.21%) 14.95s (± 0.18%) -0.06s (- 0.38%) 14.91s 14.99s p=0.013 n=6
Monaco - node (v16.17.1, x64)
Memory used 347,109k (± 0.01%) 347,102k (± 0.01%) ~ 347,081k 347,129k p=0.575 n=6
Parse Time 2.68s (± 0.30%) 2.69s (± 0.39%) ~ 2.67s 2.70s p=0.865 n=6
Bind Time 0.99s (± 0.00%) 0.99s (± 0.00%) ~ 0.99s 0.99s p=1.000 n=6
Check Time 7.90s (± 0.38%) 7.91s (± 0.13%) ~ 7.89s 7.92s p=0.806 n=6
Emit Time 4.25s (± 0.28%) 4.26s (± 0.38%) ~ 4.23s 4.28s p=0.549 n=6
Total Time 15.83s (± 0.19%) 15.83s (± 0.17%) ~ 15.80s 15.88s p=0.570 n=6
TFS - node (v16.17.1, x64)
Memory used 301,119k (± 0.01%) 301,119k (± 0.00%) ~ 301,101k 301,135k p=1.000 n=6
Parse Time 2.18s (± 0.61%) 2.18s (± 0.56%) ~ 2.16s 2.19s p=0.863 n=6
Bind Time 1.11s (± 1.51%) 1.11s (± 0.57%) ~ 1.10s 1.12s p=0.720 n=6
Check Time 7.22s (± 0.40%) 7.22s (± 0.32%) ~ 7.19s 7.26s p=1.000 n=6
Emit Time 3.97s (± 0.23%) 3.97s (± 0.36%) ~ 3.95s 3.99s p=1.000 n=6
Total Time 14.48s (± 0.33%) 14.48s (± 0.13%) ~ 14.46s 14.51s p=0.687 n=6
material-ui - node (v16.17.1, x64)
Memory used 479,389k (± 0.01%) 479,417k (± 0.01%) ~ 479,335k 479,503k p=0.575 n=6
Parse Time 3.15s (± 0.24%) 3.15s (± 0.13%) ~ 3.15s 3.16s p=0.389 n=6
Bind Time 0.91s (± 0.45%) 0.91s (± 0.00%) ~ 0.91s 0.91s p=0.405 n=6
Check Time 17.90s (± 0.25%) 17.92s (± 0.33%) ~ 17.85s 18.00s p=0.872 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 21.96s (± 0.21%) 21.98s (± 0.25%) ~ 21.91s 22.05s p=0.748 n=6
xstate - node (v16.17.1, x64)
Memory used 542,764k (± 0.01%) 544,461k (± 0.01%) +1,696k (+ 0.31%) 544,400k 544,557k p=0.005 n=6
Parse Time 3.70s (± 0.17%) 3.70s (± 0.17%) ~ 3.69s 3.71s p=1.000 n=6
Bind Time 1.38s (± 4.67%) 1.40s (± 4.45%) ~ 1.34s 1.46s p=1.000 n=6
Check Time 3.30s (± 2.68%) 3.41s (± 2.21%) ~ 3.33s 3.49s p=0.128 n=6
Emit Time 0.08s (± 0.00%) 0.08s (± 6.19%) ~ 0.08s 0.09s p=0.174 n=6
Total Time 8.46s (± 0.38%) 8.58s (± 0.23%) +0.13s (+ 1.48%) 8.55s 8.60s p=0.005 n=6
System info unknown
Hosts
  • node (v16.17.1, x64)
Scenarios
  • Angular - node (v16.17.1, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Monaco - node (v16.17.1, x64)
  • TFS - node (v16.17.1, x64)
  • material-ui - node (v16.17.1, x64)
  • xstate - node (v16.17.1, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@jakebailey
Copy link
Member Author

jakebailey commented Jul 24, 2024

Surely if I keep retrying it, it will eventually not break perf, right?

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 24, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this faster ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 24, 2024

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 192,998k (± 0.78%) 194,107k (± 1.01%) ~ 192,286k 195,927k p=0.689 n=6
Parse Time 1.30s (± 0.90%) 1.30s (± 0.69%) ~ 1.29s 1.31s p=0.933 n=6
Bind Time 0.71s 0.71s ~ ~ ~ p=1.000 n=6
Check Time 9.56s (± 0.41%) 9.56s (± 0.50%) ~ 9.49s 9.62s p=0.936 n=6
Emit Time 2.73s (± 1.97%) 2.74s (± 0.63%) ~ 2.72s 2.76s p=0.565 n=6
Total Time 14.30s (± 0.50%) 14.31s (± 0.44%) ~ 14.22s 14.40s p=0.936 n=6
angular-1 - node (v18.15.0, x64)
Errors 7 7 ~ ~ ~ p=1.000 n=6
Symbols 945,532 945,679 +147 (+ 0.02%) ~ ~ p=0.001 n=6
Types 409,507 409,555 +48 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 1,221,125k (± 0.00%) 1,221,258k (± 0.00%) +133k (+ 0.01%) 1,221,189k 1,221,306k p=0.005 n=6
Parse Time 6.63s (± 0.37%) 6.63s (± 0.85%) ~ 6.55s 6.71s p=0.809 n=6
Bind Time 1.86s (± 0.66%) 1.86s (± 0.28%) ~ 1.86s 1.87s p=0.241 n=6
Check Time 31.10s (± 0.29%) 31.24s (± 0.52%) ~ 31.08s 31.50s p=0.092 n=6
Emit Time 14.99s (± 0.55%) 15.00s (± 0.60%) ~ 14.90s 15.11s p=1.000 n=6
Total Time 54.57s (± 0.30%) 54.74s (± 0.45%) ~ 54.47s 55.06s p=0.378 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,324,872 2,329,028 +4,156 (+ 0.18%) ~ ~ p=0.001 n=6
Types 949,556 951,073 +1,517 (+ 0.16%) ~ ~ p=0.001 n=6
Memory used 2,222,587k (± 0.00%) 2,225,049k (± 0.00%) +2,463k (+ 0.11%) 2,224,970k 2,225,130k p=0.005 n=6
Parse Time 6.63s (± 0.40%) 6.63s (± 0.31%) ~ 6.60s 6.65s p=0.685 n=6
Bind Time 2.32s (± 0.44%) 2.33s (± 0.52%) ~ 2.31s 2.34s p=0.675 n=6
Check Time 73.08s (± 0.45%) 72.96s (± 0.36%) ~ 72.62s 73.33s p=0.689 n=6
Emit Time 0.14s (± 3.77%) 0.14s (± 6.19%) ~ 0.13s 0.15s p=0.533 n=6
Total Time 82.17s (± 0.42%) 82.05s (± 0.32%) ~ 81.70s 82.40s p=0.630 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,120 1,230,140 +20 (+ 0.00%) ~ ~ p=0.001 n=6
Types 265,641 265,674 +33 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 2,347,505k (± 0.01%) 2,466,094k (±11.78%) ~ 2,346,788k 3,059,873k p=0.810 n=6
Parse Time 5.00s (± 0.58%) 5.00s (± 0.85%) ~ 4.93s 5.04s p=0.520 n=6
Bind Time 1.89s (± 0.64%) 1.90s (± 1.28%) ~ 1.85s 1.92s p=0.405 n=6
Check Time 34.73s (± 0.52%) 34.75s (± 0.22%) ~ 34.65s 34.86s p=0.936 n=6
Emit Time 3.28s (± 1.05%) 3.29s (± 1.05%) ~ 3.24s 3.33s p=0.872 n=6
Total Time 44.91s (± 0.46%) 44.96s (± 0.17%) ~ 44.86s 45.06s p=0.575 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,230,120 1,230,140 +20 (+ 0.00%) ~ ~ p=0.001 n=6
Types 265,641 265,674 +33 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 2,421,504k (± 0.01%) 2,421,649k (± 0.02%) ~ 2,421,151k 2,422,181k p=0.688 n=6
Parse Time 6.24s (± 0.51%) 6.27s (± 0.83%) ~ 6.22s 6.35s p=0.297 n=6
Bind Time 2.05s (± 1.29%) 2.05s (± 0.85%) ~ 2.02s 2.07s p=1.000 n=6
Check Time 41.33s (± 0.48%) 41.43s (± 0.14%) ~ 41.34s 41.52s p=0.066 n=6
Emit Time 4.06s (± 1.64%) 4.06s (± 0.77%) ~ 4.03s 4.11s p=0.686 n=6
Total Time 53.69s (± 0.51%) 53.83s (± 0.18%) ~ 53.74s 53.99s p=0.065 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,471 258,485 +14 (+ 0.01%) ~ ~ p=0.001 n=6
Types 105,534 105,561 +27 (+ 0.03%) ~ ~ p=0.001 n=6
Memory used 429,163k (± 0.05%) 429,270k (± 0.03%) ~ 429,106k 429,451k p=0.173 n=6
Parse Time 3.34s (± 0.77%) 3.34s (± 0.23%) ~ 3.33s 3.35s p=1.000 n=6
Bind Time 1.32s (± 1.64%) 1.31s (± 1.25%) ~ 1.30s 1.33s p=0.675 n=6
Check Time 17.96s (± 0.30%) 18.02s (± 0.32%) ~ 17.93s 18.10s p=0.108 n=6
Emit Time 1.63s (± 0.72%) 1.63s (± 1.21%) ~ 1.60s 1.66s p=0.605 n=6
Total Time 24.24s (± 0.27%) 24.29s (± 0.25%) ~ 24.23s 24.40s p=0.372 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,931 224,931 ~ ~ ~ p=1.000 n=6
Types 94,146 94,146 ~ ~ ~ p=1.000 n=6
Memory used 369,982k (± 0.02%) 370,106k (± 0.05%) ~ 369,888k 370,327k p=0.298 n=6
Parse Time 2.76s (± 1.13%) 2.77s (± 0.35%) ~ 2.76s 2.78s p=0.461 n=6
Bind Time 1.59s (± 1.43%) 1.59s (± 1.61%) ~ 1.56s 1.62s p=0.934 n=6
Check Time 15.65s (± 0.39%) 15.61s (± 0.45%) ~ 15.54s 15.73s p=0.378 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 20.00s (± 0.26%) 19.97s (± 0.42%) ~ 19.88s 20.10s p=0.423 n=6
vscode - node (v18.15.0, x64)
Errors 11 11 ~ ~ ~ p=1.000 n=6
Symbols 2,985,953 3,011,882 +25,929 (+ 0.87%) ~ ~ p=0.001 n=6
Types 1,027,137 1,030,528 +3,391 (+ 0.33%) ~ ~ p=0.001 n=6
Memory used 3,109,968k (± 0.00%) 3,120,841k (± 0.00%) +10,873k (+ 0.35%) 3,120,792k 3,120,914k p=0.005 n=6
Parse Time 17.12s (± 0.18%) 17.14s (± 0.17%) ~ 17.11s 17.19s p=0.293 n=6
Bind Time 5.21s (± 0.23%) 5.25s (± 1.82%) ~ 5.20s 5.45s p=0.438 n=6
Check Time 96.29s (± 0.38%) 96.30s (± 0.15%) ~ 96.09s 96.47s p=0.936 n=6
Emit Time 25.08s (± 0.47%) 25.09s (± 0.81%) ~ 24.90s 25.46s p=0.873 n=6
Total Time 143.71s (± 0.27%) 143.79s (± 0.23%) ~ 143.41s 144.25s p=0.748 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,560 267,560 ~ ~ ~ p=1.000 n=6
Types 109,076 109,076 ~ ~ ~ p=1.000 n=6
Memory used 412,464k (± 0.02%) 412,402k (± 0.01%) ~ 412,350k 412,479k p=0.128 n=6
Parse Time 3.83s (± 0.73%) 3.83s (± 0.65%) ~ 3.79s 3.86s p=0.520 n=6
Bind Time 1.71s (± 0.49%) 1.72s (± 0.30%) ~ 1.71s 1.72s p=0.533 n=6
Check Time 16.93s (± 0.66%) 16.89s (± 0.32%) ~ 16.79s 16.93s p=0.334 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.47s (± 0.54%) 22.44s (± 0.20%) ~ 22.37s 22.49s p=0.378 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 511,803 517,597 +5,794 (+ 1.13%) ~ ~ p=0.001 n=6
Types 162,088 163,963 +1,875 (+ 1.16%) ~ ~ p=0.001 n=6
Memory used 448,992k (± 0.09%) 452,469k (± 0.02%) +3,478k (+ 0.77%) 452,360k 452,659k p=0.005 n=6
Parse Time 3.90s (± 0.44%) 3.91s (± 0.56%) ~ 3.88s 3.94s p=0.408 n=6
Bind Time 1.44s (± 0.84%) 1.44s (± 0.68%) ~ 1.43s 1.46s p=0.226 n=6
Check Time 21.27s (± 0.29%) 21.15s (± 0.40%) -0.12s (- 0.56%) 21.02s 21.23s p=0.020 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 26.62s (± 0.29%) 26.51s (± 0.41%) ~ 26.34s 26.62s p=0.065 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@Andarist Andarist mentioned this pull request Apr 5, 2025
Open
@jakebailey
Copy link
Member Author

jakebailey commented Dec 4, 2025

@typescript-bot test it

@typescript-bot
Copy link
Collaborator

typescript-bot commented Dec 4, 2025 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
test top400 ✅ Started ✅ Results
user test this ✅ Started ✅ Results
run dt ✅ Started ✅ Results
perf test this faster ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

typescript-bot commented Dec 4, 2025

Hey @jakebailey, the results of running the DT tests are ready.

Everything looks the same!

You can check the log here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Dec 4, 2025

@jakebailey Here are the results of running the user tests with tsc comparing main and refs/pull/55222/merge:

There were infrastructure failures potentially unrelated to your change:

  • 1 instance of "Git clone failed"

Otherwise...

Everything looks good!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Dec 4, 2025

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 1 1 ~ ~ ~ p=1.000 n=6
Symbols 62,370 62,370 ~ ~ ~ p=1.000 n=6
Types 50,387 50,387 ~ ~ ~ p=1.000 n=6
Memory used 195,514k (± 0.78%) 195,619k (± 0.84%) ~ 192,286k 196,445k p=0.575 n=6
Parse Time 1.59s (± 2.43%) 1.60s (± 2.42%) ~ 1.53s 1.64s p=0.519 n=6
Bind Time 0.90s (± 1.63%) 0.90s (± 1.72%) ~ 0.88s 0.91s p=0.805 n=6
Check Time 11.92s (± 0.35%) 11.94s (± 0.66%) ~ 11.84s 12.04s p=0.630 n=6
Emit Time 3.46s (± 4.37%) 3.38s (± 4.08%) ~ 3.25s 3.59s p=0.575 n=6
Total Time 17.87s (± 0.92%) 17.82s (± 0.69%) ~ 17.70s 17.98s p=0.575 n=6
angular-1 - node (v18.15.0, x64)
Errors 2 2 ~ ~ ~ p=1.000 n=6
Symbols 956,048 956,195 +147 (+ 0.02%) ~ ~ p=0.001 n=6
Types 415,904 415,952 +48 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 1,255,165k (± 0.00%) 1,255,346k (± 0.01%) +181k (+ 0.01%) 1,255,202k 1,255,516k p=0.005 n=6
Parse Time 8.01s (± 0.63%) 7.97s (± 0.83%) ~ 7.91s 8.09s p=0.261 n=6
Bind Time 2.38s (± 0.95%) 2.36s (± 0.44%) ~ 2.35s 2.38s p=0.288 n=6
Check Time 38.79s (± 0.30%) 38.65s (± 0.26%) ~ 38.53s 38.82s p=0.066 n=6
Emit Time 17.92s (± 0.37%) 17.93s (± 0.30%) ~ 17.86s 18.01s p=0.809 n=6
Total Time 67.08s (± 0.25%) 66.91s (± 0.24%) ~ 66.74s 67.14s p=0.128 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,722,102 2,724,078 +1,976 (+ 0.07%) ~ ~ p=0.001 n=6
Types 937,520 938,711 +1,191 (+ 0.13%) ~ ~ p=0.001 n=6
Memory used 3,051,260k (± 0.00%) 3,052,716k (± 0.01%) +1,456k (+ 0.05%) 3,052,514k 3,052,913k p=0.005 n=6
Parse Time 10.45s (± 0.63%) 10.42s (± 0.76%) ~ 10.36s 10.55s p=0.628 n=6
Bind Time 2.81s (± 0.87%) 2.81s (± 0.95%) ~ 2.77s 2.84s p=0.935 n=6
Check Time 112.54s (± 2.02%) 112.73s (± 0.82%) ~ 111.61s 114.04s p=0.230 n=6
Emit Time 0.38s (± 2.69%) 0.39s (± 4.22%) ~ 0.36s 0.40s p=0.612 n=6
Total Time 126.18s (± 1.79%) 126.35s (± 0.71%) ~ 125.19s 127.62s p=0.173 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,251,654 1,251,674 +20 (+ 0.00%) ~ ~ p=0.001 n=6
Types 259,840 259,873 +33 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 2,881,523k (±13.06%) 3,125,265k (± 0.02%) 🔻+243,742k (+ 8.46%) 3,124,529k 3,126,106k p=0.045 n=6
Parse Time 6.60s (± 1.89%) 6.68s (± 0.89%) ~ 6.61s 6.78s p=0.378 n=6
Bind Time 2.24s (± 2.32%) 2.21s (± 0.64%) ~ 2.19s 2.23s p=0.521 n=6
Check Time 42.99s (± 0.19%) 43.10s (± 0.31%) +0.12s (+ 0.27%) 42.85s 43.22s p=0.045 n=6
Emit Time 3.54s (± 2.96%) 3.49s (± 2.46%) ~ 3.35s 3.57s p=0.378 n=6
Total Time 55.36s (± 0.31%) 55.47s (± 0.29%) ~ 55.24s 55.63s p=0.471 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,251,654 1,251,674 +20 (+ 0.00%) ~ ~ p=0.001 n=6
Types 259,840 259,873 +33 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 3,067,566k (± 9.63%) 3,067,444k (± 9.66%) ~ 2,462,263k 3,188,975k p=1.000 n=6
Parse Time 6.77s (± 1.70%) 6.76s (± 0.87%) ~ 6.66s 6.81s p=0.575 n=6
Bind Time 2.27s (± 1.11%) 2.27s (± 1.20%) ~ 2.24s 2.31s p=1.000 n=6
Check Time 43.16s (± 0.47%) 43.20s (± 0.31%) ~ 43.04s 43.38s p=0.810 n=6
Emit Time 3.50s (± 1.97%) 3.55s (± 2.03%) ~ 3.48s 3.68s p=0.378 n=6
Total Time 55.68s (± 0.59%) 55.78s (± 0.27%) ~ 55.60s 56.04s p=0.936 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 264,287 264,301 +14 (+ 0.01%) ~ ~ p=0.001 n=6
Types 104,051 104,078 +27 (+ 0.03%) ~ ~ p=0.001 n=6
Memory used 443,307k (± 0.02%) 443,789k (± 0.25%) ~ 443,239k 446,014k p=0.377 n=6
Parse Time 3.51s (± 0.86%) 3.51s (± 0.82%) ~ 3.47s 3.55s p=0.683 n=6
Bind Time 1.37s (± 1.03%) 1.38s (± 1.00%) ~ 1.36s 1.39s p=0.623 n=6
Check Time 19.14s (± 0.32%) 19.20s (± 0.56%) ~ 19.05s 19.35s p=0.227 n=6
Emit Time 1.55s (± 1.39%) 1.55s (± 0.78%) ~ 1.53s 1.56s p=0.739 n=6
Total Time 25.56s (± 0.27%) 25.64s (± 0.42%) ~ 25.47s 25.77s p=0.261 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 72 72 ~ ~ ~ p=1.000 n=6
Symbols 225,493 225,493 ~ ~ ~ p=1.000 n=6
Types 94,373 94,373 ~ ~ ~ p=1.000 n=6
Memory used 370,119k (± 0.01%) 370,238k (± 0.06%) ~ 370,053k 370,581k p=0.810 n=6
Parse Time 2.83s (± 1.58%) 2.82s (± 1.04%) ~ 2.79s 2.87s p=0.687 n=6
Bind Time 1.64s (± 1.00%) 1.64s (± 0.83%) ~ 1.62s 1.66s p=0.512 n=6
Check Time 16.64s (± 0.21%) 16.62s (± 0.48%) ~ 16.52s 16.72s p=0.520 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 21.11s (± 0.31%) 21.09s (± 0.44%) ~ 20.98s 21.21s p=0.575 n=6
vscode - node (v18.15.0, x64)
Errors 11 11 ~ ~ ~ p=1.000 n=6
Symbols 4,077,459 4,098,151 +20,692 (+ 0.51%) ~ ~ p=0.001 n=6
Types 1,284,570 1,287,348 +2,778 (+ 0.22%) ~ ~ p=0.001 n=6
Memory used 3,863,540k (± 0.00%) 3,871,966k (± 0.01%) +8,426k (+ 0.22%) 3,871,670k 3,872,304k p=0.005 n=6
Parse Time 15.69s (± 0.23%) 15.74s (± 0.33%) ~ 15.64s 15.78s p=0.075 n=6
Bind Time 5.29s (± 0.71%) 5.29s (± 0.51%) ~ 5.25s 5.33s p=1.000 n=6
Check Time 113.21s (± 3.03%) 113.96s (± 2.48%) ~ 109.99s 117.61s p=0.689 n=6
Emit Time 40.16s (± 7.13%) 44.29s (±13.92%) ~ 38.89s 52.76s p=0.199 n=6
Total Time 174.35s (± 2.34%) 179.28s (± 3.24%) ~ 172.64s 186.98s p=0.230 n=6
webpack - node (v18.15.0, x64)
Errors 40 40 ~ ~ ~ p=1.000 n=6
Symbols 380,758 380,783 +25 (+ 0.01%) ~ ~ p=0.001 n=6
Types 166,811 166,860 +49 (+ 0.03%) ~ ~ p=0.001 n=6
Memory used 540,920k (± 0.02%) 540,958k (± 0.01%) ~ 540,869k 541,015k p=0.630 n=6
Parse Time 4.72s (± 0.35%) 4.69s (± 0.29%) ~ 4.68s 4.71s p=0.059 n=6
Bind Time 2.04s (± 1.80%) 2.05s (± 1.56%) ~ 2.01s 2.10s p=0.746 n=6
Check Time 22.99s (± 0.27%) 22.97s (± 0.26%) ~ 22.89s 23.04s p=0.872 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 29.73s (± 0.15%) 29.72s (± 0.14%) ~ 29.67s 29.77s p=0.746 n=6
xstate-main - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 692,951 693,393 +442 (+ 0.06%) ~ ~ p=0.001 n=6
Types 210,425 210,996 +571 (+ 0.27%) ~ ~ p=0.001 n=6
Memory used 588,755k (± 0.01%) 589,200k (± 0.00%) +445k (+ 0.08%) 589,174k 589,239k p=0.005 n=6
Parse Time 3.34s (± 0.68%) 3.35s (± 0.61%) ~ 3.32s 3.37s p=0.514 n=6
Bind Time 1.19s (± 0.43%) 1.18s (± 0.44%) ~ 1.18s 1.19s p=0.311 n=6
Check Time 18.01s (± 0.19%) 18.04s (± 0.52%) ~ 17.97s 18.22s p=0.936 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.54s (± 0.13%) 22.58s (± 0.35%) ~ 22.51s 22.73s p=0.418 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@typescript-bot
Copy link
Collaborator

typescript-bot commented Dec 4, 2025

@jakebailey Here are the results of running the top 400 repos with tsc comparing main and refs/pull/55222/merge:

Everything looks good!

@jakebailey
Copy link
Member Author

jakebailey commented Dec 5, 2025

Hm, is this finally not bad?

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Dec 5, 2025 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this faster ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

typescript-bot commented Dec 5, 2025

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 1 1 ~ ~ ~ p=1.000 n=6
Symbols 62,370 62,370 ~ ~ ~ p=1.000 n=6
Types 50,387 50,387 ~ ~ ~ p=1.000 n=6
Memory used 195,557k (± 0.77%) 195,726k (± 0.63%) ~ 193,217k 196,344k p=0.378 n=6
Parse Time 1.60s (± 0.76%) 1.61s (± 1.18%) ~ 1.59s 1.63s p=0.251 n=6
Bind Time 0.91s (± 2.41%) 0.91s (± 1.51%) ~ 0.89s 0.92s p=0.804 n=6
Check Time 11.98s (± 0.29%) 11.95s (± 0.47%) ~ 11.88s 12.02s p=0.375 n=6
Emit Time 3.52s (± 3.33%) 3.40s (± 4.99%) ~ 3.23s 3.59s p=0.199 n=6
Total Time 18.00s (± 0.37%) 17.87s (± 1.06%) ~ 17.65s 18.10s p=0.422 n=6
angular-1 - node (v18.15.0, x64)
Errors 2 2 ~ ~ ~ p=1.000 n=6
Symbols 956,048 956,195 +147 (+ 0.02%) ~ ~ p=0.001 n=6
Types 415,904 415,952 +48 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 1,255,070k (± 0.01%) 1,255,129k (± 0.00%) ~ 1,255,098k 1,255,151k p=0.066 n=6
Parse Time 6.56s (± 0.66%) 6.55s (± 0.43%) ~ 6.52s 6.60s p=0.872 n=6
Bind Time 1.96s (± 0.21%) 1.96s (± 0.32%) ~ 1.95s 1.97s p=0.673 n=6
Check Time 32.40s (± 0.37%) 32.34s (± 0.25%) ~ 32.22s 32.44s p=0.575 n=6
Emit Time 14.87s (± 0.88%) 14.89s (± 0.20%) ~ 14.86s 14.94s p=0.936 n=6
Total Time 55.78s (± 0.47%) 55.74s (± 0.17%) ~ 55.61s 55.88s p=0.471 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,722,128 2,724,104 +1,976 (+ 0.07%) ~ ~ p=0.001 n=6
Types 937,531 938,722 +1,191 (+ 0.13%) ~ ~ p=0.001 n=6
Memory used 3,050,992k (± 0.00%) 3,052,546k (± 0.00%) +1,554k (+ 0.05%) 3,052,405k 3,052,716k p=0.005 n=6
Parse Time 8.55s (± 0.17%) 8.55s (± 0.27%) ~ 8.52s 8.58s p=0.871 n=6
Bind Time 2.31s (± 0.35%) 2.32s (± 0.59%) ~ 2.30s 2.34s p=0.133 n=6
Check Time 93.10s (± 0.56%) 93.36s (± 0.34%) ~ 92.85s 93.76s p=0.335 n=6
Emit Time 0.31s (± 2.41%) 0.31s ~ ~ ~ p=0.598 n=6
Total Time 104.28s (± 0.50%) 104.54s (± 0.31%) ~ 104.01s 104.95s p=0.298 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,251,654 1,251,674 +20 (+ 0.00%) ~ ~ p=0.001 n=6
Types 259,840 259,873 +33 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 2,392,939k (± 0.02%) 2,393,323k (± 0.03%) ~ 2,392,743k 2,394,348k p=0.173 n=6
Parse Time 5.16s (± 0.74%) 5.18s (± 0.87%) ~ 5.14s 5.26s p=0.936 n=6
Bind Time 1.82s (± 1.39%) 1.82s (± 0.85%) ~ 1.81s 1.85s p=0.745 n=6
Check Time 35.57s (± 0.25%) 35.55s (± 0.21%) ~ 35.43s 35.65s p=1.000 n=6
Emit Time 3.01s (± 1.56%) 2.96s (± 1.52%) ~ 2.90s 3.02s p=0.149 n=6
Total Time 45.56s (± 0.22%) 45.52s (± 0.16%) ~ 45.41s 45.63s p=0.575 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,251,654 1,251,674 +20 (+ 0.00%) ~ ~ p=0.001 n=6
Types 259,840 259,873 +33 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 2,459,758k (± 0.02%) 2,459,534k (± 0.01%) ~ 2,459,267k 2,459,961k p=0.575 n=6
Parse Time 5.34s (± 0.85%) 5.33s (± 1.11%) ~ 5.28s 5.44s p=0.521 n=6
Bind Time 1.87s (± 0.48%) 1.85s (± 1.00%) ~ 1.82s 1.87s p=0.084 n=6
Check Time 35.55s (± 0.27%) 35.57s (± 0.39%) ~ 35.44s 35.77s p=0.936 n=6
Emit Time 3.03s (± 1.05%) 3.03s (± 1.06%) ~ 3.00s 3.07s p=1.000 n=6
Total Time 45.82s (± 0.21%) 45.80s (± 0.45%) ~ 45.63s 46.18s p=0.575 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 264,287 264,301 +14 (+ 0.01%) ~ ~ p=0.001 n=6
Types 104,051 104,078 +27 (+ 0.03%) ~ ~ p=0.001 n=6
Memory used 443,332k (± 0.01%) 443,391k (± 0.01%) ~ 443,325k 443,489k p=0.093 n=6
Parse Time 3.52s (± 0.55%) 3.53s (± 0.66%) ~ 3.51s 3.57s p=0.413 n=6
Bind Time 1.38s (± 0.59%) 1.37s (± 1.25%) ~ 1.35s 1.39s p=0.235 n=6
Check Time 19.23s (± 0.52%) 19.29s (± 0.65%) ~ 19.13s 19.43s p=0.471 n=6
Emit Time 1.54s (± 1.28%) 1.55s (± 1.18%) ~ 1.52s 1.57s p=0.801 n=6
Total Time 25.68s (± 0.41%) 25.74s (± 0.53%) ~ 25.56s 25.86s p=0.260 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 72 72 ~ ~ ~ p=1.000 n=6
Symbols 225,493 225,493 ~ ~ ~ p=1.000 n=6
Types 94,373 94,373 ~ ~ ~ p=1.000 n=6
Memory used 370,358k (± 0.05%) 370,160k (± 0.02%) -198k (- 0.05%) 370,087k 370,307k p=0.045 n=6
Parse Time 2.82s (± 0.76%) 2.82s (± 1.13%) ~ 2.78s 2.86s p=0.808 n=6
Bind Time 1.65s (± 0.92%) 1.65s (± 1.55%) ~ 1.61s 1.67s p=0.371 n=6
Check Time 16.61s (± 0.39%) 16.62s (± 0.26%) ~ 16.55s 16.68s p=0.747 n=6
Emit Time 0.00s (±244.70%) 0.00s (±244.70%) ~ 0.00s 0.01s p=1.000 n=6
Total Time 21.08s (± 0.44%) 21.09s (± 0.18%) ~ 21.04s 21.14s p=0.873 n=6
vscode - node (v18.15.0, x64)
Errors 11 11 ~ ~ ~ p=1.000 n=6
Symbols 4,080,067 4,100,759 +20,692 (+ 0.51%) ~ ~ p=0.001 n=6
Types 1,285,599 1,288,377 +2,778 (+ 0.22%) ~ ~ p=0.001 n=6
Memory used 3,865,718k (± 0.00%) 3,874,178k (± 0.00%) +8,460k (+ 0.22%) 3,874,012k 3,874,282k p=0.005 n=6
Parse Time 15.76s (± 0.69%) 15.69s (± 0.78%) ~ 15.53s 15.86s p=0.423 n=6
Bind Time 5.27s (± 0.35%) 5.32s (± 1.31%) ~ 5.23s 5.44s p=0.170 n=6
Check Time 115.63s (± 3.82%) 111.82s (± 1.15%) ~ 110.37s 114.23s p=0.093 n=6
Emit Time 42.03s (±11.39%) 41.20s (±12.80%) ~ 38.91s 51.97s p=0.471 n=6
Total Time 178.70s (± 4.97%) 174.02s (± 2.70%) ~ 171.30s 183.40s p=0.173 n=6
webpack - node (v18.15.0, x64)
Errors 40 40 ~ ~ ~ p=1.000 n=6
Symbols 380,702 380,727 +25 (+ 0.01%) ~ ~ p=0.001 n=6
Types 166,804 166,853 +49 (+ 0.03%) ~ ~ p=0.001 n=6
Memory used 540,934k (± 0.02%) 540,962k (± 0.01%) ~ 540,892k 541,033k p=0.471 n=6
Parse Time 4.72s (± 0.53%) 4.73s (± 0.31%) ~ 4.71s 4.75s p=0.370 n=6
Bind Time 2.03s (± 1.58%) 2.05s (± 1.21%) ~ 2.02s 2.08s p=0.333 n=6
Check Time 22.96s (± 0.47%) 23.01s (± 0.23%) ~ 22.92s 23.06s p=0.377 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 29.72s (± 0.49%) 29.80s (± 0.20%) ~ 29.71s 29.87s p=0.378 n=6
xstate-main - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 692,951 693,393 +442 (+ 0.06%) ~ ~ p=0.001 n=6
Types 210,425 210,996 +571 (+ 0.27%) ~ ~ p=0.001 n=6
Memory used 588,912k (± 0.02%) 589,418k (± 0.03%) +506k (+ 0.09%) 589,257k 589,609k p=0.005 n=6
Parse Time 4.18s (± 0.48%) 4.18s (± 0.41%) ~ 4.16s 4.21s p=1.000 n=6
Bind Time 1.41s (± 1.98%) 1.40s (± 1.38%) ~ 1.38s 1.43s p=0.625 n=6
Check Time 21.34s (± 1.32%) 21.10s (± 1.83%) ~ 20.69s 21.53s p=0.230 n=6
Emit Time 0.00s (±244.70%) 0.00s (±154.76%) ~ 0.00s 0.01s p=0.595 n=6
Total Time 26.92s (± 1.07%) 26.68s (± 1.46%) ~ 26.26s 27.12s p=0.298 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@typescript-bot typescript-bot added the For Backlog Bug PRs that fix a backlog bug label Dec 5, 2025
@jakebailey
Copy link
Member Author

jakebailey commented Dec 5, 2025

@typescript-bot test it

@typescript-bot
Copy link
Collaborator

typescript-bot commented Dec 5, 2025 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
test top400 ✅ Started ✅ Results
user test this ✅ Started ✅ Results
run dt ✅ Started 👀 Results
perf test this faster ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

typescript-bot commented Dec 5, 2025

Hey @jakebailey, the results of running the DT tests are ready.

There were interesting changes:

Branch only errors:

Package: ember
Error:

Error: Debug Failure. Expected false === true. Cached relationship does not match recalculated result
Occurred while linting /mnt/vss/_work/1/DefinitelyTyped/types/ember/test/private/observable-tests.ts:2
Rule: "@definitelytyped/expect"
    at recursiveTypeRelatedTo (/mnt/vss/_work/1/s/built/local/typescript.js:70311:15)
    at isRelatedTo (/mnt/vss/_work/1/s/built/local/typescript.js:69718:122)
    at typeRelatedToSomeType (/mnt/vss/_work/1/s/built/local/typescript.js:70010:11)
    at unionOrIntersectionRelatedTo (/mnt/vss/_work/1/s/built/local/typescript.js:69909:16)
    at structuredTypeRelatedToWorker (/mnt/vss/_work/1/s/built/local/typescript.js:70559:23)
    at structuredTypeRelatedTo (/mnt/vss/_work/1/s/built/local/typescript.js:70359:21)
    at recursiveTypeRelatedTo (/mnt/vss/_work/1/s/built/local/typescript.js:70305:19)
    at isRelatedTo (/mnt/vss/_work/1/s/built/local/typescript.js:69718:122)
    at eachTypeRelatedToType (/mnt/vss/_work/1/s/built/local/typescript.js:70094:25)
    at unionOrIntersectionRelatedTo (/mnt/vss/_work/1/s/built/local/typescript.js:69906:174)

Package: ember/v3
Error:

Error: Debug Failure. Expected false === true. Cached relationship does not match recalculated result
Occurred while linting /mnt/vss/_work/1/DefinitelyTyped/types/ember/v3/test/private/observable-tests.ts:2
Rule: "@definitelytyped/expect"
    at recursiveTypeRelatedTo (/mnt/vss/_work/1/s/built/local/typescript.js:70311:15)
    at isRelatedTo (/mnt/vss/_work/1/s/built/local/typescript.js:69718:122)
    at typeRelatedToSomeType (/mnt/vss/_work/1/s/built/local/typescript.js:70010:11)
    at unionOrIntersectionRelatedTo (/mnt/vss/_work/1/s/built/local/typescript.js:69909:16)
    at structuredTypeRelatedToWorker (/mnt/vss/_work/1/s/built/local/typescript.js:70559:23)
    at structuredTypeRelatedTo (/mnt/vss/_work/1/s/built/local/typescript.js:70359:21)
    at recursiveTypeRelatedTo (/mnt/vss/_work/1/s/built/local/typescript.js:70305:19)
    at isRelatedTo (/mnt/vss/_work/1/s/built/local/typescript.js:69718:122)
    at eachTypeRelatedToType (/mnt/vss/_work/1/s/built/local/typescript.js:70094:25)
    at unionOrIntersectionRelatedTo (/mnt/vss/_work/1/s/built/local/typescript.js:69906:174)

You can check the log here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Dec 5, 2025

@jakebailey Here are the results of running the user tests with tsc comparing main and refs/pull/55222/merge:

There were infrastructure failures potentially unrelated to your change:

  • 1 instance of "Git clone failed"

Otherwise...

Everything looks good!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Dec 5, 2025

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 1 1 ~ ~ ~ p=1.000 n=6
Symbols 62,370 62,370 ~ ~ ~ p=1.000 n=6
Types 50,387 50,387 ~ ~ ~ p=1.000 n=6
Memory used 193,363k (± 0.78%) 194,538k (± 1.01%) ~ 192,609k 196,456k p=0.575 n=6
Parse Time 1.30s (± 0.63%) 1.30s (± 0.80%) ~ 1.29s 1.32s p=0.865 n=6
Bind Time 0.75s 0.75s ~ ~ ~ p=1.000 n=6
Check Time 9.91s (± 0.39%) 9.88s (± 0.41%) ~ 9.83s 9.93s p=0.416 n=6
Emit Time 2.73s (± 0.89%) 2.74s (± 0.91%) ~ 2.71s 2.77s p=0.405 n=6
Total Time 14.69s (± 0.21%) 14.69s (± 0.37%) ~ 14.61s 14.75s p=0.936 n=6
angular-1 - node (v18.15.0, x64)
Errors 2 2 ~ ~ ~ p=1.000 n=6
Symbols 956,048 956,195 +147 (+ 0.02%) ~ ~ p=0.001 n=6
Types 415,904 415,952 +48 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 1,255,040k (± 0.00%) 1,255,171k (± 0.00%) +131k (+ 0.01%) 1,255,095k 1,255,262k p=0.008 n=6
Parse Time 6.50s (± 0.29%) 6.53s (± 0.75%) ~ 6.48s 6.61s p=0.124 n=6
Bind Time 1.96s (± 0.62%) 1.97s (± 0.28%) ~ 1.96s 1.97s p=0.863 n=6
Check Time 32.29s (± 0.36%) 32.36s (± 0.35%) ~ 32.27s 32.57s p=0.109 n=6
Emit Time 14.79s (± 1.04%) 14.85s (± 0.74%) ~ 14.64s 14.96s p=0.423 n=6
Total Time 55.54s (± 0.25%) 55.71s (± 0.36%) ~ 55.43s 55.94s p=0.198 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,722,128 2,724,104 +1,976 (+ 0.07%) ~ ~ p=0.001 n=6
Types 937,531 938,722 +1,191 (+ 0.13%) ~ ~ p=0.001 n=6
Memory used 3,050,982k (± 0.00%) 3,052,548k (± 0.00%) +1,566k (+ 0.05%) 3,052,471k 3,052,668k p=0.005 n=6
Parse Time 8.57s (± 0.23%) 8.55s (± 0.06%) ~ 8.55s 8.56s p=0.276 n=6
Bind Time 2.32s (± 0.45%) 2.32s (± 0.71%) ~ 2.30s 2.35s p=0.315 n=6
Check Time 93.10s (± 0.27%) 93.04s (± 0.48%) ~ 92.35s 93.60s p=0.936 n=6
Emit Time 0.32s (± 1.74%) 0.31s (± 1.31%) ~ 0.31s 0.32s p=0.282 n=6
Total Time 104.30s (± 0.23%) 104.23s (± 0.44%) ~ 103.51s 104.78s p=0.936 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,251,654 1,251,678 +24 (+ 0.00%) ~ ~ p=0.001 n=6
Types 259,840 259,875 +35 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 2,514,190k (±11.82%) 2,514,865k (±11.80%) ~ 2,392,442k 3,121,396k p=0.298 n=6
Parse Time 5.20s (± 1.33%) 5.19s (± 0.99%) ~ 5.12s 5.27s p=0.873 n=6
Bind Time 1.82s (± 0.96%) 1.81s (± 1.42%) ~ 1.77s 1.85s p=0.870 n=6
Check Time 35.59s (± 0.86%) 35.61s (± 0.43%) ~ 35.40s 35.81s p=0.471 n=6
Emit Time 3.01s (± 0.78%) 3.00s (± 1.65%) ~ 2.91s 3.04s p=0.747 n=6
Total Time 45.64s (± 0.77%) 45.62s (± 0.39%) ~ 45.45s 45.86s p=0.575 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,251,654 1,251,678 +24 (+ 0.00%) ~ ~ p=0.001 n=6
Types 259,840 259,875 +35 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 3,188,621k (± 0.02%) 3,188,574k (± 0.03%) ~ 3,187,283k 3,189,855k p=1.000 n=6
Parse Time 6.80s (± 0.51%) 6.80s (± 1.17%) ~ 6.73s 6.92s p=0.378 n=6
Bind Time 2.26s (± 0.71%) 2.24s (± 0.88%) ~ 2.22s 2.27s p=0.297 n=6
Check Time 43.21s (± 0.24%) 43.21s (± 0.15%) ~ 43.14s 43.33s p=0.689 n=6
Emit Time 3.51s (± 2.76%) 3.54s (± 1.00%) ~ 3.50s 3.59s p=0.873 n=6
Total Time 55.79s (± 0.24%) 55.79s (± 0.16%) ~ 55.70s 55.93s p=1.000 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 264,287 264,305 +18 (+ 0.01%) ~ ~ p=0.001 n=6
Types 104,051 104,080 +29 (+ 0.03%) ~ ~ p=0.001 n=6
Memory used 443,277k (± 0.01%) 443,347k (± 0.01%) +70k (+ 0.02%) 443,280k 443,418k p=0.031 n=6
Parse Time 3.52s (± 0.74%) 3.51s (± 0.55%) ~ 3.48s 3.53s p=0.872 n=6
Bind Time 1.36s (± 0.76%) 1.37s (± 0.61%) ~ 1.35s 1.37s p=0.672 n=6
Check Time 19.16s (± 0.45%) 19.19s (± 0.19%) ~ 19.13s 19.23s p=0.688 n=6
Emit Time 1.54s (± 0.95%) 1.54s (± 1.19%) ~ 1.52s 1.57s p=0.681 n=6
Total Time 25.59s (± 0.39%) 25.61s (± 0.16%) ~ 25.57s 25.68s p=0.872 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 72 72 ~ ~ ~ p=1.000 n=6
Symbols 225,493 225,493 ~ ~ ~ p=1.000 n=6
Types 94,373 94,373 ~ ~ ~ p=1.000 n=6
Memory used 370,146k (± 0.01%) 370,239k (± 0.05%) ~ 370,128k 370,590k p=0.261 n=6
Parse Time 2.82s (± 1.07%) 2.83s (± 1.02%) ~ 2.79s 2.87s p=0.419 n=6
Bind Time 1.64s (± 1.51%) 1.65s (± 1.73%) ~ 1.62s 1.70s p=0.744 n=6
Check Time 16.62s (± 0.33%) 16.65s (± 0.26%) ~ 16.59s 16.72s p=0.418 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 21.08s (± 0.29%) 21.14s (± 0.33%) ~ 21.05s 21.23s p=0.261 n=6
vscode - node (v18.15.0, x64)
Errors 11 11 ~ ~ ~ p=1.000 n=6
Symbols 4,080,068 4,100,760 +20,692 (+ 0.51%) ~ ~ p=0.001 n=6
Types 1,285,599 1,288,377 +2,778 (+ 0.22%) ~ ~ p=0.001 n=6
Memory used 3,865,623k (± 0.00%) 3,874,298k (± 0.00%) +8,676k (+ 0.22%) 3,874,076k 3,874,445k p=0.005 n=6
Parse Time 15.73s (± 0.34%) 15.70s (± 0.53%) ~ 15.56s 15.79s p=0.575 n=6
Bind Time 5.28s (± 0.54%) 5.29s (± 0.65%) ~ 5.25s 5.34s p=0.571 n=6
Check Time 113.82s (± 2.04%) 113.55s (± 2.45%) ~ 111.00s 117.08s p=0.689 n=6
Emit Time 41.07s (± 7.26%) 40.59s (± 6.35%) ~ 38.94s 45.35s p=0.936 n=6
Total Time 175.90s (± 1.84%) 175.12s (± 2.66%) ~ 171.11s 183.40s p=0.575 n=6
webpack - node (v18.15.0, x64)
Errors 40 0 ~ ~ ~ p=1.000 n=6+0
Symbols 380,702 0 ~ ~ ~ p=1.000 n=6+0
Types 166,804 0 ~ ~ ~ p=1.000 n=6+0
Memory used 540,951k (± 0.02%) 0k ~ ~ ~ p=1.000 n=6+0
Parse Time 4.74s (± 0.35%) 0s ~ ~ ~ p=1.000 n=6+0
Bind Time 2.03s (± 1.24%) 0s ~ ~ ~ p=1.000 n=6+0
Check Time 23.09s (± 0.53%) 0s ~ ~ ~ p=1.000 n=6+0
Emit Time 0.00s 0s ~ ~ ~ p=1.000 n=6+0
Total Time 29.85s (± 0.45%) 0s ~ ~ ~ p=1.000 n=6+0
xstate-main - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 692,951 693,393 +442 (+ 0.06%) ~ ~ p=0.001 n=6
Types 210,425 210,996 +571 (+ 0.27%) ~ ~ p=0.001 n=6
Memory used 588,936k (± 0.03%) 589,496k (± 0.03%) +560k (+ 0.10%) 589,271k 589,689k p=0.005 n=6
Parse Time 4.16s (± 0.41%) 4.17s (± 0.66%) ~ 4.14s 4.21s p=0.747 n=6
Bind Time 1.42s (± 1.51%) 1.41s (± 0.83%) ~ 1.40s 1.43s p=0.328 n=6
Check Time 21.09s (± 1.79%) 20.98s (± 2.02%) ~ 20.59s 21.54s p=0.936 n=6
Emit Time 0.00s (±244.70%) 0.00s (±244.70%) ~ 0.00s 0.01s p=1.000 n=6
Total Time 26.67s (± 1.44%) 26.57s (± 1.62%) ~ 26.18s 27.19s p=0.572 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@jakebailey
Copy link
Member Author

jakebailey commented Dec 5, 2025

@typescript-bot pack this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Dec 5, 2025 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
pack this ✅ Started ✅ Results

@typescript-bot
Copy link
Collaborator

typescript-bot commented Dec 5, 2025

Hey @jakebailey, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/166713/artifacts?artifactName=tgz&fileId=88B8D7DC4F8B30D70847FF5C5138FD4CC81F42D5276520090996FB91B371E03E02&fileName=/typescript-6.0.0-insiders.20251205.tgz"
    }
}

and then running npm install.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Dec 5, 2025

@jakebailey Here are the results of running the top 400 repos with tsc comparing main and refs/pull/55222/merge:

Everything looks good!

@Andarist
Copy link
Contributor

Andarist commented Dec 5, 2025

Repro for the ember break above:

type ObserverMethod<Target, Sender> = keyof Target | ((sender: Sender) => void);

type ExtractPropertyNamesOfType<T, S> = {
  [K in keyof T]: T[K] extends S ? K : never;
}[keyof T];

interface Observable {
  notifyPropertyChange(): this;
  removeObserver<Target>(method: ObserverMethod<Target, this>): this;
  decrementProperty(keyName: keyof this): number;
}

declare class DemoObservable implements Observable {
  baz: number;

  notifyPropertyChange(): this;
  removeObserver<Target>(method: ObserverMethod<Target, this>): any;
  decrementProperty(
    keyName: ExtractPropertyNamesOfType<this, number | undefined>,
  ): number;
}

const o = new DemoObservable();

declare function assertType<T>(value: T): T;
assertType<Observable>(o.notifyPropertyChange("jeanShorts"));

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@weswigham weswigham Awaiting requested review from weswigham

Copilot code review Copilot Awaiting requested review from Copilot Copilot will automatically review once the pull request is marked ready for review

At least 1 approving review is required to merge this pull request.

Assignees

@jakebailey jakebailey

Labels

Author: Team Experiment A fork with an experimental idea which might not make it into master For Backlog Bug PRs that fix a backlog bug For Milestone Bug PRs that fix a bug with a specific milestone

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Error: Debug Failure. No error for last overload signature Conditional type triggers "No error for last overload signature" exception

5 participants

@jakebailey @typescript-bot @weswigham @Andarist @sandersn