Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add resolveLibrary method on hosts and store resolvedLibraries in program so that resolutions can be reused #53877

Merged
merged 6 commits into from
Apr 20, 2023
Merged

Add resolveLibrary method on hosts and store resolvedLibraries in program so that resolutions can be reused #53877

merged 6 commits into from
Apr 20, 2023

Conversation

sheetalkamat
Copy link
Member

@sheetalkamat sheetalkamat commented Apr 17, 2023 •
edited by jakebailey
Loading

With this PR:

  • Program maintains resolvedLibraries which contains the @typescript/libxxx resolution and the actual file used as lib so that resolutions are not re-done just to answer if the file is "lib" (c5dc1a0 though this one is only maintaining the actual lib file used the next commit: fc8c4da modifies to store the resolution to so it can be reused)
  • Just like resolveModuleNameLiterals or resolveTypeReferenceDirectiveReferences we add resolveLibrary method on all hosts and corresponding hasInvalidatedLibResolutions to determine if lib resolution is valid. This enables reuse of the library resolution and caching it across program boundaries. This also fixes to use different module resolution cache for library resolutions since its the cache with different options than module resolution. fc8c4da
  • In watch mode and tsserver, we watch library resolutions just like any other module resolution so on changes we can invoke program update and continue to reuse resolutions that are not invalidated. fc8c4da

Fixes #52759
Fixes #52707

@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Apr 17, 2023
@sheetalkamat sheetalkamat force-pushed the libResolution branch 6 times, most recently from 924030f to d7cfc01 Compare April 20, 2023 02:52
@sheetalkamat sheetalkamat marked this pull request as ready for review April 20, 2023 04:59
@sheetalkamat sheetalkamat requested review from jakebailey and andrewbranch and removed request for jakebailey April 20, 2023 04:59
@sheetalkamat sheetalkamat linked an issue Apr 20, 2023 that may be closed by this pull request
TS Server keeps checking for files under node_modules/@typescript even when that directory doesn't exist #52707
Closed
@sheetalkamat sheetalkamat changed the title Lib resolution Add resolveLibrary method on hosts and store resolvedLibraries in program so that resolutions can be reused Apr 20, 2023
@andrewbranch
Copy link
Member

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Apr 20, 2023 •
edited
Loading

Heya @andrewbranch, I've started to run the perf test suite on this PR at fc8c4da. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@andrewbranch
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - main..53877
Metric main 53877 Delta Best Worst p-value
Angular - node (v18.10.0, x64)
Memory used 365,734k (± 0.00%) 365,733k (± 0.00%) ~ 365,704k 365,750k p=0.872 n=6
Parse Time 3.38s (± 0.64%) 3.37s (± 0.53%) ~ 3.35s 3.40s p=0.808 n=6
Bind Time 1.11s (± 0.37%) 1.11s (± 0.00%) ~ 1.11s 1.11s p=0.405 n=6
Check Time 8.70s (± 0.29%) 8.70s (± 0.39%) ~ 8.64s 8.73s p=1.000 n=6
Emit Time 7.43s (± 0.38%) 7.42s (± 0.32%) ~ 7.39s 7.46s p=0.686 n=6
Total Time 20.62s (± 0.27%) 20.60s (± 0.20%) ~ 20.53s 20.65s p=0.420 n=6
Compiler-Unions - node (v18.10.0, x64)
Memory used 191,878k (± 1.18%) 191,903k (± 1.17%) ~ 190,968k 196,490k p=0.128 n=6
Parse Time 1.49s (± 0.50%) 1.49s (± 0.42%) ~ 1.48s 1.50s p=0.718 n=6
Bind Time 0.77s (± 0.82%) 0.77s (± 0.82%) ~ 0.76s 0.78s p=1.000 n=6
Check Time 9.50s (± 0.50%) 9.52s (± 0.52%) ~ 9.48s 9.61s p=0.936 n=6
Emit Time 2.71s (± 0.98%) 2.70s (± 0.76%) ~ 2.68s 2.74s p=1.000 n=6
Total Time 14.47s (± 0.25%) 14.49s (± 0.37%) ~ 14.43s 14.57s p=0.686 n=6
Monaco - node (v18.10.0, x64)
Memory used 346,501k (± 0.01%) 346,498k (± 0.01%) ~ 346,470k 346,538k p=1.000 n=6
Parse Time 2.59s (± 0.87%) 2.59s (± 0.82%) ~ 2.57s 2.63s p=0.808 n=6
Bind Time 1.01s (± 0.81%) 1.00s (± 0.41%) ~ 0.99s 1.00s p=0.056 n=6
Check Time 7.12s (± 0.22%) 7.14s (± 0.63%) ~ 7.10s 7.21s p=0.807 n=6
Emit Time 4.24s (± 0.85%) 4.23s (± 0.91%) ~ 4.17s 4.27s p=0.628 n=6
Total Time 14.95s (± 0.44%) 14.95s (± 0.62%) ~ 14.87s 15.11s p=0.873 n=6
TFS - node (v18.10.0, x64)
Memory used 300,722k (± 0.01%) 300,738k (± 0.01%) ~ 300,715k 300,766k p=0.172 n=6
Parse Time 2.06s (± 0.75%) 2.03s (± 0.90%) -0.03s (- 1.38%) 2.01s 2.06s p=0.029 n=6
Bind Time 1.14s (± 0.66%) 1.14s (± 0.66%) ~ 1.13s 1.15s p=0.487 n=6
Check Time 6.53s (± 0.43%) 6.52s (± 0.18%) ~ 6.51s 6.54s p=0.625 n=6
Emit Time 3.86s (± 1.07%) 3.88s (± 0.41%) ~ 3.86s 3.90s p=0.375 n=6
Total Time 13.59s (± 0.38%) 13.57s (± 0.14%) ~ 13.55s 13.60s p=0.627 n=6
material-ui - node (v18.10.0, x64)
Memory used 482,296k (± 0.02%) 482,283k (± 0.01%) ~ 482,187k 482,367k p=0.936 n=6
Parse Time 3.08s (± 0.38%) 3.10s (± 0.44%) ~ 3.08s 3.11s p=0.119 n=6
Bind Time 0.92s (± 0.60%) 0.91s (± 0.57%) -0.01s (- 0.91%) 0.90s 0.91s p=0.038 n=6
Check Time 16.77s (± 0.28%) 16.85s (± 0.22%) +0.08s (+ 0.50%) 16.81s 16.91s p=0.006 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 20.76s (± 0.23%) 20.85s (± 0.26%) +0.09s (+ 0.43%) 20.79s 20.93s p=0.012 n=6
xstate - node (v18.10.0, x64)
Memory used 563,700k (± 0.03%) 563,004k (± 0.02%) -697k (- 0.12%) 562,861k 563,165k p=0.005 n=6
Parse Time 3.87s (± 0.60%) 3.82s (± 0.58%) -0.05s (- 1.38%) 3.78s 3.84s p=0.012 n=6
Bind Time 1.64s (± 0.51%) 1.63s (± 0.51%) ~ 1.63s 1.65s p=0.101 n=6
Check Time 2.81s (± 1.00%) 2.79s (± 0.98%) ~ 2.77s 2.84s p=0.627 n=6
Emit Time 0.08s (± 0.00%) 0.08s (± 0.00%) ~ 0.08s 0.08s p=1.000 n=6
Total Time 8.40s (± 0.57%) 8.33s (± 0.26%) -0.08s (- 0.91%) 8.31s 8.37s p=0.022 n=6
Angular - node (v16.17.1, x64)
Memory used 365,150k (± 0.01%) 365,195k (± 0.01%) +45k (+ 0.01%) 365,165k 365,225k p=0.020 n=6
Parse Time 3.52s (± 0.71%) 3.54s (± 0.46%) ~ 3.52s 3.56s p=0.145 n=6
Bind Time 1.17s (± 0.44%) 1.17s (± 0.64%) ~ 1.16s 1.18s p=0.784 n=6
Check Time 9.51s (± 0.70%) 9.51s (± 0.47%) ~ 9.42s 9.54s p=0.936 n=6
Emit Time 7.91s (± 0.39%) 7.89s (± 0.48%) ~ 7.85s 7.95s p=0.413 n=6
Total Time 22.12s (± 0.46%) 22.11s (± 0.30%) ~ 22.01s 22.21s p=0.936 n=6
Compiler-Unions - node (v16.17.1, x64)
Memory used 193,183k (± 0.71%) 193,757k (± 0.86%) ~ 192,532k 196,054k p=0.230 n=6
Parse Time 1.59s (± 1.15%) 1.58s (± 1.18%) ~ 1.55s 1.60s p=0.805 n=6
Bind Time 0.83s (± 0.62%) 0.83s (± 0.66%) ~ 0.82s 0.83s p=0.640 n=6
Check Time 10.21s (± 0.42%) 10.23s (± 0.64%) ~ 10.14s 10.31s p=0.520 n=6
Emit Time 3.00s (± 0.94%) 2.99s (± 0.65%) ~ 2.97s 3.02s p=0.517 n=6
Total Time 15.63s (± 0.29%) 15.63s (± 0.43%) ~ 15.53s 15.72s p=0.871 n=6
Monaco - node (v16.17.1, x64)
Memory used 345,792k (± 0.01%) 345,773k (± 0.01%) ~ 345,725k 345,796k p=0.689 n=6
Parse Time 2.72s (± 0.57%) 2.73s (± 0.68%) ~ 2.71s 2.75s p=0.451 n=6
Bind Time 1.08s (± 1.23%) 1.09s (± 0.50%) ~ 1.08s 1.09s p=0.228 n=6
Check Time 7.80s (± 0.44%) 7.84s (± 0.56%) ~ 7.80s 7.90s p=0.124 n=6
Emit Time 4.42s (± 0.44%) 4.46s (± 0.66%) +0.04s (+ 0.94%) 4.43s 4.51s p=0.023 n=6
Total Time 16.03s (± 0.27%) 16.11s (± 0.41%) +0.08s (+ 0.53%) 16.03s 16.22s p=0.037 n=6
TFS - node (v16.17.1, x64)
Memory used 300,060k (± 0.01%) 300,077k (± 0.01%) ~ 300,044k 300,122k p=0.378 n=6
Parse Time 2.16s (± 1.16%) 2.16s (± 0.83%) ~ 2.14s 2.19s p=0.683 n=6
Bind Time 1.24s (± 1.39%) 1.24s (± 1.10%) ~ 1.21s 1.25s p=0.743 n=6
Check Time 7.20s (± 0.42%) 7.19s (± 0.45%) ~ 7.15s 7.24s p=0.511 n=6
Emit Time 4.35s (± 0.64%) 4.37s (± 1.05%) ~ 4.32s 4.43s p=0.518 n=6
Total Time 14.95s (± 0.36%) 14.95s (± 0.46%) ~ 14.86s 15.05s p=1.000 n=6
material-ui - node (v16.17.1, x64)
Memory used 481,527k (± 0.01%) 481,557k (± 0.01%) ~ 481,520k 481,628k p=0.298 n=6
Parse Time 3.25s (± 0.27%) 3.24s (± 0.32%) -0.01s (- 0.46%) 3.22s 3.25s p=0.039 n=6
Bind Time 0.94s (± 0.55%) 0.94s (± 0.80%) ~ 0.93s 0.95s p=0.241 n=6
Check Time 17.85s (± 0.41%) 17.76s (± 0.37%) ~ 17.66s 17.84s p=0.065 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.05s (± 0.35%) 21.93s (± 0.31%) -0.12s (- 0.53%) 21.83s 22.02s p=0.029 n=6
xstate - node (v16.17.1, x64)
Memory used 561,316k (± 0.02%) 560,564k (± 0.02%) -752k (- 0.13%) 560,407k 560,695k p=0.005 n=6
Parse Time 4.04s (± 0.41%) 3.98s (± 0.22%) -0.06s (- 1.61%) 3.97s 3.99s p=0.005 n=6
Bind Time 1.77s (± 0.29%) 1.77s (± 0.43%) ~ 1.76s 1.78s p=0.784 n=6
Check Time 3.02s (± 0.53%) 3.03s (± 0.81%) ~ 3.01s 3.06s p=0.732 n=6
Emit Time 0.09s (± 0.00%) 0.09s (± 0.00%) ~ 0.09s 0.09s p=1.000 n=6
Total Time 8.93s (± 0.34%) 8.87s (± 0.23%) -0.06s (- 0.67%) 8.84s 8.90s p=0.008 n=6
Angular - node (v14.21.3, x64)
Memory used 359,161k (± 0.00%) 359,181k (± 0.01%) ~ 359,143k 359,204k p=0.109 n=6
Parse Time 3.66s (± 0.33%) 3.65s (± 0.17%) ~ 3.64s 3.66s p=0.344 n=6
Bind Time 1.21s (± 0.69%) 1.21s (± 0.67%) ~ 1.21s 1.23s p=0.673 n=6
Check Time 9.93s (± 0.37%) 9.90s (± 0.62%) ~ 9.82s 9.98s p=0.377 n=6
Emit Time 8.27s (± 0.39%) 8.23s (± 0.58%) ~ 8.18s 8.30s p=0.197 n=6
Total Time 23.08s (± 0.24%) 23.00s (± 0.33%) ~ 22.91s 23.10s p=0.107 n=6
Compiler-Unions - node (v14.21.3, x64)
Memory used 187,724k (± 0.02%) 188,017k (± 0.02%) +293k (+ 0.16%) 187,978k 188,059k p=0.005 n=6
Parse Time 1.59s (± 0.51%) 1.60s (± 0.52%) ~ 1.59s 1.61s p=1.000 n=6
Bind Time 0.84s (± 0.61%) 0.85s (± 0.99%) ~ 0.84s 0.86s p=0.923 n=6
Check Time 10.33s (± 0.73%) 10.33s (± 0.38%) ~ 10.27s 10.39s p=0.810 n=6
Emit Time 3.45s (± 1.05%) 3.12s (± 0.71%) 🟩-0.33s (- 9.61%) 3.09s 3.15s p=0.005 n=6
Total Time 16.21s (± 0.56%) 15.89s (± 0.21%) -0.31s (- 1.94%) 15.85s 15.95s p=0.005 n=6
Monaco - node (v14.21.3, x64)
Memory used 340,841k (± 0.00%) 340,857k (± 0.01%) ~ 340,832k 340,885k p=0.128 n=6
Parse Time 2.79s (± 0.49%) 2.78s (± 0.32%) ~ 2.77s 2.79s p=0.054 n=6
Bind Time 1.11s (± 0.76%) 1.11s (± 1.67%) ~ 1.10s 1.15s p=0.652 n=6
Check Time 8.14s (± 0.70%) 8.14s (± 0.60%) ~ 8.09s 8.23s p=1.000 n=6
Emit Time 4.65s (± 0.92%) 4.67s (± 0.48%) ~ 4.63s 4.69s p=0.227 n=6
Total Time 16.69s (± 0.58%) 16.70s (± 0.35%) ~ 16.62s 16.77s p=0.747 n=6
TFS - node (v14.21.3, x64)
Memory used 295,231k (± 0.00%) 295,244k (± 0.00%) +13k (+ 0.00%) 295,231k 295,256k p=0.025 n=6
Parse Time 2.38s (± 0.62%) 2.39s (± 0.57%) ~ 2.37s 2.41s p=0.188 n=6
Bind Time 1.07s (± 1.09%) 1.07s (± 0.84%) ~ 1.06s 1.08s p=0.672 n=6
Check Time 7.54s (± 0.45%) 7.49s (± 0.96%) ~ 7.38s 7.60s p=0.227 n=6
Emit Time 4.30s (± 0.88%) 4.31s (± 1.29%) ~ 4.27s 4.41s p=0.936 n=6
Total Time 15.29s (± 0.28%) 15.27s (± 0.81%) ~ 15.14s 15.49s p=0.422 n=6
material-ui - node (v14.21.3, x64)
Memory used 477,157k (± 0.01%) 477,122k (± 0.01%) ~ 477,077k 477,155k p=0.078 n=6
Parse Time 3.32s (± 0.25%) 3.31s (± 0.30%) ~ 3.30s 3.32s p=0.498 n=6
Bind Time 1.00s (± 0.84%) 1.00s (± 0.52%) ~ 0.99s 1.00s p=0.533 n=6
Check Time 18.74s (± 0.59%) 18.64s (± 0.32%) ~ 18.55s 18.71s p=0.093 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 23.06s (± 0.48%) 22.94s (± 0.25%) ~ 22.87s 23.03s p=0.092 n=6
xstate - node (v14.21.3, x64)
Memory used 550,244k (± 0.00%) 549,529k (± 0.00%) -715k (- 0.13%) 549,503k 549,544k p=0.005 n=6
Parse Time 4.24s (± 0.38%) 4.18s (± 0.38%) -0.06s (- 1.41%) 4.16s 4.20s p=0.005 n=6
Bind Time 1.66s (± 0.31%) 1.64s (± 2.19%) ~ 1.57s 1.67s p=0.437 n=6
Check Time 3.16s (± 0.24%) 3.15s (± 0.57%) ~ 3.13s 3.18s p=0.250 n=6
Emit Time 0.09s (± 0.00%) 0.09s (± 0.00%) ~ 0.09s 0.09s p=1.000 n=6
Total Time 9.15s (± 0.21%) 9.06s (± 0.54%) -0.09s (- 0.95%) 8.97s 9.10s p=0.005 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.21.3, x64)
Scenarios
  • Angular - node (v18.10.0, x64)
  • Angular - node (v16.17.1, x64)
  • Angular - node (v14.21.3, x64)
  • Compiler-Unions - node (v18.10.0, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Compiler-Unions - node (v14.21.3, x64)
  • Monaco - node (v18.10.0, x64)
  • Monaco - node (v16.17.1, x64)
  • Monaco - node (v14.21.3, x64)
  • TFS - node (v18.10.0, x64)
  • TFS - node (v16.17.1, x64)
  • TFS - node (v14.21.3, x64)
  • material-ui - node (v18.10.0, x64)
  • material-ui - node (v16.17.1, x64)
  • material-ui - node (v14.21.3, x64)
  • xstate - node (v18.10.0, x64)
  • xstate - node (v16.17.1, x64)
  • xstate - node (v14.21.3, x64)
Benchmark Name Iterations
Current 53877 6
Baseline main 6

TSServer

Comparison Report - main..53877
Metric main 53877 Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 2,409ms (± 0.57%) 2,539ms (± 0.36%) +131ms (+ 5.42%) 2,529ms 2,554ms p=0.005 n=6
Req 2 - geterr 5,620ms (± 0.91%) 5,674ms (± 2.64%) ~ 5,416ms 5,884ms p=0.066 n=6
Req 3 - references 337ms (± 0.29%) 343ms (± 1.56%) +6ms (+ 1.78%) 338ms 351ms p=0.011 n=6
Req 4 - navto 285ms (± 0.41%) 283ms (± 0.93%) ~ 281ms 288ms p=0.125 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 84ms (± 2.22%) 81ms (± 8.07%) ~ 68ms 85ms p=0.618 n=6
CompilerTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 2,520ms (± 1.73%) 2,636ms (± 0.71%) +116ms (+ 4.60%) 2,601ms 2,654ms p=0.005 n=6
Req 2 - geterr 4,216ms (± 1.35%) 4,248ms (± 0.67%) ~ 4,195ms 4,276ms p=0.336 n=6
Req 3 - references 345ms (± 1.08%) 346ms (± 0.38%) ~ 345ms 348ms p=0.253 n=6
Req 4 - navto 289ms (± 0.44%) 289ms (± 0.57%) ~ 287ms 291ms p=0.620 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 67ms (± 6.48%) 67ms (± 5.35%) ~ 63ms 71ms p=0.746 n=6
xstateTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 3,027ms (± 0.38%) 3,075ms (± 0.50%) +49ms (+ 1.61%) 3,057ms 3,096ms p=0.005 n=6
Req 2 - geterr 1,606ms (± 1.03%) 1,623ms (± 1.15%) ~ 1,604ms 1,653ms p=0.127 n=6
Req 3 - references 106ms (± 0.71%) 106ms (± 1.25%) ~ 104ms 108ms p=1.000 n=6
Req 4 - navto 362ms (± 0.48%) 363ms (± 0.61%) ~ 361ms 367ms p=0.566 n=6
Req 5 - completionInfo count 2,862 (± 0.00%) 2,862 (± 0.00%) ~ 2,862 2,862 p=1.000 n=6
Req 5 - completionInfo 376ms (± 0.90%) 373ms (± 1.68%) ~ 364ms 382ms p=0.419 n=6
Compiler-UnionsTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,519ms (± 0.87%) 2,691ms (± 0.87%) +172ms (+ 6.83%) 2,670ms 2,720ms p=0.005 n=6
Req 2 - geterr 6,054ms (± 0.38%) 6,036ms (± 2.00%) ~ 5,876ms 6,157ms p=0.378 n=6
Req 3 - references 352ms (± 0.76%) 357ms (± 1.62%) ~ 351ms 366ms p=0.090 n=6
Req 4 - navto 286ms (± 1.33%) 285ms (± 0.56%) ~ 284ms 288ms p=0.459 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 89ms (± 5.03%) 81ms (± 7.54%) 🟩-9ms (- 9.53%) 74ms 91ms p=0.042 n=6
CompilerTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,689ms (± 0.46%) 2,825ms (± 0.43%) +135ms (+ 5.03%) 2,810ms 2,841ms p=0.005 n=6
Req 2 - geterr 4,624ms (± 0.40%) 4,623ms (± 0.53%) ~ 4,583ms 4,651ms p=0.936 n=6
Req 3 - references 360ms (± 0.46%) 362ms (± 0.25%) +2ms (+ 0.56%) 361ms 363ms p=0.021 n=6
Req 4 - navto 284ms (± 0.29%) 283ms (± 0.61%) ~ 280ms 285ms p=1.000 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 68ms (± 1.31%) 67ms (± 0.00%) -1ms (- 1.47%) 67ms 67ms p=0.028 n=6
xstateTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 3,166ms (± 0.26%) 3,224ms (± 0.40%) +58ms (+ 1.84%) 3,208ms 3,240ms p=0.005 n=6
Req 2 - geterr 1,759ms (± 0.87%) 1,769ms (± 1.19%) ~ 1,745ms 1,794ms p=0.229 n=6
Req 3 - references 120ms (± 0.68%) 121ms (± 1.62%) ~ 118ms 123ms p=0.121 n=6
Req 4 - navto 340ms (± 0.36%) 340ms (± 0.30%) ~ 339ms 342ms p=0.408 n=6
Req 5 - completionInfo count 2,862 (± 0.00%) 2,862 (± 0.00%) ~ 2,862 2,862 p=1.000 n=6
Req 5 - completionInfo 395ms (± 1.53%) 399ms (± 0.89%) ~ 396ms 406ms p=0.054 n=6
Compiler-UnionsTSServer - node (v14.21.3, x64)
Req 1 - updateOpen 2,613ms (± 0.25%) 2,784ms (± 0.75%) +171ms (+ 6.54%) 2,757ms 2,820ms p=0.005 n=6
Req 2 - geterr 6,271ms (± 2.01%) 6,269ms (± 0.75%) ~ 6,227ms 6,349ms p=0.336 n=6
Req 3 - references 371ms (± 4.82%) 367ms (± 1.18%) ~ 362ms 373ms p=0.520 n=6
Req 4 - navto 285ms (± 0.47%) 291ms (± 0.96%) +6ms (+ 2.11%) 288ms 295ms p=0.005 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 98ms (± 7.89%) 97ms (± 6.92%) ~ 86ms 103ms p=0.465 n=6
CompilerTSServer - node (v14.21.3, x64)
Req 1 - updateOpen 2,783ms (± 0.22%) 2,946ms (± 0.51%) +163ms (+ 5.87%) 2,931ms 2,974ms p=0.005 n=6
Req 2 - geterr 4,544ms (± 1.49%) 4,521ms (± 0.35%) ~ 4,503ms 4,549ms p=0.936 n=6
Req 3 - references 373ms (± 0.60%) 373ms (± 0.41%) ~ 370ms 374ms p=0.935 n=6
Req 4 - navto 298ms (± 0.73%) 296ms (± 0.28%) ~ 294ms 296ms p=0.096 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 76ms (± 2.88%) 76ms (± 1.86%) ~ 74ms 78ms p=1.000 n=6
xstateTSServer - node (v14.21.3, x64)
Req 1 - updateOpen 3,405ms (± 1.59%) 3,461ms (± 1.14%) ~ 3,386ms 3,505ms p=0.093 n=6
Req 2 - geterr 1,861ms (± 0.76%) 1,850ms (± 0.85%) ~ 1,822ms 1,870ms p=0.297 n=6
Req 3 - references 128ms (± 2.70%) 129ms (± 2.05%) ~ 125ms 133ms p=0.465 n=6
Req 4 - navto 395ms (± 0.98%) 395ms (± 1.21%) ~ 389ms 400ms p=0.746 n=6
Req 5 - completionInfo count 2,862 (± 0.00%) 2,862 (± 0.00%) ~ 2,862 2,862 p=1.000 n=6
Req 5 - completionInfo 419ms (± 2.42%) 427ms (± 1.95%) ~ 415ms 436ms p=0.128 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.21.3, x64)
Scenarios
  • Compiler-UnionsTSServer - node (v18.10.0, x64)
  • Compiler-UnionsTSServer - node (v16.17.1, x64)
  • Compiler-UnionsTSServer - node (v14.21.3, x64)
  • CompilerTSServer - node (v18.10.0, x64)
  • CompilerTSServer - node (v16.17.1, x64)
  • CompilerTSServer - node (v14.21.3, x64)
  • xstateTSServer - node (v18.10.0, x64)
  • xstateTSServer - node (v16.17.1, x64)
  • xstateTSServer - node (v14.21.3, x64)
Benchmark Name Iterations
Current 53877 6
Baseline main 6

Startup

Comparison Report - main..53877
Metric main 53877 Delta Best Worst p-value
tsc-startup - node (v16.17.1, x64)
Execution time 141.78ms (± 0.19%) 142.10ms (± 0.18%) +0.32ms (+ 0.23%) 141.44ms 144.89ms p=0.000 n=600
tsserver-startup - node (v16.17.1, x64)
Execution time 220.22ms (± 0.21%) 221.12ms (± 0.21%) +0.90ms (+ 0.41%) 219.90ms 227.12ms p=0.000 n=600
tsserverlibrary-startup - node (v16.17.1, x64)
Execution time 221.95ms (± 0.23%) 222.34ms (± 0.18%) +0.39ms (+ 0.18%) 221.22ms 226.14ms p=0.000 n=600
typescript-startup - node (v16.17.1, x64)
Execution time 204.20ms (± 0.30%) 204.11ms (± 0.18%) -0.09ms (- 0.05%) 203.18ms 208.50ms p=0.003 n=600
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • tsc-startup - node (v16.17.1, x64)
  • tsserver-startup - node (v16.17.1, x64)
  • tsserverlibrary-startup - node (v16.17.1, x64)
  • typescript-startup - node (v16.17.1, x64)
Benchmark Name Iterations
Current 53877 6
Baseline main 6

Developer Information:

Download Benchmark

@andrewbranch
Copy link
Member

IIRC module resolution is reported as part of parse time in the perf results? If so, looks like a win.

@jakebailey
Copy link
Member

@typescript-bot perf test faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Apr 20, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the abridged perf test suite on this PR at fc8c4da. You can monitor the build here.

Update: The results are in!

@sheetalkamat
Copy link
Member Author

@andrewbranch @jakebailey do we want to report library resolution times in --extendedDiagnostics? like module resolution times and type ref resolution times. I thought it wouldnt be much and hence not needed but open to adding it if you think that would be helpful to track.

@sheetalkamat
Copy link
Member Author

Also for now made the API internal and can make it public if really needed.

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..53877

Metric main 53877 Delta Best Worst p-value
Angular - node (v16.17.1, x64)
Memory used 365,185k (± 0.01%) 365,163k (± 0.00%) ~ 365,148k 365,183k p=0.575 n=6
Parse Time 3.54s (± 0.69%) 3.52s (± 0.61%) ~ 3.50s 3.56s p=0.329 n=6
Bind Time 1.18s (± 0.54%) 1.18s (± 0.54%) ~ 1.17s 1.19s p=1.000 n=6
Check Time 9.52s (± 0.25%) 9.52s (± 0.29%) ~ 9.49s 9.57s p=0.808 n=6
Emit Time 7.93s (± 0.41%) 7.95s (± 0.73%) ~ 7.88s 8.03s p=0.687 n=6
Total Time 22.17s (± 0.25%) 22.17s (± 0.36%) ~ 22.05s 22.28s p=1.000 n=6
Compiler-Unions - node (v16.17.1, x64)
Memory used 192,598k (± 0.04%) 192,655k (± 0.02%) ~ 192,597k 192,704k p=0.128 n=6
Parse Time 1.59s (± 0.79%) 1.60s (± 0.51%) ~ 1.59s 1.61s p=0.065 n=6
Bind Time 0.82s (± 0.99%) 0.83s (± 0.49%) ~ 0.83s 0.84s p=0.056 n=6
Check Time 10.24s (± 0.63%) 10.29s (± 0.73%) ~ 10.19s 10.36s p=0.228 n=6
Emit Time 2.98s (± 0.56%) 3.01s (± 0.93%) ~ 2.95s 3.03s p=0.097 n=6
Total Time 15.64s (± 0.44%) 15.73s (± 0.48%) ~ 15.64s 15.82s p=0.126 n=6
Monaco - node (v16.17.1, x64)
Memory used 345,792k (± 0.01%) 345,775k (± 0.00%) ~ 345,760k 345,803k p=1.000 n=6
Parse Time 2.74s (± 0.60%) 2.74s (± 0.54%) ~ 2.72s 2.76s p=0.870 n=6
Bind Time 1.08s (± 1.17%) 1.09s (± 0.47%) ~ 1.08s 1.09s p=0.418 n=6
Check Time 7.83s (± 0.41%) 7.83s (± 0.40%) ~ 7.80s 7.87s p=0.871 n=6
Emit Time 4.46s (± 0.75%) 4.46s (± 0.76%) ~ 4.41s 4.49s p=0.935 n=6
Total Time 16.11s (± 0.23%) 16.12s (± 0.38%) ~ 16.01s 16.20s p=0.568 n=6
TFS - node (v16.17.1, x64)
Memory used 300,061k (± 0.01%) 300,052k (± 0.00%) ~ 300,040k 300,070k p=0.630 n=6
Parse Time 2.16s (± 0.95%) 2.17s (± 0.86%) ~ 2.16s 2.21s p=0.514 n=6
Bind Time 1.23s (± 0.66%) 1.24s (± 0.66%) ~ 1.23s 1.25s p=0.666 n=6
Check Time 7.22s (± 0.43%) 7.20s (± 0.44%) ~ 7.16s 7.25s p=0.377 n=6
Emit Time 4.35s (± 0.76%) 4.36s (± 0.73%) ~ 4.31s 4.41s p=0.683 n=6
Total Time 14.97s (± 0.43%) 14.97s (± 0.50%) ~ 14.89s 15.09s p=0.809 n=6
material-ui - node (v16.17.1, x64)
Memory used 481,598k (± 0.02%) 481,573k (± 0.00%) ~ 481,552k 481,596k p=0.688 n=6
Parse Time 3.25s (± 0.23%) 3.24s (± 0.32%) ~ 3.23s 3.25s p=0.530 n=6
Bind Time 0.95s (± 0.79%) 0.94s (± 0.43%) -0.01s (- 1.05%) 0.94s 0.95s p=0.029 n=6
Check Time 17.87s (± 0.26%) 17.81s (± 0.42%) ~ 17.71s 17.90s p=0.230 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.07s (± 0.26%) 21.99s (± 0.35%) ~ 21.90s 22.09s p=0.109 n=6
xstate - node (v16.17.1, x64)
Memory used 561,292k (± 0.01%) 560,601k (± 0.03%) -691k (- 0.12%) 560,502k 560,870k p=0.005 n=6
Parse Time 4.04s (± 0.31%) 4.00s (± 0.34%) -0.04s (- 1.07%) 3.98s 4.01s p=0.005 n=6
Bind Time 1.77s (± 0.23%) 1.78s (± 0.59%) ~ 1.76s 1.79s p=0.528 n=6
Check Time 3.03s (± 0.13%) 3.04s (± 0.54%) ~ 3.01s 3.05s p=0.326 n=6
Emit Time 0.09s (± 0.00%) 0.09s (± 0.00%) ~ 0.09s 0.09s p=1.000 n=6
Total Time 8.92s (± 0.11%) 8.90s (± 0.23%) ~ 8.87s 8.93s p=0.068 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • Angular - node (v16.17.1, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Monaco - node (v16.17.1, x64)
  • TFS - node (v16.17.1, x64)
  • material-ui - node (v16.17.1, x64)
  • xstate - node (v16.17.1, x64)
Benchmark Name Iterations
Current 53877 6
Baseline main 6

Developer Information:

Download Benchmark

@typescript-bot typescript-bot added For Milestone Bug PRs that fix a bug with a specific milestone and removed For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Apr 20, 2023
jakebailey
jakebailey approved these changes Apr 20, 2023
View reviewed changes
Copy link
Member

@jakebailey jakebailey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want to report library resolution times in --extendedDiagnostics? like module resolution times and type ref resolution times. I thought it wouldnt be much and hence not needed but open to adding it if you think that would be helpful to track.

If it's easy, then I feel like it can't hurt, but, it might not be worth it, yeah.

@sheetalkamat sheetalkamat merged commit ddd5084 into main Apr 20, 2023
@sheetalkamat sheetalkamat deleted the libResolution branch April 20, 2023 20:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andrewbranch andrewbranch andrewbranch approved these changes

@jakebailey jakebailey jakebailey approved these changes

Assignees

@sheetalkamat sheetalkamat

Labels
Author: Team For Milestone Bug PRs that fix a bug with a specific milestone
Projects
None yet
Milestone
No milestone
4 participants
@sheetalkamat @andrewbranch @typescript-bot @jakebailey