svd slowdown on master

[ViralBShah]

svd has slowed down on master compared to 1.4.2. This is using OPENBLAS_NUM_THREADS=16.

viralbshah@antarctic:~$ ./julia-1.4.2/bin/julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.4.2 (2020-05-23)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> using LinearAlgebra

julia> a = randn(4000,4000);

julia> @time svd(a);
 15.395474 seconds (229.83 k allocations: 745.186 MiB, 1.15% gc time)

julia> @time svd(a);
 14.116012 seconds (14 allocations: 732.911 MiB, 1.22% gc time)

viralbshah@antarctic:~$ ./julia/julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.6.0-DEV.129 (2020-05-29)
 _/ |\__'_|_|_|\__'_|  |  Commit 38a373af12 (0 days old master)
|__/                   |

julia> a = randn(4000,4000);

julia> using LinearAlgebra

julia> @time svd(a);
 16.817350 seconds (235.57 k allocations: 745.553 MiB, 0.64% gc time)

julia> @time svd(a);
 16.545849 seconds (13 allocations: 732.911 MiB, 1.42% gc time)

julia> versioninfo()
Julia Version 1.6.0-DEV.129
Commit 38a373af12 (2020-05-29 17:04 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: Intel(R) Xeon(R) Silver 4114 CPU @ 2.20GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, skylake)

cc @haampie

[ViralBShah]

MKL on the same machine is able to use 20 cores and compute in 10 seconds. OpenBLAS slows down if I add more cores than 16.

[andreasnoack]

Not sure why we are seeing that slowdown. The MKL implementation can't be compared to the one from OpenBLAS. The latter is "just" vanilla LAPACK but the MKL folks have optimized the bidiagonal reduction and parallelized the solver.

[ViralBShah]

I meant from a user perspective, it is faster. So we lose out in comparisons with anyone using Python+MKL or any system using MKL.

[ViralBShah]

Related discussion: JuliaParallel/Elemental.jl#66 (comment)

[ViralBShah]

We now have MKL.jl and a better solution to swap in a faster BLAS would be through #27.

[ViralBShah]

FWIW, svd does appear to be slightly slower on Julia 1.6 than 1.4, but not much we can do about that in here.