Fix issue with __init__ when built in sysimg

[sjkelly]

see: JuliaLang/PackageCompiler.jl#896

[maleadt]

Hmm, that's not great, because it will mean that if you build multiple sysimages with different GPUCompiler.jl versions, the compiled artifacts will alias. In the case of PackageCompiler.jl apps, I assume the depot will be unique to the package, with with regular sysimages this can definitely happen.

@KristofferC Is there a way to query the version of a package (i.e. Base.pkgversion) that's been built in the system image?

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 72.74%. Comparing base (ee9077d) to head (e1324f5).
Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #589      +/-   ##
==========================================
- Coverage   74.75%   72.74%   -2.02%     
==========================================
  Files          24       24              
  Lines        3414     3386      -28     
==========================================
- Hits         2552     2463      -89     
- Misses        862      923      +61     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[sjkelly]

How about we move the pkgversion call outside of the __init__ so it is compiled-in.

[KristofferC]

Is there a way to query the version of a package (i.e. Base.pkgversion) that's been built in the system image?

pkgversion just seem badly implemented... The information should be there:

julia> [(k.name => v.version) for (k,v) in Base.pkgorigins]
37-element Vector{Pair{String}}:
   "LibSSH2_jll" => v"1.11.0+1"
  "OpenBLAS_jll" => v"0.3.23+4"
   "nghttp2_jll" => v"1.52.0+1"
 "Serialization" => nothing
   "LibGit2_jll" => v"1.6.4+0"
...

[maleadt]

How about we move the pkgversion call outside of the __init__ so it is compiled-in.

That sounds reasonable, but doing during precompilation sounds like there may be corner cases too. I'll have a closer look tomorrow, maybe we should just look in Base.pkgorigins as Kristoffer shows above until pkgversion is fixed in Base.

[maleadt]

I cannot reproduce the underlying issue in isolation.

❯ jl --project
(test) pkg> add Example PackageCompiler
julia> using Example, PackageCompiler
julia> PackageCompiler.create_sysimage(["Example"]; sysimage_path="example-sysimage.so")

❯ jl --project -e 'using Example; @show pkgversion(Example)'
pkgversion(Example) = v"0.5.3"

❯ jl --sysimage=example-sysimage.so --project -e 'using Example; @show pkgversion(Example)'
pkgversion(Example) = v"0.5.3"

Or even using GPUCompiler:

julia> PackageCompiler.create_sysimage(["GPUCompiler"]; sysimage_path="gpucompiler-sysimage.so")

❯ jl --sysimage=gpucompiler-sysimage.so -e 'using GPUCompiler; @show GPUCompiler.compile_cache'
GPUCompiler.compile_cache = "/home/tim/.julia/scratchspaces/61eb1bfa-7361-4325-ad38-22787b887f55/compiled/v1.10/v0.26"

[KristofferC]

How about if you delete the .julia/packages/GPUCompiler folder?

[maleadt]

Still works fine:

❯ jl --sysimage=gpucompiler-sysimage.so --project
julia> using GPUCompiler

julia> GPUCompiler.compile_cache
"/tmp/test/depot/scratchspaces/61eb1bfa-7361-4325-ad38-22787b887f55/compiled/v1.10/v0.26"

julia> pathof(GPUCompiler)
"/home/tim/.julia/packages/GPUCompiler/nWT2N/src/GPUCompiler.jl"

julia> isdir(ans)
false

[KristofferC]

Okay but how. Looking at

https://github.com/JuliaLang/julia/blob/8791d54d142cb27690e2e7d74b1e01733f7d9a3a/base/loading.jl#L560

and

https://github.com/JuliaLang/julia/blob/8791d54d142cb27690e2e7d74b1e01733f7d9a3a/base/loading.jl#L529

if the project file doesn't exist it seems it should return nothing, or?

Why does calling pkgversion outside of __init__ not work? It should give the same version as that in __init__ or what am I misunderstanding? How can the version change between precompilation time and runtime?

[maleadt]

Why does calling pkgversion outside of __init__ not work? It should give the same version as that in __init__ or what am I misunderstanding? How can the version change between precompilation time and runtime?

It shouldn't. I'm wasn't suggesting the proposed workaround of putting the call to Base.pkgversion outside of __init__ is wrong or anything, I just failed to observe the failure with Base.pkgversion in combination with packages embedded in a system image in the first place.

I was just trying to validate the 'workaround', but if @sjkelly has observed that this fixes his issue, this is fine for me.