auto Pkg manager fails when using JULIA_DEPOT_PATH different from default ".julia"

[ablaom]

In a completely new Julia Depot I add Pluto.jl to the default env, do using Pluto; Pluto.run() and in a new notebook try using Example. This leads to error message below. Julia version = 1.9.0 but Julia 1.8.5 has the same issue. I've tried some earlier patch releases of Pluto within 0.19.* but get same issue.

ArgumentError: Package Example [7876af07-990d-54b4-ab0e-23690620f79a] is required but does not seem to be installed:
- Run `Pkg.instantiate()` to install all recorded dependencies.
_require(::Base.PkgId, ::String)@loading.jl:1739
_require_prelocked(::Base.PkgId, ::String)@loading.jl:1625
macro [email protected]:1613[inlined]
macro [email protected]:267[inlined]
require(::Module, ::Symbol)@loading.jl:1576
top-level scope@[Local: 1](http://localhost:1234/edit?id=4d4f8646-0fba-11ee-1c9c-873640933d40#)

pluto_fail.mov

The pluto script, after fail

julia> versioninfo()
Julia Version 1.9.0
Commit 8e630552924 (2023-05-07 11:25 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin22.4.0)
  CPU: 12 × Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.6 (ORCJIT, skylake)
  Threads: 5 on 12 virtual cores
Environment:
  JULIA_LTS_PATH = /Applications/Julia-1.6.app/Contents/Resources/julia/bin/julia
  JULIA_PATH = /Applications/Julia-1.9.app/Contents/Resources/julia/bin/julia
  JULIA_DEPOT_PATH = temporary_julia_depot
  JULIA_EGLOT_PATH = /Applications/Julia-1.6.app/Contents/Resources/julia/bin/julia
  JULIA_NUM_THREADS = 5
  DYLD_LIBRARY_PATH = /usr/local/homebrew/Cellar/libomp/9.0.1/lib/
  JULIA_NIGHTLY_PATH = /Applications/Julia-1.9.app/Contents/Resources/julia/bin/julia

[ablaom]

Sorry in advance for nagging. Been a little while since I posted this. Any progress? This issue means I need to hack my Julia workshop prep. Giving my course one year later and this issue is still open.

My use case is this: When running a workshop, I want to use a fresh DEPOT to more realistically simulate what Julia beginners will encounter during the workshop (and make testing of my workshop more reliable).

I realise we are all very busy, but it would be good to have some idea of where this might go, if anywhere.

[fonsp]

Hey @ablaom, thanks for the reminder! I'll take a look

[fonsp]

Weird, it looks like running import Example in the notebook does the instantiate and precompile correctly (files are added in temporary_julia_depot/packages and temporary_julia_depot/compiled), but the package does not load.

Running DEPOT_PATH in the notebook process gives temporary_julia_depot, that's correct.

Running Pkg.instantiate() in the notebook gave this terminal output

Installing known registries into `yoloremoveme`
   Installed Preferences ───────── v1.4.3
   Installed FixedPointNumbers ─── v0.8.5
   Installed TensorCore ────────── v0.1.1
   Installed Parsers ───────────── v2.8.1
   Installed PlotlyBase ────────── v0.8.19
   Installed JSON ──────────────── v0.21.4
   Installed ColorTypes ────────── v0.11.5
   Installed ColorVectorSpace ──── v0.10.0
   Installed PrecompileTools ───── v1.2.1
   Installed OrderedCollections ── v1.6.3
   Installed Example ───────────── v0.5.3
   Installed LaTeXStrings ──────── v1.3.1
   Installed Reexport ──────────── v1.2.2
   Installed DelimitedFiles ────── v1.9.1
   Installed Requires ──────────── v1.3.0
   Installed Parameters ────────── v0.12.3
   Installed Colors ────────────── v0.12.11
   Installed UnPack ────────────── v1.0.2
   Installed DocStringExtensions ─ v0.9.3
   Installed ColorSchemes ──────── v3.25.0
Precompiling project...
  ✓ Example
  ✓ LaTeXStrings
  ✓ TensorCore
  ✓ CompilerSupportLibraries_jll
  ✓ OrderedCollections
  ✓ Reexport
  ✓ Zlib_jll
  ✓ DelimitedFiles
  ✓ Requires
  ✓ SuiteSparse_jll
  ✓ UnPack
  ✓ DocStringExtensions
  ✓ Preferences
  ✓ PrecompileTools
  ✓ Parameters
  ✓ SparseArrays
  ✓ Statistics
  ✓ FixedPointNumbers
  ✓ ColorTypes
  ✓ ColorVectorSpace
  ✓ Colors
  ✓ Parsers
  ✓ ColorSchemes
  ✓ JSON
  ✓ PlotlyBase
  25 dependencies successfully precompiled in 16 seconds
  2 dependencies precompiled but different versions are currently loaded. Restart julia to access the new versions

Which says that it's installing registries... which means that it's a different folder.

[fonsp]

Aha!

The problem is that the depot is given as a relative path, and the notebook process resolves it differently from the terminal, because the notebook is in a different folder.

This is solved by using an absolute path for the depot:

export JULIA_DEPOT_PATH="/Users/fons/temporary_julia_depot"

And indeed this seems to work. Use that for your next workshop!

[fonsp]

I made f135748 to prevent this in the future.

Thanks for the nice issue report with a video!

[ablaom]

Ahh. That's a little embarrassing. Thanks for sorting this out for me.