RFC: Refactor of random.jl, and stick into modules.

[ViralBShah]

I have started refactoring random.jl, so that the librandom interfaces are in a separate module. I also want to add the capability of having multiple RNGs with different states, and generally rework the entire RNG interface.

The first bit of work is to simply keep the existing interfaces, but inside an RNG module, and refactor the librandom functions into a LibRandom module.

This pull request will serve to evolve the new RNG interfaces, which have been discussed in scattered places before.

Cc: @johnmyleswhite @dmbates

[ViralBShah]

I currently get error during bootstrap: librandom not defined. I can't seem to figure out what I am missing.

[StefanKarpinski]

You have to define librandom in Base at the top of the file like you did before.

[ViralBShah]

I am doing everything that is done for FFTW, but I continue to get:

$ make
    JULIA usr/lib/julia/sys.ji
Warning: replacing module Base
librandom not defined
 in include at boot.jl:243
 in load at util.jl:230
 in load at util.jl:242
 in process_options at client.jl:175
 in _start at client.jl:227
at RNG.jl:98
 in load at util.jl:230
 in load at util.jl:242
 in process_options at client.jl:175
 in _start at client.jl:227
at /Users/viral/julia/base/sysimg.jl:115
 in load at util.jl:253
 in process_options at client.jl:175
 in _start at client.jl:227
make[1]: *** [/Users/viral/julia/usr/lib/julia/sys.ji] Error 1
make: *** [release] Error 2

[ViralBShah]

@JeffBezanson Do the library names in start_image.jl have to continue to live in Base, or can we move them into the modules? For example, no user code except RNG.jl should really need access to the librandom dynamic library handle, and the contents of the LibRandom module. Only DSP requires access to the FFTW module and the libfftw handle.

Is it possible to hide all of this inside the modules, and not even expose in Base? I don't know what the impact would be on creation of sys.ji.

[ViralBShah]

I can't quite figure out how to reproduce it, but after adding librandom to RNG.jl it builds sys.ji fine sometimes, but most of the times, while building I get:

 $ make
    JULIA usr/lib/julia/sys.ji
error during init:
could not show value of type LoadError
make[1]: *** [/Users/viral/julia/usr/lib/julia/sys.ji] Error 1
make: *** [release] Error 2

[StefanKarpinski]

I've spent a bunch of time trying to debug this, but I can't figure out what's messed up. You've changed a lot of stuff here, which ends up being impossible to debug when bootstrapping because the process is so brittle and tricky to debug. I suggest going back and making incremental changes, each of which preserves the system as a working whole.

Also, since we consistently name files in lowercase, it would be nice to stick to that until such a time as we decide to capitalize file names and do so across the board.

[JeffBezanson]

start_image and the current way of using libraries is going to go away.

[StefanKarpinski]

That will be great but until then, this can still be refactored.

[ViralBShah]

Thank GOD.

-viral

On 27-Aug-2012, at 11:37 PM, Jeff Bezanson [email protected] wrote:

start_image and the current way of using libraries is going to go away.

—
Reply to this email directly or view it on GitHub.

[ViralBShah]

When is start_image going away? I'd rather just wait till then.

[ViralBShah]

The changes in here are really straightforward. I factored out the librandom calls into a separate module, and put the rest of the rng stuff in its own module. Then all that remains is the dlopen stuff. Two possible troublesome things related to bootstrapping are the compile_hints and initialization in client.jl.

Can we clearly separate such modules out of the bootstrapping process?

[StefanKarpinski]

That's a lot of stuff to change during bootstrap. Every time I've made a bunch of changes to bootstrap code, I end up doing binary search on the changes to find out what the hell broke. It might be possible to entirely extract RNG stuff from base, honestly. I'm not sure where in Base you would need random numbers.

[ViralBShah]

I just left it there just because it was there before. We should move as many libraries out of Base as possible. There is no real reason for them to be in Base.

One thing that RNG needs is initialisation on loading of the module. Is that possible currently?

Of course, I would be annoyed if I had to do import RNG or some such thing every time I start julia. That is just a case of loading a bunch of modules apart from Base, by default.

-viral

On 28-Aug-2012, at 8:59 AM, Stefan Karpinski wrote:

That's a lot of stuff to change during bootstrap. Every time I've made a bunch of changes to bootstrap code, I end up doing binary search on the changes to find out what the hell broke. It might be possible to entirely extract RNG stuff from base, honestly. I'm not sure where in Base you would need random numbers.

—
Reply to this email directly or view it on GitHub.