cache dlsym lookup

[vtjnash]

I didn't do a test for whether this actually makes a significant difference in speed, but I wanted to see whether and how this would work (I'm not certain if it is actually better)

(note: there should probably be a type assert that func is actually a Symbol, otherwise we cache the result of a dynamic variable)

[vtjnash]

this fixes a separate issue where Mac seems to always return a relative path that should be prefixed by /System/Library/Frameworks

[stevengj]

@vtjnash, sorry I didn't notice this earlier. How much of a performance difference does this make? e.g. on a simple benchmark like @time ccall(pyfunc(:Py_IsInitialized), Int32, ())?

[stevengj]

Hmm, looks like it makes a big difference:

julia> @time for i = 1:10^6; ccall(pyfunc(:Py_IsInitialized), Int32, ()); end
elapsed time: 0.373013973236084 seconds

julia> f = pyfunc(:Py_IsInitialized)
Ptr{Void} @0x00007fe7edb60e60
julia> @time for i = 1:10^6; ccall(f::Ptr{Void}, Int32, ()); end
elapsed time: 0.0026140213012695312 seconds

A factor of 100 may be worth caching for...

[vtjnash]

i figured. a dictionary lookup is not as fast as a constant reference. (as a side note, the type assertion on the first argument of the ccall is unnecessary -- it should generate nearly the same code, but accept any Ptr{T}})

i tried merging this forward, but it was tricky since this doesn't check that the symbol it receives is actually constant and just blindly caches the value regardless which can cause some surprises

[stevengj]

In all the places where the argument to pyfunc is not a constant, it should be possible to rearrange the code slightly to get a constant expression. This has to be done manually, of course; I don't think any macro magic could go back up the call tree of pyisinstance, for example, but it is straightforward to change pyisinstance(o, :Foo) to pyisinstance(o, @pyfunc :Foo) and make pyisinstance accept a PyPtr (= Ptr{Void}) argument.

[vtjnash]

that sounds even better. i wasn't certain if it needed to be a runtime value somewhere, so i was trying to just propagate the macro calls outward (almost as a primitive form of inlining)

[stevengj]

Your type annotation global $z::Ptr{Void} doesn't seem to do anything. It seems like you have to attach the annotation $z::Ptr{Void} to every usage of $z. See also my comment at the end of Julia issue #964.

[vtjnash]

were you able to characterize the performance impact of this approach in actual code?

[stevengj]

No, I didn't try anything except for the synthetic benchmark above. (I suspect that in most realistic circumstances the overhead of the dlsym is negligible compared to the Python overhead, but it is a clean enough change that I figured it doesn't hurt.)