Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compatibility with Julia 1.3 #13

Open
wants to merge 38 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
9e7adcd
Update legacy functions.
henry2004y Feb 11, 2020
4a5f456
Fix typo
henry2004y Feb 11, 2020
a7a3057
First modification, not working yet.
henry2004y Feb 11, 2020
662f260
Got stuck with "clnttcp_create: RPC: Remote system error - Connection…
henry2004y Feb 11, 2020
b39255b
Update legacy functions.
henry2004y Feb 13, 2020
0b49f86
Interface in Julia 1.3
henry2004y Feb 14, 2020
4892e97
Format adjustment.
henry2004y Feb 14, 2020
d401d3c
Commented out line continuation part for testing.
henry2004y Feb 14, 2020
da199c9
Use __init__() function for initializing the package.
henry2004y Feb 14, 2020
af7f327
Added Project.toml file.
henry2004y Feb 14, 2020
33a5d9b
Added Manifest.toml file.
henry2004y Feb 14, 2020
fd7926e
First working test.
henry2004y Feb 14, 2020
a2b158a
Add Test as extra dependency.
henry2004y Feb 14, 2020
c8b3b94
Updated.
henry2004y Feb 14, 2020
0eb33f7
Simplify the line continuation part but won't work if [;\$] inside qu…
henry2004y Feb 15, 2020
8781e48
syntax improvements.
henry2004y Feb 15, 2020
4b1823a
Add more tests.
henry2004y Feb 15, 2020
ce304bf
Updated.
henry2004y Feb 15, 2020
7bda91c
isbits->isbitstype
henry2004y Feb 15, 2020
2841eea
First working version in Julia 1.3
henry2004y Feb 15, 2020
cc41578
Indentation adjustment.
henry2004y Feb 16, 2020
d4dbeb8
Add REPL as dependency.
henry2004y Feb 16, 2020
e94b93a
Moved the init function one level up.
henry2004y Feb 16, 2020
0accb4e
Focus on RPC for now; correctly handle IDL directories; move __init__…
henry2004y Feb 16, 2020
4bcdb81
Updated.
henry2004y Feb 16, 2020
1ce8e37
Rewind to 4 whitespaces.
henry2004y Feb 16, 2020
accd788
Fix typo.
henry2004y Feb 16, 2020
8a83e2a
Add Libdl to dependency.
henry2004y Feb 16, 2020
77f4f5c
Rename init function.
henry2004y Feb 16, 2020
77d9709
Rename init function, called from one level up.
henry2004y Feb 16, 2020
a3d1f44
Added info about convenient functions.
henry2004y Feb 16, 2020
d3b0503
Rename IDLCall.jl -> IDL.jl; better __init__() function.
henry2004y Feb 16, 2020
bcec124
Polish the tests. Currently only work if you comment out init_repl() …
henry2004y Feb 16, 2020
804474f
Fix active_repl not found for testing.
henry2004y Feb 17, 2020
ac204df
:IDL_Init -> :IDL_Initialize
henry2004y Feb 17, 2020
34597ff
Update comments.
henry2004y Feb 17, 2020
5d49831
Reset IDL RPC during initialization.
henry2004y Feb 17, 2020
835ad0d
Mentioned callable issue on Mac.
henry2004y Feb 17, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions Manifest.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# This file is machine-generated - editing it directly is not advised

[[Base64]]
uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"

[[InteractiveUtils]]
deps = ["Markdown"]
uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240"

[[Libdl]]
uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb"

[[Markdown]]
deps = ["Base64"]
uuid = "d6f4376e-aef5-505a-96c1-9c027394607a"

[[REPL]]
deps = ["InteractiveUtils", "Markdown", "Sockets"]
uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb"

[[Sockets]]
uuid = "6462fe0b-24de-5631-8697-dd941f90decc"
14 changes: 14 additions & 0 deletions Project.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
name = "IDL"
uuid = "d8fbff9c-ffdd-4f12-b08d-aa41cc43ce74"
authors = ["Bob Portmann", "Hongyang Zhou <[email protected]>"]
version = "0.1.0"

[deps]
Libdl = "8f399da3-3557-5675-b5ff-fb832c97cbdb"
REPL = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb"

[extras]
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

[targets]
test = ["Test"]
80 changes: 40 additions & 40 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,77 +1,77 @@
# IDL interface for the Julia language

[![Build Status](https://travis-ci.org/BobPortmann/IDLCall.jl.svg?branch=master)](https://travis-ci.org/BobPortmann/IDLCall.jl)
[![Build Status](https://travis-ci.org/BobPortmann/`IDL.jl`.jl.svg?branch=master)](https://travis-ci.org/BobPortmann/`IDL.jl`.jl)

IDLCall is an interface to call IDL from the Julia language. Note that you must have a valid IDL
license to use IDL from julia.
The `IDL.jl` package is an interface for calling IDL from the Julia language. You must have a valid IDL license to use IDL from Julia.

## Installation

Within Julia, use the package manager:
```julia
Pkg.clone("https://github.com/BobPortmann/IDLCall.jl.git")
Pkg.clone("https://github.com/BobPortmann/IDL.jl.git")
```

IDLCall should find and load the IDL library automatically on Mac and Linux. It has not been
tested on Windows so please file an issue if you use Windows and want to help make it work.
`IDL.jl` should find and load the IDL library automatically on Mac and Linux. It has not been tested on Windows so please file an issue if you use Windows and want to help make it work.

IDL can be called using either the `RPC` or `Callable` interface. On windows only the `Callable`
interface is available. You can set an environmental variable `JL_IDL_TYPE` to `RPC` or `CALLABLE`
to force the use of that interface.
Alternatively you can set `ENV["JL_IDL_TYPE]` within julia before starting IDLCall.
Note that by default IDLCall uses the `RPC` interface
on Mac and Linux and `Callable` on Windows. The biggest difference between these is that:
IDL can be called using either the `RPC` or `Callable` interface. On windows only the `Callable` interface is available. You can set an environmental variable `JL_IDL_TYPE` to `RPC` or `CALLABLE` to force the use of that interface.
Alternatively you can set `ENV["JL_IDL_TYPE"]` within julia before starting `IDL.jl`.
Note that by default `IDL.jl` uses the `RPC` interface on Mac and Linux and `Callable` on Windows. The biggest difference between these is that:

- `Callable` IDL runs in one program space and thus arrays can be shared between julia and IDL.
In `RPC` all arrays are copied between processes. Note that I have run into issues with IDL
loading DLM's while using `Callable` (e.g., NetCDF).

- IDL `RPC` is not supported on windows
- IDL `RPC` is not supported on windows.

- `Callable` is always managed by IDLCall while `RPC` can be managed by IDLCall or the user.
By managed we mean that it is opened it when you load IDLCall and closed it when you close julia.
To manage `RPC` yourself run `idlrpc` in a shell before starting IDLCall. This allows the `idlrpc`
- `Callable` is always managed by `IDL.jl` while `RPC` can be managed by `IDL.jl` or the user.
By managed we mean that it is opened it when you load `IDL.jl` and closed it when you close julia.
To manage `RPC` yourself run `idlrpc` in a shell before starting `IDL.jl`. This allows the `idlrpc`
session to persist and julia can be restarted without killing the `idlrpc` process.
Currently `RPC` is always reset after the initialization of `IDL.jl`.

## Quickstart
There is an [issue](https://github.com/JuliaLang/julia/issues/7004) of opening `libidl` on Mac.

I recommend you start your code with
## Quickstart

```julia
import IDLCall
idl = IDLCall
```
Then you can add a julia variable to the IDL process with

```
idl.put_var(x, "x")
using IDL
```

and you can retrieve variable into julia using

You can add a Julia variable to the IDL process with
```julia
x = 1
put_var(x, "x")
```
x = idl.get_var("x")
and you can retrieve variable into Julia using
```julia
x = get_var("x")
```

You can run an arbitrary chunk of code in IDL using
```julia
idl.execute("any valid idl code")
```
Note that only primitive data types are supported at this time (e.g., structure variables are not supported yet). Also, `[;|\$]` inside quotes won't be correctly recognized.

Many convenient functions are provided:
```
idl.execute("any valid idl code")
IDL.help
IDL.idlhelp
IDL.shell_command
IDL.reset
IDL.full_reset
IDL.dotrun
```
Note that only primitive data types are supported at this time (e.g., structure variables
are not supported yet).

See more examples in the [test script](test/runtests.jl)

## REPL

You can drop into an IDL REPL by typing `>` at the julia prompt. Then you can type any valid
IDL commands, including using continuation characters `$` for multi-line commands. One
experimental feature I have added is the use of `%var` will auto-magically import the julia
variable `var` into the IDL process. This works at the IDL prompt or in strings passed into the
`execute` function.
You can drop into an IDL REPL by typing `>` at the Julia prompt. Then you can type any valid IDL commands, including using continuation characters `$` for multi-line commands. One experimental feature I have added is the use of `%var` will auto-magically import the Julia variable `var` into the IDL process. This works at the IDL prompt or in strings passed into the `execute` function.

## ToDo
## Note

- Add tests
The IDL `RPC` library won't be automatically reset if you exit Julia. Therefore, currently you need to type `IDL.reset()` to cleanup the library.

## ToDo

- Make more flexible to install on all platforms

Expand Down
44 changes: 44 additions & 0 deletions src/IDL.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
module IDL

export get_var, put_var, @get_var, @put_var, execute

# Find IDL library directory if on Linux
if Sys.isunix()
idl_exec = chomp(read(`which idl`,String))
if islink(idl_exec)
idl_dir = dirname(readlink(idl_exec))
else
idl_dir = dirname(idl_exec)
end
idl_lib_dir = joinpath(idl_dir,"bin.darwin.x86_64")
const libidl_rpc = joinpath(idl_lib_dir,"libidl_rpc.dylib")
const idlrpc = joinpath(idl_dir,"idlrpc")
const idlcall = joinpath(idl_lib_dir,"libidl.dylib")
else # Windows
const idlcall = "libidl"
const idlrpc = "libidl_rpc"
end


jl_idl_type = get(ENV, "JL_IDL_TYPE", Sys.iswindows() ? "CALLABLE" : "RPC")

jl_idl_type == "RPC" ? include("IDLRPC.jl") :
jl_idl_type == "CALLABLE" ? include("IDLCallable.jl") :
Sys.iswindows() ? error("JL_IDL_TYPE must be CALLABLE on windows") :
error("JL_IDL_TYPE must be RPC or CALLABLE")

include("IDLREPL.jl")

function __init__()
if jl_idl_type == "RPC"
rpc_init()
full_reset() # Reset IDL
elseif jl_idl_type == "CALLABLE"
callable_init() # not yet working for MaxOS and Linux
end

# Initializing REPL
idl_repl()
end

end
32 changes: 0 additions & 32 deletions src/IDLCall.jl

This file was deleted.

35 changes: 17 additions & 18 deletions src/IDLCallable.jl
Original file line number Diff line number Diff line change
@@ -1,22 +1,21 @@

include("idl_types.jl")
include("common-funcs.jl")
include("common-macros.jl")

if is_apple()
using Libdl

if Sys.isapple()
cd(idl_lib_dir) do
Libdl.dlopen("libidl")
end
end

function init()
ecode = ccall((:IDL_Init, idlcall), Cint, (Cint, Ptr{Cint}, Ptr{Ptr{UInt8}}),
0, C_NULL, C_NULL)
if ecode == 0
error("IDL.init: IDL init failed")
end
function callable_init()
ecode = ccall((:IDL_Initialize, idlcall), Cint,
(Cint, Ptr{Cint}, Ptr{Ptr{UInt8}}), 0, C_NULL, C_NULL)
ecode == 0 && error("IDL.init: IDL init failed")
global output_cb
ccall((:IDL_ToutPush, idlcall), Void, (Ptr{Void},), output_cb)
ccall((:IDL_ToutPush, idlcall), Nothing, (Ptr{Nothing},), output_cb)
end

# function execute{T<:AbstractString}(strarr::Array{T,1})
Expand Down Expand Up @@ -49,7 +48,7 @@ function get_output(flags::Cint, buf::Ptr{UInt8}, n::Cint)
return
end

output_cb = cfunction(get_output, Void, (Cint, Ptr{UInt8},Cint))
output_cb = @cfunction(get_output, Nothing, (Cint, Ptr{UInt8},Cint))

# function exit()
# # probably better to do a .full_reset instead
Expand All @@ -68,17 +67,17 @@ function done_with_var(p::Ptr{UInt8})
return
end

free_cb = cfunction(done_with_var, Void, (Ptr{UInt8},))
free_cb = @cfunction(done_with_var, Nothing, (Ptr{UInt8},))

function put_var{T,N}(arr::Array{T,N}, name::AbstractString)
function put_var(arr::Array{T,N}, name::AbstractString) where {T,N}
if !isbits(eltype(arr)) || (idl_type(arr) < 0)
error("IDL.put_var: only works with some vars containing bits types")
end
dim = zeros(Int, IDL_MAX_ARRAY_DIM)
dim[1:N] = [size(arr)...]
vptr = ccall((:IDL_ImportNamedArray, idlcall), Ptr{IDL_Variable},
(Ptr{UInt8}, Cint, IDL_ARRAY_DIM, Cint, Ptr{UInt8}, IDL_ARRAY_FREE_CB , Ptr{Void}),
name, N, dim, idl_type(arr), pointer(arr), free_cb, C_NULL)
(Ptr{UInt8}, Cint, IDL_ARRAY_DIM, Cint, Ptr{UInt8}, IDL_ARRAY_FREE_CB , Ptr{Nothing}),
name, N, dim, idl_type(arr), pointer(arr), free_cb, C_NULL)
if vptr == C_NULL
error("IDL.put_var: failed")
end
Expand All @@ -89,15 +88,15 @@ end
function put_var(x, name::AbstractString)
# Sort of a HACK: import as one-element array and then truncate to scalar
# IDL_ImportArray(int n_dim, IDL_MEMINT dim[], int type,
# UCHAR *data, IDL_ARRAY_FREE_CB free_cb, void *s)
# UCHAR *data, IDL_ARRAY_FREE_CB free_cb, Nothing *s)
if !isbits(x) || (idl_type(x) < 0)
error("IDL.put_var: only works with some vars containing bits types")
end
dim = zeros(Int, IDL_MAX_ARRAY_DIM)
dim[1] = 1
ccall((:IDL_ImportNamedArray, idlcall), Ptr{Void},
(Ptr{UInt8}, Cint, Ptr{IDL_MEMINT}, Cint, Ptr{UInt8}, Ptr{Void}, Ptr{Void}),
name, 1, dim, idl_type(x), pointer([x]), C_NULL, C_NULL)
ccall((:IDL_ImportNamedArray, idlcall), Ptr{Nothing},
(Ptr{UInt8}, Cint, Ptr{IDL_MEMINT}, Cint, Ptr{UInt8}, Ptr{Nothing}, Ptr{Nothing}),
name, 1, dim, idl_type(x), pointer([x]), C_NULL, C_NULL)
execute("$name = $name[0]")
return
end
Expand Down
51 changes: 26 additions & 25 deletions src/IDLREPL.jl
Original file line number Diff line number Diff line change
@@ -1,15 +1,16 @@
import Base: LineEdit, REPL
import REPL:respond, LineEdit, mode_keymap

function idl_repl()

# Setup idl prompt
prompt = LineEdit.Prompt("IDL> ";
prompt_prefix=Base.text_colors[:blue],
prompt_suffix=Base.text_colors[:white])

prompt_prefix=Base.text_colors[:blue],
prompt_suffix=Base.text_colors[:white])

!isdefined(Base, :active_repl) && return

repl = Base.active_repl

prompt.on_done = REPL.respond(repl,prompt) do line
prompt.on_done = respond(repl,prompt) do line
ok2, line, msg = convert_continuations(line)
if !ok2
println(msg)
Expand All @@ -36,38 +37,38 @@ function idl_repl()
repl.interface.modes[i_mode] = prompt
end


hp = main_mode.hist
hp.mode_mapping[:idl] = prompt
prompt.hist = hp
const idl_keymap = Dict{Any,Any}(
'>' => function (s,args...)
if isempty(s)
if !haskey(s.mode_state,prompt)
s.mode_state[prompt] = LineEdit.init_state(repl.t,prompt)
end
LineEdit.transition(s,prompt)
else
LineEdit.edit_insert(s,'>')
end
end)

idl_keymap = Dict{Any,Any}(
'>' => function (s,args...)
if isempty(s)
if !haskey(s.mode_state,prompt)
s.mode_state[prompt] = LineEdit.init_state(repl.t,prompt)
end
LineEdit.transition(s,prompt)
else
LineEdit.edit_insert(s,'>')
end
end)

search_prompt, skeymap = LineEdit.setup_search_keymap(hp)
mk = REPL.mode_keymap(main_mode)
mk = mode_keymap(main_mode)

b = Dict{Any,Any}[skeymap, mk, LineEdit.history_keymap, LineEdit.default_keymap,
LineEdit.escape_defaults]
prompt.keymap_dict = LineEdit.keymap(b)
b = Dict{Any,Any}[skeymap, mk, LineEdit.history_keymap,
LineEdit.default_keymap, LineEdit.escape_defaults]
prompt.keymap_dict = LineEdit.keymap(b)

main_mode.keymap_dict = LineEdit.keymap_merge(main_mode.keymap_dict, idl_keymap);
main_mode.keymap_dict = LineEdit.keymap_merge(main_mode.keymap_dict,
idl_keymap)
nothing
end

function find_prompt_in_modes(modes, name)
j = -1
for (i,mode) in enumerate(modes)
if :prompt in fieldnames(mode) && mode.prompt == name
if :prompt in fieldnames(typeof(mode)) && mode.prompt == name
j = i
break
end
Expand Down
Loading