Erjang Projects

Here is a list of relevant projects that you can do to help Erjang moving forward.

Setup continuous integration server for Erjang.

We need a hosted service somewhere that will check out new versions of Erjang from GitHub, build them, run various tests, and report any errors. We need to be able to run several different kinds of tests:

• JUnit tests
• eunit tests
• QuickCheck tests based on Triq (http://github.com/krestenkrab/triq)
• Tests based on ERJANG/test_server

The last one is a light weight clone of the test framework that comes with OTP; which we will use for the time being. The original OTP test server dependes depends on many internal BEAM-specific features that it will be a while before we can start running with it.

Based on the above, there should ideally be some kind of reporting, online html access to test results, coverage reports, etc.

At present, there is no coherent structure in how to run the various kinds of tests.

Complete bit string support

The bit string support in Erjang is not complete; but complete enough to run a whole lot of Erlang code.

• Typing <<1,2,3>>. in the Eshell will expose that we’re missing translation for BEAM instruction bs_init_writable. This instruction needs to be handled all the way through the compiler and runtime system.
• Inspecting EBinMatchState and EBitStringBuilder (the run time modules for binary matching/construction) looking for throws new NotImplemented() you will find a number of instructions are only partially implemented.

We need these finished, and some kind of test suite that will validate that the new implementations work. Pick some relevant tests in OTP/lib/compiler/test where there are some tests related to binary construction and matching. Use/extend the test framework in ERJANG/test_server so the tests can be run.

Implement more efficient String handling

One of the drawbacks of Erlang is the excessive memory usage for strings; and that is one of the areas where Erjang has real potential to provide better behavior than BEAM. Developing Erjang I have experimented with various kinds of list representations, and I think there is great potential to optimize this area.

The core idea is that since the cons operation is really a virtual method on the tail of the new cons cell; we can easily customize list structures depending on what is cons’ed in front. The class erjang.EStringList implements one such behavior, which is to behave as a byte buffer that grows to the left; passing “ownership” of an underlying buffer to anyone cons’ing a byte onto it. I.e., NIL.cons(0..255) yields an EStringList, and EStringList.cons(0..255) shares the underlying buffer, but with ownership passed on from the previous cons cell.

I know that there is some old research in this area from the good old lisp days that could also be dug out.

Implement an epmd that can be embedded into Erjang.

Currently, Erjang depends on the “Erlang Port-Mapper Daemon” binary from an OTP distribution to use Erlang distribution. We need an implementation of epmd which is either written in Erlang itself, or alternatively written in Java. Ideally, epmd was launched part of the Erlang/OTP startup sequence (written as a gen_tcp server), and used some kind of leader election mechanism to handle cases with multiple nodes running on the same machine. This Erlang-based epmd should monitor if there is an existing epmd running, and then pick up if it dies.

Alternatively (also a good option because it would not require startup changes for OTP) would be to simply write one that runs in Java (ideally using Kilim, non-blocking I/O and Erjang’s NIOSelector mechanism) that would run as part of Erjang’s startup sequence before it tries booting OTP.

Which solution to pick up would depend on your skills in Erlang vs. Erjang.

Improve the ERLConsole application

Erjang comes with a javax.swing – based GUI console; which can perhaps be used as the foundation for running Erjang as an Applet and/or as a Java WebStart’ed application. The GUI console is hacked together very quickly, and my GUI programming skills leave much to be desired so…

• Line editing sometimes barf that the program is trying to insert text into the TextArea outside content limits.
• When the embedded Erjang exits it prints out the error messages and then the window closes immediately (so you can’t see what error happened).
• Perhaps there should be a start-up window where you can choose VM options (such as the node name).

Also, if you’re into hacking this; it would be really interesting to figure out how to package Erjang+OTP into a single jar file; i.e. boot OTP form the jar file itself in stead of needing an OTP release to hang around in your file system.

Get CouchDB, EjabberD and RabbitMQ running on Erjang, and provide performance numbers.

I don’t think we’re far from being able to do this; but you probably need to do some Java-level debugging to get things to work.

• I know EjabberD uses some native code (either NIFs or drivers) to parse XMPP; but you can surely find some Java code somewhere which will do that too.
• Use rhino (http://www.mozilla.org/rhino/), a JVM-based JS-engine to do CouchDB’s JavaScript parts. [I’m working on porting Basho’s erlang_js which is a different JavaScript API].
• The web access also depends on Mochiweb, which uses the Erlang module crypto, which needs some NIFs re-implemented in Java (make sure to use R14A’s crypto which has NIFs instead of a driver, that’s a lot easier).
• Maybe other stuff…

This project could just involve one or all of the above mentioned products.

Figure out how to run the Mnesia benchmark, and provide performance results.

A plain Erlang/OTP release has a benchmark for Mnesia (it’s in lib/mnesia-4.4.13/examples/bench), but I have not tried to run it. Baically mnesia works, but how does it perform under pressure.

• I think primarily this is a matter of getting the launch scripts to run Erjang (ERJANG/ej command) where it presently uses OTP/bin/erl command.
• Report problems and/or performance numbers.
• Try to run it in a Java profiler (such as http://www.ej-technologies.com/products/jprofiler/overview.html) and figure out how it could run faster.

Try to replace the erlang slave in ErlIDE with an embedded Erjang.

ErlIDE is written entirely in Java, and so it should be entirely possible to embed Erjang into the IDE (It’s based on Eclipse).

• This could potentially be much cleaner and run faster.
• Currently Erjang’s startup procedure is designed to run as a main program, not as an embedded thing; so we need to devise some new API and make sure that Erjang code erlang:halt/1 doesn’t exit the entire JVM.
• Some of the above mentioned binary matching/construction may be necessary to get the Erlang compiler to compile everything!
• Figure out what’s missing, and fix it.