Clustering Kawa with Terracotta

In my recent post I described a method of building a cluster of JScheme instances using Terracotta object management system. I also prepared a small library for JScheme that makes using Terracotta in JScheme easy - it is LGPL licensed and free for download. Today I tried the same trick with Kawa, which is not only a full-featured Scheme implementation in Java, but also a complete framework for implementing other programming languages to run on Java platform. Knowing how to cluster JScheme, making Kawa work with Terracotta was a piece of cake.
First you start a Terracotta server. Then you run a Kawa shell through the following command:

java -Xbootclasspath/p:[terracotta boot jar] -Dtc.config=tc-config.xml -Dtc.install-root=[terracotta install dir] -jar kawa.jar

Just as with JScheme, boot jar is is a stub jar file that starts a Terracotta client before the main application and tc-config is a Terracotta configuration file (see the post about JScheme for details).
Next, you load the Terracotta library:

(load "kwtc.scm")

and define an object (for example an ArrayList) to share across the cluster:

(define ob (create-root "list" (make <java.util.ArrayList>)))

Now you can put values on the list:

(sync-write (lambda () (ob:add 1)) ob)

and read them back synchronously:

(sync-read (lambda () (ob:get 0)) ob))

or simply with:

(ob:get 0)

The library for Kawa can be downloaded from here.
Happy coding!

Clustering JScheme with Terracotta

In one of my previous posts I mentioned Terracotta, an open source clustering solution for Java. Its main advantage over traditional Java solutions in use (like RMI) is that it works as middleware between a JVM and an application, enabling transparent sharing of Java objects across the network. The only requirement for a programmer is to execute operations on shared objects in synchornized context, which allows you to build distributed applications or refactor already existing ones quickly.
Inspired by Jonas Bonér's experiments with JRuby I decided to give it a try with JScheme, an open source Scheme implementation running on JVM. I chose JScheme over other Scheme implementations (like Kawa or SISC) because of its very simple, clear and elegant interface to Java objects. In fact, since Terracotta operates on the JVM level, you can use it to cluster any application written in any language, as long as it compiles to Java bytecode and provides an interface to communicate with Java objects and classes.
First, you need to download and install Terracotta (current stable version is 2.7.0). Then, you have to start a server with start-tc-server.sh script. In Linux, if you encounter any strange errors running any of the scripts in the bin directory of your Terracotta installation, change the header of a problematic script from #!/bin/sh to #!/bin/bash - this should help to solve the problem. The server manages all the shared objects in Terracotta cluster and needs to be started before any client applications are run.
Next, you need to prepare a client configuration in the form of an XML file. I used a configuration provided by Jonas and stored it in tc-config.xml:

<?xml version="1.0" encoding="UTF-8"?>
<tc:tc-config xmlns:tc="http://www.terracotta.org/config">
  <servers>
    <server name="localhost"/>
  </servers>
  <clients>
    <logs>%(user.home)/terracotta/jtable/client-logs</logs>
    <statistics>%(user.home)/terracotta/jtable/client-logs</statistics>
  </clients>
  <application>
    <dso>
      <instrumented-classes>
        <include>
          <class-expression>*..*</class-expression>
        </include>
      </instrumented-classes>
    </dso>
  </application>
</tc:tc-config>

Now you can start JScheme interpreter hosted by Terracotta:

java -Xbootclasspath/p:[terracotta boot jar] -Dtc.config=tc-config.xml -Dtc.install-root=[terracotta install dir] -jar jscheme.jar

Boot jar is a stub jar file that starts a Terracotta client and connects to the server before the main application is started. You should be able to find it in lib/dso-boot folder of your Terracotta installation directory. If it isn't there, you can generate it with make-boot-jar.sh script found in Terracotta bin folder.
Now when the whole working environment has been set up, you can use com.tc.object.bytecode.ManagerUtil class to create a shared root object. Unfortunately Jonas's method which uses LOCK_TYPE_WRITE static field of com.tc.object.bytecode.Manager class to perform a write lock during this operation fails to work. It causes some strange error about missing com.tc.object.event.DmiManager class, which seems to be a problem even with Jonas's JRuby example itself. A quick solution to this problem is to define locks in a way they are defined in com.tc.object.lockmanager.api.LockLevel class:

(define TC_READ_LOCK 1)
(define TC_WRITE_LOCK 2)

Now let's define a sample object to share. It can be, for example, an ArrayList:

(define l (ArrayList.))

Next you need to create a shared root object named "list" that will hold the l object:

(import "com.tc.object.bytecode.ManagerUtil")
(ManagerUtil.beginLock "list" TC_WRITE_LOCK)
(define ob (ManagerUtil.lookupOrCreateRoot "list" l))
(ManagerUtil.commitLock "list")

Terracotta provides a great debugging tool called Terracotta Administrator Console to analyze the objects held by the server. Start the console by running admin.sh script found in bin folder of the Terracotta installation directory, then connect to localhost and go to Cluster object browser. You should see an empty ArrayList on the object list.
Now let's add a new value to the shared object:

(ManagerUtil.monitorEnter ob TC_WRITE_LOCK)
(.add ob 1)
(ManagerUtil.monitorExit ob)

Go back to Terracotta Administrator Console, select the shared ArrayList and press F5 to refresh the view. You should see that now it holds a single value: 1.
Now start another scheme shell instance and try to read the first list value on the shared list:

(define TC_READ_LOCK 1)
(define TC_WRITE_LOCK 2)
(import "com.tc.object.bytecode.ManagerUtil")
(ManagerUtil.beginLock "list" TC_WRITE_LOCK)
(define ob (ManagerUtil.lookupOrCreateRoot "list" (ArrayList.)))
(ManagerUtil.commitLock "list")
(.get ob 0)

The return value is 1. Sweet!
I wrote a small library for JScheme that allows you to perform the basic operations of creating shared root objects in Terracotta and modifying them with read and write locks. You can download it from this location.
To do the list operations described above you can simply do:

(load "jstc.scm")
(define ob (create-root "list" (ArrayList.)))
(sync-write (lambda () (.add ob 1)) ob)

in one shell and then read the list value in another shell:

(load "jstc.scm")
(define ob (create-root "list" (ArrayList.)))
(.get ob 0)

Have fun!

Erlang tips and tricks: records

The most awkward thing I ran into while programming in Erlang are things called records. I call them "things" since they are neither Erlang primitives nor data structures. In fact, records are not even understood by Erlang VM itslef (try to define a record in interactive shell). They are some kind of macros (they remind me the ones used in ANSI C) that are translated by the compiler into Erlang native data structures.
The biggest problem with records is that you cannot dynamically modify them. For example, suppose you want to create a table in Mnesia, but you don't know at the moment how many columns your table will have. With records, you have to provide all record declarations for all tables you plan to generate on compilation time, which sometimes is simply impossible.
I did some research on the topic while writing Mnapi, and I must say it wasn't that easy if you don't know where to start. Finally, I managed to decipher Erlang records secrets mostly by analyzing Mnesia source code. One more reason for using open source software, by the way :-)
It turns out that records are in fact wrappers for tuples. So a record defined as:

-record(book, {isbn, author, title}).

gets translated internally into:

{book, isbn, author, title}.

Thus, an equivalent of:

mnesia:write(#book{isbn = I, author = A, title = T}).

is:

mnesia:write({book, I, A, T}).

The only advantage of such crippled structure is that the compiler knows the positions of all elements in the record, so it can always put them in a valid tuple in correct order and find possible errors instantly at the compilation stage. No matter if you write:

#book{isbn = I, author = A, title = T}.
#book{author = A, isbn = I, title = T}.
#book{title = T, author = A, isbn = I}.

the #book record will eventually end up in Erlang as:

{book, I, A, T}.

Records try to make accessing data easier. To get an author from the tuple above you would have to use element function:

X = element(3, {book, I, A, T}).

which can become problematic if you shift fields in the tuple. With record you can do the same simply with:

X = #book.author.

no matter what the order of the record is. But it's still cumbersome and static.

Fortunately, Erlang allows you to build dictionaries, which can be used to store Key - Value pairs dynamically. For example:

D = dict:new(),
D1 = dict:store(isbn, 1234, D),
D2 = dict:store(author, myAuthor, D1),
D3 = dict:store(title, myTitle, D2),
Book = dict:store(book, D3, D3),
dict:find(author, Book).

Personally, I don't use records, unless I really have to. I use dictionaries instead, they are much more powerful and elegant. And you can evaluate them on runtime.

Mnapi 1.2

I tweaked a bit my Java/PHP API for Mnesia by adding memory caching for auxiliary table structures used by the API. My tests have shown that it speeds up fetching single rows for tables where data are kept in disc_only_copies by about 2%. Much less than I expected, I must say. But since the changes are really little and work seamlessly with data created with Mnapi 1.1 and earlier, I decided to leave them in the code. If anyone is interested in the new version, here it is.

Mnapi 1.1

Today I released a new version of my Mnesia API for Java and PHP. The main change is merging main code trunk with the Tokyo Cabinet branch and adding the ability to select storage type when creating a table. You can now choose one of the following:
* ram - for RAM only based storage,
* disc - for disc based storage with RAM copy of the table for improved performance (this is the default),
* disc_only - for disc only based storage (much slower, but uses a lot less memory),
* tokyo - for use with Tokyo Cabinet storage engine.
The API does not support table fragmentation and as for now I don't plan to introduce this feature (you can always use tokyo as a storage for data exceeding 2GB limit). But, since the library code is licensed under LGPL, you are free to extend it or modify it if you lack any feature.
You can get Mnapi 1.1 here.
Happy coding!