Friday, September 26, 2008

Living in a concurrent world

Erlang is a great platform for writing concurrent, distributed, fault-tolerant applications. But it's not the only one. Some of the popular programming languages also have tools that help developers build concurrent, highly scalable web services.


One of the main sources of Java power is that it has libraries to do almost anything you can imagine. And there are of course libraries that offer various solutions to the scaling out problem, like JMS with JCache or Scala Actors library.
But the real killer application in my opinion is Terracotta. It's an open source clustering solution for Java that works beneath the application level. It uses transactional memory to share objects between applications running on different JVM instances, most notably on different physical machines. Object pool is stored on a master server, which can have multiple passive backups. As a result you receive a clustering solution transparent for the applications you write (you only need to define objects you want to share in Terracotta configuration files). What's more, you can use Terracotta to cluster applications created in languages other than Java, but running on JVM - like JRuby or Scala.


Python programmers can use Disco. It's an open source map-reduce framework written in Erlang that allows you to scale your application easily. Unfortunately, it does not allow neither sharing any data between different application instances (like Terracotta), nor passing messages (like Erlang itself).


Ruby looks much better than Python in this aspect. First, there is JRuby, which you can cluster with Terracotta. Second, there is soon-to-come MagLev by Gemstone. It's a Ruby implementation running on a Smalltalk virtual machine, and the first public presentation of MagLev at RailsConf 2008 was quite impressive.


Not much to say here... You cannot expect much from a language created to produce dynamic HTML pages. You can only rely on external libraries like memcached to store and retrieve objects, but it's by no means an extension to the language itself.

If you know of any other interesting projects providing transparent (or almost transparent) concurrency and distribution to already existing programming languages, you are welcome to tell me about it!


Muchacho said...

I Don't see any difference between Python and Ruby, At the moment both are awful for concurrency. There is Jython it is Python over Java and soon it will reach the 2.5 release, There is stackless python, there is PyPy project that it will get python to the next level.

Python is very capable and have better performance now even than Ruby at the moment.


Krzysztof (Christopher) Kliś said...

If you have a look at some benchmarks (like the one at The Computer Language Shootout) it turns out that almost all languages have better performance than Ruby :-)
I deeply admire Python for its combination of power and ease of use, although I despise its indentation as code block delimiter. It drives me crazy when the program stops working after modifying source code in another editor just because it changed the code formatting.
But I like a very smart sentence by Damien Katz: "There are the languages everyone complains about, and there are the languages no one uses".