Steven Parkes : smparkes.net

Tim Bray: My experience in the Wide Finder didn’t particularly make an Erlang-lover, but I sure did come to appreciate its message-interchange framework.

So is this going to be the year of message passing? I’m old enough to remember many years that were going to be the “year of LAN”. For years, people that used LANs couldn’t figure out how people that didn’t have them could do without. Then LANs broke out and everybody did have them. And before long, nobody could figure out how they could do without. But it was years.

Will that happen with concurrent programing, and with message passing based concurrency in particular? Or will it always be a niche for the somewhat crazy?

It does seem like once you start, it’s a bit hard to image going back. You might not need to go hog wild and create a telco switch with tens of thousands of processes every day but once you have easy concurrency, you find lots of little uses for it, where you had rather annoying non-concurrent code in the past.

That is, that’s true for some definition of easy concurrency. There’s the rub. How much harder does writing concurrent programs have to be? Can we make it not harder at all? Or only a little harder? Because if we can’t, I doubt it ever will break out. That despite the dreams of all the multicore CPU vendors.

I’m optimistic, but there are a lot of degrees of freedom when it comes to deciding how to implement concurrency, even in message passing systems. So it seems unlikely that the same “right” approach is going to be obvious to everyone. I don’t how this will evolve, just the thought that the paradox of choice (not to mention FUD) is likely to keep a lot of people from even experimenting until things are a bit better understood.

One thing that seems a lot less debatable is tools. The tools we have for debugging concurrent programs … well, they pretty much suck. Prints and stack traces (which, in a concurrent world virtually always end up in the scheduler) are significantly less useful than they are in non-concurrent programs. Though it seems we want them, we seem to have not spent as much time on building them. And till we do …

So I have big doubts about this year being the year of message passing. Or even next. Etc. But just like it was with LANs, it doesn’t keep me from having plenty of fun with concurrency while I wait to surf the wave that I do think, everything above notwithstanding, is inevitable.

Charlie Nutter gives a good roundup of Ruby implementations past to present, with a few guesses as to the future.

I’ve played with four: 1.8/MRI, 1.9/YARV, JRuby, and Rubinius. What I found a bit surprising is that I’ve come to appreciate pieces of all of them. MRI is pretty stable and widespread. 1.9 allows me to play with OS threads faster. JRuby allows me to play with the JVM crowd (and push threads even more). And I think we all owe the Rubinius for the spec work.

But there’s one thing that having four versions has helped immensely with: writing Ruby with threads. Threads can be such a bloody pain to develop. In all but the simplest programs, it gets pretty tough pretty quickly to be sure there are no unexpected sequences of events. When you run primarily on one platform/library, you pretty quickly surface the low hanging fruit: the races/sequences that occur most frequently on that platform. But there are all those rarer cases that kill you. Since they’re rare, they’re typically gone by the time you notice them and it’s virtually impossible to get them to repeat. So you start to rationalize: maybe I didn’t really see it; maybe I misinterpreted what I saw. Maybe it was a side effect of something else I already fixed.

But in the back of your mind, you wonder if you’re lying to yourself.

It’s the stuff of nightmares, a monster you just know is hiding under your bed.

In the case of Ruby, we have four implementations that share a pretty solid thread model … with wildly varying dynamics. In developing some threaded code, I started on MRI and made it solid. And then ran on 1.9 and everything went to hell because things were happening in a different order. Ditto for JRuby. So far, the only case where I didn’t surface more corner cases was Rubinius, which either means I’m getting farther out on the latent bug curve or I simply haven’t run enough tests on it.

If you’re writing threaded code on Ruby and it’s feasible, it might be worth considering developing on more than one VM, even if you only expect to actually roll one VM into production.

Of course, avoiding writing-down-and-dirty with threads is not the only nor (arguably) the best approach to concurrency, but you already knew that …

A lot of interest has been growing in actors in the last year or two. This is almost certainly due in no small part to Joe’s book on Erlang. But I also think there’s a growing need for actors: I see more and more people running into limitations in serial programming, both on the web and off. I know I have.

Last year, I started putting together a simple webapp to help me search and reserve books at the several local libraries I use (and Amazon). As coding progressed, things got slower and slower as more features required accessing these various web services, one after another through multiple steps. Many of them are none too fast individually; in aggregate, they crawl. Most of these services could be accessed concurrently if I could express concurrency in my app.

Enter threads. I’ll admit it. I hate threads. Threads know nothing about my carefully crafted classes and instances: they run roughshod through them. I want my objects and concurrency, too.

Which is pretty much the description of actors: they’re chimeras: part object, part thread. We’ve known about them for a long time: the actor model is thirty years old. But we’re trying to do more with our interconneted systems than we have in the past. At least I am. So now I care more than I did thirty years ago.

Unfortunately, the languages I use the most, ruby and python, don’t have actors and while I respect (and am happy to copy the great ideas from) Erlang and OTP, it’s not really feasible for me to jump to a functional language. There are several actor libraries for python and ruby but their design targets, in terms of functionality and platform, are different than my needs.

So today I put out a (pre)release of dramatis, an actor library for dynamic languages … which is, right off, a bit of a stretch. The current prerelease is Ruby only (though it does run on 1.8.6, 1.9. and jruby … and at least a little on Rubinius). But the python implementation has started and since it’s a fairly straightforward reimplementation of the Ruby version, should be available soon.

dramatis shares a lot in common with other actor implementations. In fact, I tried to copy a lot of the best practices from other implementations, though there’s much more to be copied. It does have a few things that make it a bit unique. It’s pretty tightly integrated with the native language: writing programs with dramatis actors looks an awful lot like writing programs with normal objects. Hopefully this makes adding concurrency to programs easier, though I suppose only time will tell.

It’s at prerelease in as much as it isn’t packaged (there’s no ruby gem yet) and it hasn’t been vetted by many. But it’s downloadable from the git repository and comes with docs, a tutorial, and a few examples. Tested on Linux and OS X.

If you’re an adventurous type and brave enough to do some of that aforementioned vetting, have at …