Anyone know of any small to medium cluster examples showing Erlang vs Python vs whatever?

cbmeeks
http://cbmeeks.blogspot.com

Thanks.

thank you very much for your comments drawing attention to a number of interesting aspects I had not covered in the article. I have learned quite a bit by exchanging views with some of you which is one of the benefits of publishing in the blogosphere

In retrospect I think it is fair to state that running the benchmark on my single-CPU system did not really do justice to Erlang which appears to have advanced capabilities when it comes to distributing processes on multi-CPU machines or even clusters.

Furthermore, a proper benchmark would have to tackle problems that are more amenable to parallelisation in order to allow the languages/systems involved to show off their true potential.

Anyway, it was an interesting experience, and I for one will continue to experiment with both Erlang and Python.

Best regards — Muharem

A few comments:

Stackless Python has its concurrency support built in C, but Python
is not a compiled language. I don’t know if the E code was compiled?

Stackless development for CPython has not gone much further for the last 3 years
because I was involved in the PyPy project, which is going to be
the superior Python at some point. Stackless has ben implemented
on top of PyPy, with support for multiple concurrency models at
the same time, supporting coroutines directly in the core.
At some point, I will write a similar benchmark that compiles
to native code.

This site might be interesting: http://codespeak.net/pypy/

Stackless’ concurrency model is a little bit modelled after the Limbo language
from Plan9. My intention at that point was to have a simple
model with anonymous rendevouz points. Meanwhile, we have
found a scalable way to handle multiple concurrency models
in a composable way, with or without explicit naming of jump targets,
and the old Stackless model is now just a special case.
Stay tuned, there is a lot more to come.

About the GIL:
Yes, it is always a problem with CPython, and although there once was
a huge patch to replace it by extensive object locking, this will probably
not vanish from CPython in any near future. PyPy has the flexibility to try alternatives
to a GIL, there was just not enough time during the EU funded
project, but it will come for sure. We will be looking into ways
to do better separation of objects, to avoid to much locking expense.

On real threads:
Stackless has limited support to control real threads with a tasklet.
You can have tasklets be running in different threads. This helps
with blocking OS calls, but not with the GIL problem. So again,
if there will be a solution for real concurrency and multiple CPUs,
this will first appear in PyPy.

On number of processes:
I’d like to point out that Stackless’ tasklets are very tiny objects.
I would be interested to know how things scale in Erlang.
Try your example with 100000 tasks. This is a cakewalk with
Stackless, since a tasklet is a few 100 bytes, only.

Thanks again for this nice blog, I will follow this.

cheers - chris