[Shootout-list] Missing programs and score calculation (Was: Re:TinyCC and Io)

[grfgguvf at gmail.com]

On 9/15/05, Dave <davejf at frontiernet.net> wrote:
> > d) some tests dont benchmark the languages implementation. for example
> > the "threads-flow" description says:
> >   Each program should create, keep alive, and send integer messages
> >   between N explicitly-linked threads. Programs may use kernel threads,
> >   lightweight threads, cooperative threads…
> > so what do you want to benchmark? i thought of contributing a c
> > version. but how to implement it? use posix threads? or fork 500
> > processes? linux would just emulate the threads with processes. use
> > rfork? what are we benchmarking? it may be the kernels fork overhead
> > or the pthreads library, but doesnt have anything to do with the
> > compiler or the language. (for example try creating 500 posix threads
> 
> The results, on the same machine, are still wildy different for different
> languages, which is the point. I think it's pretty obvious what it is trying
> to measure: How efficiently an implementation can create, use and dispose of
> many threads, which of course can be important for a lot of server software.
> It would be nice to maybe see more rational for why a test was included in
> the About section though.

Well, my point was, more specifically, that the "threads-flow"
benchmark is pointless. Because: it's comparing userspace threading
with kernel threading. No surprise there isn't a C version, if there
was one, it would show the inefficiency of the POSIX threads
implementation in GNU libc, and not gcc or C in general. (If the
benchmarks were run on Windows, OpenVMS, FreeBSD, etc... this won't be
a problem, becuase those systems have a libc with an efficient POSIX
threads impl.)

Why is comparing userspace threading with kernel threading pointless?
Because: the whole point of threading is to harness the power of
hardware capable of parallel execution (that'd be SMP these days).
With userspace threading all threads run on the same CPU. On the other
hand, the GNU libc's POSIX threads implement 1:1 threading; one kernel
process per thread. This is also clearly inefficient when the number
of threads > number of CPUs (as with "threads-flow"'s 500 threads).
(An other question is what's the point in having more threads than
processors...)