[Shootout-list] Threads and Threads-Flow Proposal

[Kevin Carson]

I've just submitted some working examples of the Threads and Threads-Flow
benchmarks written in Python.

They are working, correct but may fail the actual benchmark run.  This got
me thinking about these two benchmarks and what their real intent should
be...

The reason the benchmarks may fail is that the upper bound of how many
Python threads are allowed to be created varies with OS, kernel revision,
libc revision, etc. as well as the thread structures of Python.  This is not
an easy thing to fix from within a portable program written in any language.
I've tested the program on Windows XP, and two subrevisions of Linux 2.6.10
on two different architectures.  On one platform the program could allocate
~1000 threads, another didn't get past ~380 and one succeeded with the
benchmark requirement of 3000.

Obviously both threads want to create a "lot" of threads.  I suspect to make
signficant timings for some of the languages, the value was upped to
unlikely-to-be-seen-in-production 3000 threads.  But one could more portably
and usefully wrap the main loop in an encompassing repetition of the test.
This makes particular sense if one agrees with the following
characterization of the benchmarks' intent:

Threads -- test the creation, start, and tear down time of a chain of
threads.  This models systems that dynamically spawn tasks so 10 repetitions
over a chain of 300 threads is just as valid as 1 repetition with 3000
threads -- the same number of creates, starts, and stops get exercised.

Threads-Flow -- test the overhead of moving messages repeatedly through a
group of threads in a worst case scenario where each thread is woken up with
a tiny bit of work to do before passing the message onwards.  Once again,
measurement of the moving overhead is just as valid for 1500 messages over
300 threads as 150 messages over 3000 threads.  The same number of messages
are moved.

We are testing the limits of the languages for a type of application
behaviour; not the limits of a 2.6 Linux environment running on a 1.1Ghz
Athlon.

I hoping this can be changed.

Kevin