[Shootout-list] Many tests cheat

[Bengt Kleberg]

skaller wrote:
...deleted
> Ok, another question then. The ring of messages test wants
> pre-emptive threading. It's possible to organise this

are you sure? i have been trying to improve the ringmsg test and the 
latest about-this-test looks like (below). does it mention pre-emptive 
threads?
when sufficient agreement has been reached about the new ringmsg i will 
suggest that the shootout webpage is update.


About this test

For this test, each program should be implemented to do the same thing.

This is a test of interprocess communication using message passing.

This test has been designed to allow the number of simultaneous 
processes to grow with N. The purpose of this is to see how the message 
passing timings, for a particular language, changes as the number of 
processes increase.

Each test program should spawn N execution contexts (processes, though 
languages with limited concurrency can use threads). N will be the first 
argument to the test program. The number of messages between these 
processes will be 128*N. This number (128) comes from a study of the 
first language used to implement this test. The cost of creating a 
process was compared to the cost of sending a message. At 128 messages 
to each process the time spent in process creation was less than 10% of 
the total run time (excluding start/stop).

Note that this value will have to be the worst case value for all the 
languages in the test. Ie, if language X needs 512 then all languages 
will have to run with 512, even those that can get by with 8. When your 
favorite language is added we will check the current value to see if it 
gives the correct relative performance of process creation/message 
passing on the for the computer/operating system that is used by the 
shootout. To make it simple to change we have 128 as the second argument 
to the test program.

When the test program spawn N execution contexts one execution context 
should be the master/source. It will behave differently from the others. 
One will be the sink. It, too, will behave differently from the others. 
The others (N-2) should all do the same thing. When the sink stops it 
should print total number of messages received. This should normally be 
the only thing printed by the test. If at any time there is an error in 
the program, the test program must report this in its output. Since this 
is a failure the test may stop right there.

The algorithm for the first process (master/source) shall be as follows:

    1. The first process (source) will wait for a ''start'' message from 
the last process (sink).
    2. The first process will send (128*N)-1 messages to the second 
process in the ring. The contents does not really matter, but should not 
be ''stop''.
    3. As the last message, send a message with the contents ''stop'' to 
the second process in the ring.

The algorithm for the last processes (sink) shall be as follows:

    1. Send a ''start'' message to the first process (source).
    2. Recive messages until a ''stop'' message arrives. Then print the 
total number of messages received (should be 128*N) and terminate the test.

The algorithm for all the other processes shall be as follows:

    1. Recive messages from the process before the process itself.
    2. Send the message to the process after the process itself.

...deleted

> Is a microthreaded solution fair? It will cream the pthread
> implementations :) I think it probably isn't fair. 

i think it would be fair.


> So perhaps, two tests -- both ring of messages,
> one requiring pthreads, and one without that

the producer/consumer test mandates pre-emptive threads. that is _not_ 
the same as pthreads, afaik. last time i looked pthreads meant 
posix-threads.

personally i think pre-emptive threads are a good idea. they are easy to 
put into the test design as a the-same-thing, if pre-emptive threads are 
what is wanted.


bengt