[Shootout-list] X per second scoring system

[Bengt Kleberg]

Brandon J. Van Every wrote:
> Bengt Kleberg wrote:
> 
>>''x per second'' is much more difficult to measure than
>>''time to do x''
> 
> 
> C'mon, we're all computer guys here.  Surely you jest.

no, i was beeing stright. stupid, but honest.


>>it would also take a long time to run all tests on all
>>languages since
>>we are using trial-and-error to find the right n for each
>>language.
> 
> 
> Which is why you shouldn't 'find the right n'.  You run any given test
> for 60 seconds.  However many 'n' times it runs, that's your score!
> Works fine for Viewperf.  Everything is scored in terms of 'frames per
> second'.  You don't sit around trying to guess n's.

but this is the thing i think is difficult. to run 60 seconds and then 
stop the test from the outside, without disturbing the test in 
hard-to-account-for ways.

...deleted very well thought out arguments that mostly comes back to 
this here below:
> Just loop it again.  I think the problem is you're thinking that "loop
> timers" around the source are a bad idea or something.  I will need to
> look at the Shootout source code to understand exactly what your
> objection is here.  What you're saying is making no sense compared to
> other benchmarking suites I'm familiar with.  I would simply have start,
> stop, loop, and counter functions implemented in a trivial C library.  A
> language test would either have to call the C library or implement
> equivalent timers.

the current shootout, and all the test, are timed from without the test 
code. i did not realise that we should be prepared to rewrite all the 
tests in all the languages. again i have been caught beeing lazy.

i would much prefer not having to add timers to all test cases. it makes 
code harder to read.


bengt