[Shootout-list] X per second scoring system

[Bengt Kleberg]

Brandon J. Van Every wrote:
...deleted
> *Timed*, yes.  But it seems you *count* from within the code.  It's
> invasive.

often the count is the number of times something should be done (create 
n threads).
other times n is used to create input data that decides the amount of 
work done by the test.
both ways are easier to keep accurate than having external timers stop 
the test and still manage to extract correct values from it.


...deleted

> You already have obfuscating loop code, and obfuscating command line
> argument processing code.  Adding a timer would be a trivial amount of
> additional text.  The real difficulty is creating and adding the timers
> in different languages.  It's gruntwork.  It would make it possible to
> run the benchmark twice as fast, however, assuming you're using a
> doubling calibration for N.  How are you actually calibrating N?

you are correct about there already beeing obfuscating code present. i 
think it should be kept at a minimum. adding timers is adding, even if 
it takes a trivial amount of additional code. not to mention that some 
languages might need more than trivial. perhaps.

you are also correct about the kind of work needed to add invasive 
timers. with a (small) fear of beeing repetetious: i am lazy.

the calibration of n is somebody-elses-problem. as mentioned in one of 
my previous emails to the list it does not matter so much since we have 
a table for each test with lots of n. this is a good thing (tm), see 
Timing Trials, or, the Trials of Timing 
(http://cm.bell-labs.com/cm/cs/who/bwk/interps/pap.html) as to why.


as an aside: why is there no link from the shootout to Timing Trials?


bengt