[Shootout-list] X per second scoring system, resume

[Bengt Kleberg]

Brandon J. Van Every wrote:
...deleted
> 
> Even for 1 language it's a mistake.  What you're starting up varies *by
> test*, although I don't know how much it actually is in various cases.

yes, this is true. provided that we do not want the things done 
preparing to run the test included in the test. but i want them. the 
reason why was explain in the previous email.


>>>Shutdown time is highly dependent on the data structures and control
>>>flow of the test.
>>
>>this is correct. and, hopefully a good thing (tm). ie, we
>>have a fixed
>>shutdown time that is measured for a as-short-as-possible program.
> 
> 
> It varies *per test*.  What if the language has all sorts of
> housekeeping that needs to be performed between the end of the program
> and the end of the process, for normal termination?  Especially for
> things like system memory, threads, and IO.
> 

yes, this is true. provided that we do not want the things done 
preparing to run the test included in the test. but i want them. the 
reason why was explain in the previous email.


> 
>>it is not totally correct. but i would say that it is much more
>>truth-like that having the results of many years ago archived
> 
> 
> Um, no, the *archiving* part is uncontroversial...
> 
> 
>>and used for comparing with todays results :-)
> 

yes, this is true. it is not the archiving by itself i object to. it is 
what comes after the ''and'' in my sentence i object to.



> Sure, if you can prove startup / shutdown is 'sufficiently bounded'.
> Just measuring a different test isn't proof though. 

yes, this is true. provided that we do not want the things done 
preparing to run the test included in the test. but i want them. the 
reason why was explain in the previous email.


...deleted

>>what if we run the smallest program several times and see how much
>>variation there is in the results?
> 
> 
> You need to allocate system memory, start threads, perform IO...

i do not htink i should do that. i want to avoid having those things 
done in this partiucular case. instead i want the fastest possible 
shutdown. when shutdown times gets longer in the other tests i want that 
increase to be counted towards the test.


bengt