[Shootout-list] bench mark test run time, max bad score

[Bengt Kleberg]

greetings,

i suggested that the run time for the fastest implementation on a 
particular test should always be (about) 1 second. And a hard limit 
should be in place at 300 (or 512, or something else) seconds. stop all 
entries that run for longer. give them max-bad-score.

if we adopt lowest-score-is-best then we could have seconds as our score 
units. penalise the failures with N multiplied with the hard limit, ie 
this is max-bad-score. N could be the number of tests in the common 
benchmark, thus making sure that a single failure would always result in 
a score that is worse than the score of a language entry that manages to 
complete all tests.


bengt