[Shootout-list] demoting marginal languages

[Brent Fulgham]

On 2004-09-28 03:33:23 -0700 Bengt Kleberg 
<bengt.kleberg@ericsson.com> wrote:

> 1 we have yet to decide if a good performance means a low score or a 
> high 
> score. or has that already been done?

Good performance means "low use of resources":  Shorter CPU cycles, 
less memory,
shorter lines-of-code is better.

However, these data are then handled per me earlier e-mail, which 
places "low use" scores
at the high-end of the range (so they bubble up to the top)

Therefore, a high score means a "more efficient" or "better" 
implementation.

> 2 yes, i did intend for the missing test to be awarded maximum bad 
> score, 
> giving this language a loss to a language with complete tests. this 
> would 
> save us the trouble of having to invent a separate 'Provisonal' 
> category.

Right now, it gets a score of zero if it is not implemented, or if the 
program fails to complete
in the alloted time (currently 5 minutes of CPU time).

> in a way it is. we could have one test for all string handling. that 
> test 
> would then be subject to debate about how much of it that should be  
> set 
> aside for the different string handling primitives that exist.
> do we want
> 10% execution time for string add
> 20% execution time for string search
> 20% execution time for string regexp
> perhaps string reverse search should be counted in string search? 
> etc? hours 
> of endless fun awaits :-)

Ugh!  Can't we just have simple tests that do one thing?  :-)

> the good thing with the very more difficult to administrate approach 
> of 
> having several string tests is that we could then weight the various 
> results 
> much more easily.

yes -- plus for we lazy people, we don't have to change anything...

-Brent