[Shootout-list] when benchmarks are advocacy

[Brandon J. Van Every]

Isaac Gouy wrote:
>
> imo adding a test to show-off the strengths of a language is not
> keeping the spirit of honest comparison. It's simple advocacy.

The problem is not so much having a test.  The problem is how tests are
weighted and congealed with other tests to produce a composite score or
ranking.

I think the 'main, frontlined' composite score should obey the Least
Common Denominator of all the languages.  It should not include any
tests for language-specific features, like garbage collection or
concurrency or thread locking or big integers or whatever.  I mean, why
should C++ be penalized just because Erlang has some great concurrency
stuff in it?  I'm not personally writing a concurrent application, I
don't care.

It would be more appropriate to have secondary scores and test suites
for things like:

- garbage collection
- thread control
- big number support
- safety (if someone can define an acceptable test)

or whatever.  These should not be combined with the primary language
score.  Some languages wouldn't be able to have these secondary scores,
i.e. no inherent support in the language for the capability.  We'd also
need to decide what's ok to have in a library and what must be in the
language proper.  It's nice to know that different languages have
better/worse bignum libraries, for instance, but that's getting beyond
the scope of the Shootout.  Better to have someone put up a "survey of
bignum implementations" website or some such, focused wholly on what
mathematicians want.


Cheers,                         www.indiegamedesign.com
Brandon Van Every               Seattle, WA

20% of the world is real.
80% is gobbledygook we make up inside our own heads.