[Shootout-list] Stuff

[Brent Fulgham]

--- Isaac Gouy <igouy2@yahoo.com> wrote:
> Ralph's posting to comp.lang.functional summarized
> the inherent stupidity of large-scale cross-language
> benchmarking:

Well .... stupid is as stupid does, I guess.

Should we all pack up our toys and go home since
we live in an imperfect world?  :-)

> - compare language performance using different
> algorithms and data structures; and then it's a 
> comparison of algorithms and data structures and 
> programming skill, not a comparison of language
> performance.

I would find this argument more compelling if all
tests were implemented by myself.  I think that the
large number of contributors (and reviewers) to the
shootout make this second argument less valid.  Sure,
we will always (to some extent) be measuring the
skill of the implementors, but by harnessing the
network effects of the distributed work, we should
minimize the amount of advantage one uber-hacker in
ML gets over an average Joe using Java.

Even if the initial results skew towards the
uber-hacker, eventually a Java expert will take a
look (probably due to the uber-hacker flaming the
Java people on comp.lang.java or someplace) and will
propose a better solution.

This is why I think the shootout is fun!  The C++
people harassed the Ada folks using the shootout
benchmarks as 'evidence' of Ada's inferiority, and
*blam* we got tons of great Ada implementations 
(and some fixes to the Shootout build environment)
provided by angry Ada hackers.  The shootout
wins again!

Let's not be so focused on perfection that we lose
the point of the shootout.

-Brent