[Shootout-list] New benchmark?

[Greg Buchholz]

--- Isaac Gouy <igouy2@yahoo.com> wrote:
> Is there a dearth of "same thing" benchmarks?
> reverse-complement and k-nucleotide are same-thing at present.

Hmm. I don't see either of these two tests on
http://shootout.alioth.debian.org/ (is that the proper main
page?).  
Here's a list of the "same thing" tests I see...

Message
Matrix 
Statistics
Process
Reverse file (deprecate)
Count-words
Word Frequency 

> 
> What do you think "same thing" benchmarks show?

  I think "same thing" benchmarks allow more idomatic programs for
each language.  I find most of the "same way" benchmarks too
synthetic and sterile.  Take for example the takfp test.  As it is
currently presented, the answer it computes can be calculated in
constant time, but each implementor is supposed to create a
exponentially recursive version.  Which is fine, except that we
already have benchmarks like ackermann and fibonacci which test
recursive functions.  Or take the new fannkuch test.  It sounds
like an interesing problem, but I lost all interest in coding up
examples when I noticed it was a "same way" test, because...

A) I wouldn't want to argue about whether or not an implementation
in GHC (or pick your favorite language) was close enough to the
"same way" and... 

B) I tend to think the answer is more important than how you got
there.

I had the same thoughts for both the new nsieve and pidigits
tests.  Of course I like the shootout because I like to see what
features/strength/weaknesses different languages have.  And I like
to see different way of solving the same problem.  Which usually
means I look at the lines of code metric more than the other two. 
I'm not advocating removing the "same way" tests, just adding more
"same thing" ones (or changing some "same way" into "same thing").
 And instead of just complaining, I thought I'd at least present
something.  I hope that if it doesn't get accepted, that it will
at least have inspired someone to create more of the "same thing"
tests.

Just my two cents,

Greg Buchholz





		
__________________________________ 
Do you Yahoo!? 
The all-new My Yahoo! - Get yours free! 
http://my.yahoo.com