[Shootout-list] New benchmark?

[William Douglas Neumann]

> Hmm. I don't see either of these two tests on
> http://shootout.alioth.debian.org/ (is that the proper main
> page?).

They're alpha tests in the sandbox 
<http://shootout.alioth.debian.org/sandbox/>.  I.e. not quite ready for 
prime-time.

> I think "same thing" benchmarks allow more idomatic programs for
> each language.  I find most of the "same way" benchmarks too
> synthetic and sterile.

For the most part I agree with you here, but...

You'll find that an awful lot of the "same thing" tests end up looking as 
if they were "same way" tests, as it's usually going to shave a bit of 
time off of the result to do it in an imperative style.  This is 
especially reinforced by the default CRAPS settings that count nothing but 
speed (personally I like a 3/2/1 speed/memory/LOC setting, but that's just 
me... I also think the default weight for each test should be set to 1 
instead of the current arbitrary hodgepodge, but again that might just be 
me).

Anyway, it would be nice to also see a sort of idiomatic shootout, where 
everything is a "same thing" test.  At the very least, it would be nice to 
see a few tests done in the "same way" as an idiomatic FP version. 
Something like count words or, even better, heapsort would make good 
candidates for that (far more interesting than Ackermann, at least). 
Yeoh, yeah, I know... that's not quite down with the Shootout's mission, 
but I'd still like to see it.

> Which is fine, except that we
> already have benchmarks like ackermann and fibonacci which test
> recursive functions.

I think it was more or less accepted that takfp was put in to test 
floating point + recursion...

> Or take the new fannkuch test.  It sounds
> like an interesing problem, but I lost all interest in coding up
> examples when I noticed it was a "same way" test, because...

Whoah!  When did that become a "same way" test?  Because it was a "same 
thing" when I wrote it, and it may or may not be considered to be done the 
same way as the Oberon version (certainly the control structure is 
different)...

> I had the same thoughts for both the new nsieve and pidigits
> tests.

Yeah, I'm not quite sure why the pidigits test is a "same way" test.  I 
can see forcing the overall algorithm to be the same, but I can't see why 
something like the implementation in the reference paper, or one using 
OCaml streams (like my first submitted version -- though I'd like to tweak 
that one a bit) should be disqualified (are they?  for all I know, they 
aren't).

> Of course I like the shootout because I like to see what
> features/strength/weaknesses different languages have.  And I like
> to see different way of solving the same problem.

Again, that's why I'd love to see an almost naively idiomatic version of 
the shootout (no need to tell me I can start one up, I know that... I'm 
working on something slightly different).  BTW: where have all of the 
alternate versions of the programs gone? I've lost them since the 
redesign...

William D. Neumann
<wdnx@unm.edu>

FWO to the Nth degree!!!
---
Dear Lord, please make me the kind of person
my dog thinks I am.