[Shootout-list] Ackermann

[Eric Lavigne]

>> It works by using a simple rule "don't try to cheat".
>> Is that really too hard?
>
>Yes, that really is too hard. This is exactly what I keep saying - the not=
ion
>of "cheat" is totally subjective. The only solutions is to be design the
>tests such that no arbitrary restrictions are required to make them
>worthwhile. Note that this isn't even difficult, it's just that people hav=
e
>to recognise its value.

I agree with this completely. In fact, as a programmer I would like
very much to find clever ways to "cheat." Some of these tests make it
too easy, though, so that going all out to optimize will lead to times
of 0.0 sec. Looking for patterns in a problem that allow parts to be
precomputed is a significant part of optimizing in real problems. With
only minor changes, as Jon recommends, we can go all-out to get the
best possible performance from our codes without leading to the
benchmark becoming totally meaningless. In fact, the results would be
more valuable because they would show what sort of performance a
determined programmer is capable of achieving, rather than what
performance someone gets when following (or breaking) rules about what
sort of optimization is allowed.

>Consider this: if "random" seeded the random number generator with the giv=
en
>integer then the test would not be any more "complex" but could no longer =
be
>precomputed.

I basically like this idea. There might be issues where the rankings
shift a bit depending on which random number was chosen, but that
shouldn't be a big issue. Maybe average the results of several
different numbers? The webpage for that benchmark should indicate what
range of random numbers are being used because some languages will
need to allocate arrays and other containers at compile time.