[Shootout-list] Simple tests

[Einar Karttunen]

Hello

A simple test has typically a LOC of <20 in most languages and takes one
simple parameter for input and prints out simple output. What does this 
mean?

The test is easy to implement and understand.
It is easy to analyze what different implementations do and detect if
someone is not following the spec. With the complex benchmarks the
solutions may diverge (e.g. raytracer with odd N) for values we don't 
test.

Most of the time the best performing solutions share the same
bottleneck.

It is fairly easy to understand what is tested.

The test can be precomputed. One can create an array which contains the
answers for various N. But on the other hand in the complex tests one can
provide precomputed answers for 'interesting cases' (namely the ones we
test). I don't think this is a problem in practice if we use common
sense and refrain from the "do the same thing N times".


For these reasons I think we should keep the simple tests in the
Shootout. Complex tests have their place too, but I think that
discarding the simple ones would lose much of the original spirit.


- Einar Karttunen