[Shootout-list] Directions of various benchmarks

[Jon Harrop]

On Wednesday 18 May 2005 21:14, Einar Karttunen wrote:
> Lots of inputs means lots of worthless code parsing input and
> potentially IO-bound programs.

You can read a lot of data with a tiny amount of code in most languages (e.g. 
an int list). Reading a lot of inputs (e.g. 2^256 possible inputs) does not 
require reading a lot of data (e.g. 4 64-bit ints) and is not likely to 
result in an IO-bound benchmark.

I completely agree with Skaller that the tests should be designed such that 
they cannot be replaced by completely memoizing or answer-printing code. The 
random benchmark, for example, should input the initial seed.

-- 
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
Objective CAML for Scientists
http://www.ffconsultancy.com/products/ocaml_for_scientists