[Shootout-list] testbed normalization

[Bengt Kleberg]

Brandon J. Van Every wrote:
...deleted
> So, what to normalize against?  How about the performance of the C
> benchmark on any given machine?

> 
...deleted
you can not normalize. the test results are only valid on the machine 
that ran the tests. you can not compare different computers. not even 
the same hardware with different operating systems.

this does not matter if the interested party intend to use the same 
hardware/os as the the one used for the test.

otherwise we get the interesting problem that i have to deal with. i 
have a sparcstation, with solaris. the realtive performance of erlang 
and mzscheme on my machine is such that erlang usually beats mzscheme. 
when i then submit the tests to the shootout and they are run on that 
machine, mzscheme is suddenly faster.