[Shootout-list] Benchmark page vs. side-by-side page: different numbers

[Brent Fulgham]

On Monday 22 November 2004 04:06 am, Alan Post wrote:
> The "Full CPU Time" numbers in the main benchmark page don't always
> match up with the "Full CPU Time" numbers in the corresponding
> side-by-side page.   For example, the count-words benchmark:
>
>   http://shootout.alioth.debian.org/benchmark.php?test=wc&lang=all&sort=cpu
>
> lists, for N=2500, the following "Full CPU Time" numbers:
>   gcc:    0.13
>   clean:  0.15
>   mlton:  0.17
>   ocaml:  0.19
>
> yet the side-by-side page:
>
>  
> http://shootout.alioth.debian.org/sidebyside.php?test=wc&p1=gcc-0&p2=clean-
>0&p3=mlton-0&p4=ocaml-0&sort=cpu
>
> lists, for N=2500, the following "Full CPU Time" numbers:
>   gcc:    0.08
>   clean:  0.09
>   mlton:  0.12
>   ocaml:  0.13
>
> Are different measurements being used?  I noticed similar
> discrepancies on other tests' pages.

Thanks for catching this -- I had totally missed it!

It turns out that the original shootout took the minimum CPU and maximum MEM 
measured during the three iterations of each program.  My revised data 
collection was taking the average of the three.

I've revised my stuff to correspond to the original (using min/max) to be 
consistent.

Website will be updated shortly.

Thanks,

-Brent