Fwd: Re: [Shootout-list] Science-related benchmarks (speedoptimisation)

[Brent Fulgham]

--- Bengt Kleberg <bengt.kleberg@ericsson.com> wrote:
> Brent Fulgham wrote:
> ...deleted
> > I would like to keep the compiler switches,
> because
> > it provides interesting information about what
> > compiler switches do, and how they affect the
> > output.
> 
> this is a really interesting question. it does sound
> sensible to compare 
> two compilers for the same language. but why stop
> there? why not compare 
> the normal compiler/runtime switches (vanilla) with
> the best possible 
> combination of compiler switches (all-flavors)?
> because that is even 
> more interesting.

Yes, this could be done.

> where do we stop comparing implementations? say that
> a language (c) has two compilers (gcc and icc) and 
> that is ok to put both in the shootout.
> what about another language (ocaml) that has two
> compilers (ocamlopt and ocamlc). why deny this 
> language the same treatment as c gets?

We don't.  ocamlopt and ocamlc are both on the
'Great' version of the shootout:

http://shootout.alioth.debian.org/great/benchmark.php?test=all&lang=ocaml&sort=fullcpu
http://shootout.alioth.debian.org/great/benchmark.php?test=all&lang=ocamlb&sort=fullcpu

> or what about a third language (erlang) with a
> single compiler that can produce two very different 
> results (interpreted byte code or machine 
> code). why deny this language the same treatment as
> c gets?

It does:

http://shootout.alioth.debian.org/great/benchmark.php?test=all&lang=erlang&sort=fullcpu
http://shootout.alioth.debian.org/great/benchmark.php?test=all&lang=hipe&sort=fullcpu
 
> while it is impossible to please everybody i think
> it would be a benefit for the shootout to have the 
> reasoning behind these kind of decisions 
> available in the faq.

It's pretty simple -- if we know such a feature
exists, and someone can provide the required command
line options and flags we include it.

-Brent