Fwd: Re: [Shootout-list] Science-related benchmarks (speedoptimisation)

[Bengt Kleberg]

Brent Fulgham wrote:
...deleted
> I would like to keep the compiler switches, because
> it provides interesting information about what
> compiler switches do, and how they affect the
> output.

this is a really interesting question. it does sound sensible to compare 
two compilers for the same language. but why stop there? why not compare 
the normal compiler/runtime switches (vanilla) with the best possible 
combination of compiler switches (all-flavors)? because that is even 
more interesting.
not to mention vesions of the same compiler.
narrow focus is a good thing, but unless we keep the narrow bit 
consistent we should explain about the exceptions somewhere. or forever 
have to deal with questions, over and over again.

...deleted
> John is right.  And the big flaw in the shootout
> is that we are not measuring langauges, per se,
> but implementations.  But I say, "so what".  It's

where do we stop comparing implementations? say that a language (c) has 
two compilers (gcc and icc) and that is ok to put both in the shootout.
what about another language (ocaml) that has two compilers (ocamlopt and 
ocamlc). why deny this language the same treatment as c gets?
or what about a third language (erlang) with a single compiler that can 
produce two very different results (interpreted byte code or machine 
code). why deny this language the same treatment as c gets?

while it is impossible to please everybody i think it would be a benefit 
for the shootout to have the reasoning behind these kind of decisions 
available in the faq.


bengt