[Shootout-list] main benchmark

Brandon J. Van Every vanevery@indiegamedesign.com
Mon, 27 Sep 2004 09:02:51 -0700


Bengt Kleberg wrote:
> Brandon J. Van Every wrote:
>
> ...delete
> > Fine.  The *MAIN BENCHMARK* should only test the least common
> > denominator of all languages.  It cannot, for instance,
> mandate garbage
> > collection.  3rd generation capabilities, i.e. like C, are the only
> > things all of the languages have in common.
>
> why this arbitrary stop at c? why not intercal?

Because C and other 3rd generation languages, like Fortran, are
important.  Intercal isn't.

> seriously, perhaps the java feature set would be a ''industry
> standard'' minimum?

Because C, C++, and Fortran are all still important.  For instance, one
needs to show why one should switch to Java or some such.  Actually, on
performance grounds one *doesn't* switch to Java.  That's the point of
comparing them.  One can hope that either Java speeds up due to
competitive pressure, or there's some other language with better
performance than Java.

> step one is if we should have a ''standardized scoring
> system'' or keep craps.
> if we go for a standrad system, then we have a substantial amount of
> work cout of for us. namely, how/what to standardise.
>
> i am lazy. i prefer craps.

Within reason, I can't abide laziness as a design criterion.  I mean
really, you either want to accomplish something with the Shootout or you
don't.  I'm getting the feeling that you think the Shootout 'is pretty
neat' and that's where you're going to leave it.  It is somehow
fulfilling your goals as it stands.  It doesn't fulfil any of my
commercial goals, other than being
yet-another-data-point-somewhere-on-the-internet.

Pretty much the choice is between the Shootout as it is now, or a
Shootout more in line with commercial benchmarks like Viewperf,
http://www.spec.org, Futuremark, various scoring systems used in
computer magazines, etc.  What's it gonna be?


Cheers,                         www.indiegamedesign.com
Brandon Van Every               Seattle, WA

"We live in a world of very bright people building
crappy software with total shit for tools and process."
                                - Ed McKenzie