[Shootout-list] commercial benchmarks

[Brent Fulgham]

On 2004-09-26 21:15:50 -0700 Brandon J. Van Every 
<vanevery@indiegamedesign.com> wrote:

>> Brandon, simply declare a new industry standard *MAIN BENCHMARK* as
>> tests x,y,z and publish.
> 
> To pursue the marketing agenda I think proper, I may very well have to
> do that.  I'm not convinced that you, or other parties here, have much
> interest in professional quality, commercially viable benchmarks.  A 
> lot
> of things need to happen to make a benchmark visible and important to
> PHBs.  I'm not sensing any basic will - or even basic understanding -
> around here about how to take that on.  I think the Shootout is likely
> to remain 'hacker ken'.  It may convert some guys who play with
> languages in their spare time... it's not going to persuade anybody to
> change their business model.

A lot would have to happen to make the benchmark PHB-suitable.  For 
example,

1.  It's only kept up-to-date under Unix.  No volunteer has stepped 
forward to
  run the tests under Mac OS X or Win32.  This would be a big help.
2.  It doesn't have any corporate backer or other rubber stamp entity 
to provide
  the kind of glossy brochures PHB's like to line their nests with.
3.  It's a volunteer effort, so doesn't have any set publishing dates 
or language
  inclusion policies.  Basically, I include what I can easily install 
on my machine.
  I'm not overly concerned with dealing with companies and trying to 
extract
  licenses to allow me to run tests.  Microsoft's C# license actually 
prohibits someone
  like me from running benchmarks on their software.

I'm not exactly sure which of these items (or perhaps some other items 
you have
identified that I didn't cover) you are proposing to address (if any).

-Brent

>>> never done anything as loosey-goosey as leave the scoring system up
>>> to the person running it.
>> 
>> Could that be because they make money from defining and measuring
>> benchmarks?
> 
> Yes, everyone in the OpenGL ARB has a financial stake in the results 
> of
> the benchmarks.  This is a big difference between professionals and
> hackers.  For my part, I'm interested in job creation with these 
> various
> unpopular languages.  Otherwise the PHBs tyrranize us with C++, Java,
> and C# forever.  I want to make lotsa money as a consultant using
> languages that I prefer, that are better suited to task.  That's my 
> own
> financial stake.  You may say "well just go do it then" but that is
> naive, considering how much mindshare C++, Java, and C# actually have.
> 
> If commercial concerns are not likely to be incorporated into the
> Shootout, then I'll bow out and not worry about how the Shootout
> evolves.  For all I know, someone on comp.benchmark may have a better
> marketing framework to suggest.  On the other hand, if someone else
> around here has similar concerns, now would be a good time to chime 
> in.
> 
> 
> Cheers,                         www.indiegamedesign.com
> Brandon Van Every               Seattle, WA
> 
> "We live in a world of very bright people building
> crappy software with total shit for tools and process."
>                                 - Ed McKenzie
> 
> 
> 
> _______________________________________________
> Shootout-list mailing list
> Shootout-list@lists.alioth.debian.org
> http://lists.alioth.debian.org/mailman/listinfo/shootout-list
>