Fwd: RE: [Shootout-list] main benchmark

[Brent Fulgham]

--- Brent Fulgham <bfulg@pacbell.net> wrote:

> Date: Mon, 27 Sep 2004 14:41:13 -0700 (PDT)
> From: Brent Fulgham <bfulg@pacbell.net>
> Subject: RE: [Shootout-list] main benchmark
> To: shootout@lists.alioth.debian.org
> 
> --- "Brandon J. Van Every" wrote:
> 
> > I have proposed, several times now, a framework. 
> > There should be:
> > 
> > - a main, least common denominator benchmark for
> all
> > languages
> 
> This is where our communication is breaking down.  I
> want to know what you view as a
> least-common-denominator benchmark.  You have told
> us
> what is *not* in the LCD -- bignums and apparently
> now
> string functions.  Why don't you say what *is* in
> this
> set.  Simple math?  file I/O?  What goes in and
> stays
> out?
> 
> Previously I tried to address your issue by listing
> each test and what its aim was.  Bengt did a much
> better job of this with his breakdown.
> 
> Assuming we agree that concurrency is not a LCD, and
> I
> think we DO agree that object oriented tests should
> not be included, which remaining tests should be
> retained for BARC, and which should be left for
> BITE?
> 
> > - a garbage collection benchmark
> 
> We don't currently test this explicitly.  Anyone
> have
> ideas on how best to measure this one issue?
> 
> > > I have every reason to believe that this kind of
> > > work has value to the world at large, and have 
> > > not seen any convincing argument to the
> contrary.
> > 
> > I think you're ignoring the arguments presented to
> > you because you don't value them.
> 
> Perhaps.  But so far you are the only voice from
> this
> perspective.  I want the shootout to be as fair and
> unbiased as possible, but I don't want to change
> things just for change's sake, or just to satisfy
> one
> person's requests.
> 
> I don't think we take issue with most of your
> suggestions regarding the idea of "core" tests
> versus
> "language-specific" or ancillary tests, but I don't
> know that we all agree on what belongs in each area.
> 
> For example, I don't believe that our tests
> currently
> require garbage collection, or type safety, or
> object
> oriented features.  Languages that happen to have
> these features are just able to implement some of
> the
> tests more easily/efficiently than, for example, C,
> which has to build this plumbing out of basic
> elements.
> 
> -Brent
>