[Shootout-list] Directions of various benchmarks

[skaller]

On Wed, 2005-05-18 at 16:25, Jos=C3=A9 Bollo wrote:
> Le mercredi 18 Mai 2005 08:06, Bengt Kleberg a =C3=A9crit :
> > Jos=C3=A9 Bollo wrote:
> > ...deleted
> >
> > > More generaly i dont care about LOC
> > > For me the more readable and the more understandable it is, the bes=
t it
> > > is It would be great to have a reverse sort on LOCs: the more is th=
e best
> > > ;-)
> >
> > imho it is a good idea to care about loc.
> > the amount of bugs is proportional to the amount of code.
> > less code =3D> less bugs.
>=20
> yes at a constant programming language.
>=20
> but they are not comparable.

Actually they ARE comparable, research has actually been done on this.
It shows (from memory of a second hand account :) that the maintenance
cost and bug rates for ALL code depended on the number of lines of code,
independently of language. Which means that more compact languages
have less bugs 'per unit of functionality'.

> and remind you that most of EIFFEL programs include contracts that cost=
 LOC=20
> but that reduce in a huge way the count of bug, that improve in a huge =
way=20
> the documentation of the developed systems, and that reduce in a huge w=
ay the=20
> debugging time.=20

Which are all very good features of Eiffel, should be used, and should
somehow be rewarded rather than penalised.

One way to do that is require all language to state the=20
equivalent things, in comments if they don't have formal syntax
for specifying contracts.

> but remember the poor accuracy of such a concept as LOC and the relativ=
ity of=20
> many conviction.

Yes but the 'performance' measurements are just as difficult.
On the old shootout I see Felix beating Java on every test --
as it dang well should, Java is a dog <hehe> -- and suddenly on the new
Shootout Java is beating the Ocaml native code compiler???=20

Come on! I did some tests and find Felix comfortable trashes
gcc on some tests, but gcc with a couple of optimisation switches
trashes Felix .. what are we *really* measuring?

These are rhetorical questions .. the point is that I don't see why
measuring complexity of the code is any harder to get totally wrong
than measuring speed 1/2 ;(

--=20
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850,=20
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net