[Shootout-list] Safety vs. speed

[Brent Fulgham]

On 2004-09-21 21:32:33 -0700 Brandon J. Van Every 
<vanevery@indiegamedesign.com> wrote:

> Peter Hinely wrote:
>> 
>> I think it would be good to have a column on the shootout indicating
>> whether the entry is "safe" or not.
> 
> I am becoming concerned that people aren't terribly focused on the 
> core
> mission of the Shootout.  The core mission is, or at least 
> historically
> has been, performance.

No.  The core mission has always been to have fun.

> We can't spare people an understanding of safety concerns by adding up
> some number about them.  There's no benchmark you can run about 
> safety.
> Any number or weight you might assign to it is just someone's
> gobbledygook opinion, it's not measureable.  Performance benchmarks 
> are
> enough of a lie as it stands; can we please avoid even more blatant
> lies?

I disagree.  Too many good language implementations are ignored for 
fear
that they won't measure up to C/C++, or don't have the standards that 
Java
has, or any number of other unimaginative reasons.

My vision of the shootout is for a place where people can see how all 
of
these neat languages compare across a broad spectrum of tasks.  If all
you care about is raw speed, you can remove the tests that don't show 
your
desired result and thereby cook the books in favor of C or assembly, or
whatever.  If you care about writing real-world applications that need 
to
scale across large numbers of users, you might find the concurrency 
tests
to be of some interest.  I plan on adding some search/logic style 
tests so
we can see whether a hand-coded backtracker in C is reasonable when
trying to solve an N-Queens or similar problem, as compared to 
something
with built-in search such as Oz.

But my point is that these are *task* based tests.  I'm not overly 
concerned
with validating one language's safety against another, because I don't 
know
how we would measure this.  I do think it's reasonable to ask to see 
how well
languages can compete against each other for a particular goal.  But I 
agree
with you that It's less clear how to measure whether SML's superior 
type checking
leads to safer or more robust programs.

I'm afraid the best we can do is note the safety features of these 
languages
in the notes about the version/implementations.  I allow languages to 
run
with all checks turned off, because that's what you get with C or 
assembly,
which would be the main target for performance.

If you can show that SML is able to produce blazing fast programs, or 
that
Eiffel can be just as fast as C when the assertions are turned off, 
then arguments
that this or that languages cannot compete in a commercial environment 
are
shown to be prejudice and hot air.

-Brent