[Shootout-list] External Libraries

[Brent Fulgham]

Until recently, I was safely able to ignore the whole
question of external libraries.  However, it appears
that it is time for us to decide how we want to handle
this.

On the one hand, reusable libraries are the
cornerstone
of Software Engineering.  It makes no sense to
reinvent
the wheel when excellent tools exist to solve our
problems.

However, the language shootout is designed to show
the performance of different languages for a variety
of problem types.  We don't really learn much we end
up with a bunch of entries calling the same external
library to present the results.

So, I propose the following:

1.  All shootout solutions must be written using the
native provisions of the language (no external
libraries).

2.  Solutions using external libraries are encouraged,
and will be shown on the site as "Interesting
Alternative" solutions (see
http://shootout.alioth.debian.org/benchmark.php?test=nbody&lang=all&sort=fullcpu
for an example of how this will look).

Given those two rules, we have a few complications:

a)  Many language use an external library (such
as libgmp) internally.  For example, I know this is
the case with MzScheme (it is built using libgmp).

b)  What about external libraries written in the
host language (e.g., Lua's BigInt library)?

c)  What about external libraries that are simple
wrappers around a non-host language core (e.g.,
Pythons NumPy extension?)

My judgement on these three cases are:

*  Languages that fall into category (a) luck out --
if the source for the solution does not load an
external library (of any kind) the solution is
perfectly fine.

*  Solutions that fall into category (b) are also
fine.  The only 'cheat' is in lines of code, and
that is pretty meaningless anyway.

*  I'm afraid category (c) should be placed in the
"Interesting Alternative" category, because they
provide their performance by exiting the host
language and running assembly or some other language.

===============================================

So, what does this mean?

Well, at least a few existing solutions need to
move into the "Interesting Alternative" category.
I know that Clean's regexp solution calls out to
'libpcre', and D's pidigits using 'libgmp'.  There
are probably others.

I know that these (arbitrary) rules are not indicative
of how a language would generally be used in the
real world.  However, I'm trying to balance two
differing uses for the shootout:  (1) as an objective
measure of the performance of languages for different
categories of problems, and (2) as examples of how
a given language can be used to solve a particular
problem.

I think (1) is best served by restricting solutions
to those in the host language.  I think (2) is
satisfied by showing "Interesting Alternatives" to
a problem.

I invite your comments (and criticism) on this
approach.

Thanks,

-Brent