[Shootout-list] Stuff

[Jon Harrop]

On Friday 22 April 2005 12:14, Bengt Kleberg wrote:
> > Regarding some other discussions I've read about in the archives. I think
> > it is vitally important to allow people to use common libraries (and to
> > not count the library as submitted code). For example, lots of scientific
>
> this is a good idea, but awaits the rules for what to allow, and what to
> disallow.

Yes. I like Brent's idea of allowing only libraries which have Debian packages 
as, for me also, this is highly relevant. :-)

> at one extreme we would have all languages (with a ffi) calling a
> purpose built c program to be as fast as possible.

IMHO, the solution is to define problems which are not implemented entirely in 
a library. This is easy in the case of LAPACK and FFTW as lots of real 
problems can productively exploit these libraries whilst also doing useful 
other stuff. We could explicitly state things like "FFT can use a library but 
the rest must be coded in the host language".

> > Reading the March archive I've come up with a few more ideas for
> > benchmarks:
>
> ...deleted many fine examples
>
> presumably this is in faq, but i think i saw a ''no more than 100-lines
> of code'' soft limit on the benchmark programs on the list.
> would not these programs make a lot of languages exceed this limit?

Well, 100 lines of code in what language? I've heard C#/Java but why not <100 
LOC in the 3 most succinct languages for each test? I can write a ten line 
OCaml program which requires several thousand lines of Fortran to implement 
equivalently. I believe that displaying this kind of knowledge is one of the 
main practical applications of the shootout.

I think many of those examples can be coded very succinctly. Especially if you 
are allowed to the use appropriate tools, as you would if you were coding to 
solve a real-world problem.

> >>These files would then be built by the respective
> >>programs for that language implementation.
>
> presumably this too is in the faq, but i think i saw a ''only one file
> per benchmark'' rule on the list.
> would not these 3 files exceed this limit?

Someone's going to have to change the FAQ then. :-)

> > I like this, provided you're talking about the lex and yacc sources (.mll
> > and .mly) and not the compiled ".ml", which would be huge! :-)
>
> given that the file limit problem is none existent, i wonder if the .ml
> creation is counted towards the run time of the benchmark? or towards
> the compile time? and should it matter? (yes, it is in the faq, i know.
> i will look there).

Definitely part of compilation, I'd say. Incidentally, can we add compile time 
to run time, memory use and LOCs?

While we're there. Can we make it easy to select a subset of tests which are 
relevant for scientists? Also, can we spell integrated correctly in the 
description of OCaml? :-)

-- 
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
Objective CAML for Scientists
http://www.ffconsultancy.com/products/ocaml_for_scientists