[Shootout-list] Numerical medium-sized benchmarks

[Brent Fulgham]

--- Sebastien Loisel <sloisel@gmail.com> wrote:
> > I am attaching an example (I wrote these three
> > programs.) This example is drawn from the SIAM
> > 100 digit challenge
> >
http://web.comlab.ox.ac.uk/oucl/work/nick.trefethen/hundred.html
> > question 3. Our team had two solutions, this is
> > the more brute-forceish one. The problem is to 
> > compute ten significant digits of the norm of a 
> > certain infinite matrix. 
> 
--- Isaac Gouy <igouy2@yahoo.com> wrote:

> Hmmm, is there a MathWorld explanation for the
> mathematically illiterate (like me)?

I'm no math world, but the problem statement looks
to be an answer to the Matrix test I was complaining
to you about.

If my quick glance at the problem is accurate,
it seems to test manipulation of an infinite
matrix (defined by a bounding function) to
identify its determinate (which means that it
boils down to a nice big fat number we can
easily compare with a known good result to
verify the test.)

I like it! :-)


> > Please also consider the numerical integration
> > code at
http://www.math.mcgill.ca/loisel/numerical-integration.html

I like this test because it is (a) hard work for
the compilers, and (b) can have a varying 'N' to
allow us to ramp-up the test difficulty.
 
> Where can we see a "math for dummies" explanation of
> the problem?

I'm not sure if we have an analog to this test,
since it is a larger 'meta' problem.  We aren't
testing the efficiency of array operations, or
string concatenation.  Instead, the programs are
doing 'real' work.

> We're trying to keep to a smallish number of
> benchmarks - so for every problem we add to the 
> Shootout, we look for a problem to remove. Which
> problems do you think could be replaced by the
> benchmarks you've suggested?

I would suggest jettisoning our existing matrix
test and just substitute Sebastien's determinate
test.

I'm conflicted about whether we should just add
the second test, or replace one of the purported
'floating point' tests with it.

I believe it tests more than just floating point
math, and is more of a 'whole program' compiler
test, rather than just the narrow focus of something
like the harmonic test.

I would suggest we add it as a new test.

In the same vein, I would like to look at adding
some of Jon Harrop's ideas (perhaps something like
the travelling salesman or N-closest neighbors
test), since we don't have any optimization/search
tests (an area I want to remedy).

-Brent