[Shootout-list] Directions of various benchmarks

[Brent Fulgham]

--- Bengt Kleberg <bengt.kleberg@ericsson.com> wrote:
> i can help write the benchmark specifications. i
> only need to finish working on the most important 
> project for the shootout first. it is proving to be
> a lot more complicated than i thought to get more 
> ''n'' for each test.

How many different values of N would you like to
see for the tests?

Would some tests need more N than others (I would
imagine some tests need a higher resolution to show
variations in performance?

Can you (or perhaps everyone) give me a feel for how
much we are talking about.  It's no big deal to add
a few more iterations to the run (it will just mean
the shootout will take longer to run).

But there are practical limitations.  If we want
to have 1,000 different values of N for each test,
then we move from a shootout run that takes roughly
24 hours to something that could take weeks to run
(since run times increase as N increases.

-Brent