[Shootout-list] ackermann, values for ''n''

[Bengt Kleberg]

On 2005-06-08 06:54, John Skaller wrote:
...deleted
> The Python code I recently posted avoids
> this problem in two distinct ways:
> 
> (a) it automatically seeks an appropriate time,
> allowing you to test run the benchmarking in 
> a smaller amount of time

would this mean that if a language is slow, then the automation would 
not use large ''n'' even for a fast langugae? or have i misunderstood?


> (b) if the test harness is only 90% right,
> there is a good chance that the tests that
> run will produce useful data, and that data
> is retained (all individual benchmarks individually
> contribute to the results)
> 
> (c) A test that fails 90% of the way into
> the process doesn't require rerunning that
> 90% of the process again. (the ordering
> is random)

another way of solving this problem would be to have a deterministic 
test process, but only re-run the missing 10%.
i am assuming that you mean that ''a test process'' is all the languages 
for a test, or all the tests for a language, etc. if you only mean a 
single language and a single test, then my statement is not applicable.


bengt