[Shootout-list] X per second scoring system, resume

[Brandon J. Van Every]

Einar Karttunen wrote:
> Brandon J. Van Every wrote:
> >
> > You mean like, some language dies once N hits a certain
> > threshold?  Or
> > do you mean small noisy variances?  Noise isn't important.
>
> No I mean that cache effects (and GC) can play a quite large role.
> Cache performance will degrade as N becomes larger but it is very
> implementation defined how this will happen...

I guess we'll end up back at questions of 'what are we testing'.  Maybe
it's time to talk about 'Simple Benchmark' and so forth again, instead
of all this.

> You can already measure that in a certain way. Look at the analysis
> with different N. *If* you presume that N does affect the
> running time
> with the function f, then you will have:
>
> f(x) = myfunc + (start + stop)
> f(1000) = value_1
> f(2000) = value_2
> f(3000) = value_3
>
> Now it should be trivial (knowing myfunc) to solve what
> start + stop is. I don't see a need to revamp the system...

You used the word "should."  It is theory, not measurement.  Anyways, I
already said someone could / should suggest the worst case language and
test for me to run an experiment on.  As well as what will keep me from
coughing out the results immediately.

> Lets look at haskell for example (I don't think it is one of the worst
> ones).
>
> If the timer is implemented with a signal then you run into the
> following problems:

Why are we talking 'if' ?  Does Haskell have a timer call available or
not?  How is it implemented?  Or is your point that Haskell is really
really lazy and flow control for something like a system timer is always
going to be really really weird?  I imagine the Haskell community runs
benchmarks on something, how do they solve their problems?

> Now most of the haskell benchmarks don't use IO monad in the inner
> loop and would need to be rewritten. Did I mentioned that this will
> probably affect what will get inlined?

Are you saying Haskell is silly and has done nothing practical to make
it possible to accurately benchmark in the language?

> I think that the it is quite interesting *how* slow a language is, not
> just that it is slower than some other language.

It's all about Epsilons.  Maybe you have a point about certain Epsilons,
maybe you are being overly anal.


Cheers,                     www.indiegamedesign.com
Brandon Van Every           Seattle, WA

"The pioneer is the one with the arrows in his back."
                          - anonymous entrepreneur