[Shootout-list] X per second scoring system, resume
Brandon J. Van Every
vanevery@indiegamedesign.com
Thu, 30 Sep 2004 14:59:56 -0700
Einar Karttunen wrote:
> Brandon J. Van Every wrote:
> >
> > You mean like, some language dies once N hits a certain
> > threshold? Or
> > do you mean small noisy variances? Noise isn't important.
>
> No I mean that cache effects (and GC) can play a quite large role.
> Cache performance will degrade as N becomes larger but it is very
> implementation defined how this will happen...
I guess we'll end up back at questions of 'what are we testing'. Maybe
it's time to talk about 'Simple Benchmark' and so forth again, instead
of all this.
> You can already measure that in a certain way. Look at the analysis
> with different N. *If* you presume that N does affect the
> running time
> with the function f, then you will have:
>
> f(x) = myfunc + (start + stop)
> f(1000) = value_1
> f(2000) = value_2
> f(3000) = value_3
>
> Now it should be trivial (knowing myfunc) to solve what
> start + stop is. I don't see a need to revamp the system...
You used the word "should." It is theory, not measurement. Anyways, I
already said someone could / should suggest the worst case language and
test for me to run an experiment on. As well as what will keep me from
coughing out the results immediately.
> Lets look at haskell for example (I don't think it is one of the worst
> ones).
>
> If the timer is implemented with a signal then you run into the
> following problems:
Why are we talking 'if' ? Does Haskell have a timer call available or
not? How is it implemented? Or is your point that Haskell is really
really lazy and flow control for something like a system timer is always
going to be really really weird? I imagine the Haskell community runs
benchmarks on something, how do they solve their problems?
> Now most of the haskell benchmarks don't use IO monad in the inner
> loop and would need to be rewritten. Did I mentioned that this will
> probably affect what will get inlined?
Are you saying Haskell is silly and has done nothing practical to make
it possible to accurately benchmark in the language?
> I think that the it is quite interesting *how* slow a language is, not
> just that it is slower than some other language.
It's all about Epsilons. Maybe you have a point about certain Epsilons,
maybe you are being overly anal.
Cheers, www.indiegamedesign.com
Brandon Van Every Seattle, WA
"The pioneer is the one with the arrows in his back."
- anonymous entrepreneur