[Shootout-list] X per second scoring system

[Brandon J. Van Every]

Bengt Kleberg wrote:
>
> so, i will now reformulate my previous attempt at stateing how many
> seconds we need for a good measurement:
>
> Start measuring OS slicing spikes.
>
> this statement does look odd to me, and i fail to grasp why
> it will let
> people with a human sense of time understand that i am
> willing to use 30 seconds. but i bow to your expertise.

You're going to have to actually measure stuff if you want an answer.
It's not a theoretical question, it's about what machines actually do in
practice.  I don't know how long a 'machine warm up' really needs to be.
I do know that given any granularity of sample, it's good to overkill it
by a factor of 1000 if you want accuracy.  So if your're experiencing
spikes on the order of every 0.1 second, you'd want to test for 100
seconds.  Maybe you could get away with 10 seconds / factor of 100,
maybe not.  You need to study your actual machine's behavior.  It's not
an inducting counting problem.  What if your machine burps every 20
seconds for some reason?

A good way to study these things with minimal pain is to start a process
that does some kind of measurement, then leave it running forever.
Watch as the various numbers tick by, notice if jumps happen every so
often.  Scratch your head and guess why.

> for each test there is a ''Click here for more detailed data
> and graphs.''
>
> have you not seen it?

I had not, actually.  Now I have, and I'll have to spend some time
contemplating what they say, when it isn't 6:30 AM and I haven't slept
yet.


Cheers,                         www.indiegamedesign.com
Brandon Van Every               Seattle, WA

20% of the world is real.
80% is gobbledygook we make up inside our own heads.