[Shootout-list] Re: [Gd-chatter] examples/shootout strcat.dylan, NONE, 1.1

[Einar Karttunen]

On 14.06 12:21, Andreas Bogk wrote:
> What I noticed is that for some of the tests, the startup time clearly
> dominates the runtime (for instance with the Ackermann test).  The
> page says that the startup time is subtracted, but it doesn't really
> look like it from the figures.  Maybe you should choose higher numbers
> for higher total runtime there.

I was thinking about the initial costs in relation with Erlang. It is 
clear that with test that take less than a second startup time is an
important factor. When doing the consumer/provider test I could get
results where C is ten times better with a small N and results where
erlang is ten times better with a large N. This clearly makes the
results very dependent on the N.

Would it help that in addition to calculating cpu and memory usage one 
would calculate for each benchmark 

dcpu = test_cpu - hello_cpu
dmem = test_mem - hello_mem

Where the hello_* would be the results of the hello test. These should 
not replace the current scores but rather be additions. Of course one 
would need to care that hello world test samples would remain as small
as possible.

- Einar Karttunen