[Shootout-list] Timing resolution? libbsd-resource-perl

[Einar Karttunen]

On 07.08 08:48, Isaac Gouy wrote:
> > > Maybe we should increase N to get the fastest Shootout tests back
> > > into 0.1s 1s range.
> > 
> > Increasing N would be very nice, however it has two effects - 
> > tests take more time to run and the relative order of the entries
> > changes, which is even more important. 
> 
> There's nothing sacrosanct about the current relative order of the
> entries - the current size of N is arbitrary. 
> (Perhaps for some languages it would be better to have one fixed size
> loop, and one nested loop limited by N.)

Yes, I was referring more to the fact that getting rid of the 
"order changes as per N" would be nice. It is not about fixed
size loops, but about whether memory is allocated - how bug are
the setup vs iteration costs and so on.

That is for a typical loop test we have four costs:

1) runtime system startup - should be deleted from the score.
2) initialisation for the looping - may be very large.
3) N*actual loop costs.
4) garbage collection and other background tasks of the runtime system.

e.g. for language A) they might be (1) deleted:
2) 0.0 3) 0.1 4) 0 
and results: N=10 -> 1, N=100 -> 10, N=1000 -> 100

and for language B)
2) 10 3) 0.01 4) 0.01
N=10 -> 10, N=100 -> 11, N=1000 -> 20

Is A or B better in this test?

Deciding what to measure and how is more important than 
finicly resolution.

> Is it possible to increase the timing resolution on the test machine
> using libbsd-resource-perl?

I don't know but don't have anything against it.

- Einar Karttunen