[Shootout-list] Question regarding time measurement

[Stefan Krause]

Don't want to wake sleeping dogs [and maybe I'm taking it far too serious],
but anyway I'd like to know:
How is time measured for the tests? Something like the linux command time?
(I guess so, since there's no measurement in the code itself) This might be
quite a disadvantage for all interpreted or byte code (instead of machine
code) languages. When I look at some benchmarks the time to run the test is
so short that the loading of the virtual machine or interpreter takes way
longer than the computation. This would render the results a bit useless. 
Including the time for the vm or interpreter itself is OK and not warming up
is also OK (if the average user wants to know about the average performance
and not the peek performance), but a test like matrix multiplication with
0.0360 secs for c++ is too short. This will only return information about
the loading performance not about numeric performance.  
Would you agree (at least partially ;-) with those complaints? 
Would it be viable to increase the running time for the test to something
like > 15 secs? (The scimark2 benchmark might serve as a nice example)

Thanks,
Stefan