[Shootout-list] Java benchmarks

[James McIlree]

Isaac,

	The central problem here is that the shootout benchmarks
are very small and short.

	In general, the server compiler generates higher performance
code than the client compiler. It uses a lot of resources to do so,
though. In a benchmark that runs only a few seconds, or, as many of
the shootout benchmarks, fractions of a second, it just doesn't pay.

	To me, the best fix would be to have longer running benchmarks.

	Some of the C benchmarks can complete in less time than it takes
Java to reach main(). If the computing task at hand is really that
short, it just doesn't matter what language you write it in. Even
something 100x slower is fast enough.

	As a second choice, the IBM jdk would do nicely.

	As a third choice, run both server and client versions, as
separate languages. You've already got a large batch of java(s)
in the shootout, this would do to nicely confuse the mattter :-).

	James M

On Mar 28, 2005, at 10:44 PM, Isaac Gouy wrote:

> Could you have this conversation with Keith Lea, and then get back to
> us? He beat-us-up for using  the client compiler.
>
> http://kano.net/javabench/
>
>
> --- James McIlree <ovrskeek@mac.com> wrote:
>>
>> 	I'd like to raise an issue about the Java benchmarks. It looks to
>> me as if they are all being run with the "-server" option.
>>
>> 	Would it be possible to switch to the client compiler? I believe
>> most of the benchmarks would actually complete quicker using the
>> client
>> compiler.
>>
>> 	The shootout benchmarks are all so small and quick, the server
>> compiler simply does not have time to pay for itself.
>>
>> 	As a concrete example, here are timings for knucleotide:
>>
>> IBM java
>> 1.205u 0.073s 0:01.17 108.5%    0+0k 0+0io 0pf+0w
>>
>> Sun java (-client)
>> 1.291u 0.053s 0:01.38 97.1%     0+0k 0+0io 1pf+0w
>>
>> Sun java (-server)
>> 1.785u 0.061s 0:01.45 126.8%    0+0k 0+0io 1pf+0w
>>
>> 	As you can see, it would be really nice to have the IBM jdk
>> available for testing as well. It seems to be generally faster than
>> the Sun jdk.
>>
>> 	http://www-128.ibm.com/developerworks/java/jdk/linux140/
>
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
>
> _______________________________________________
> Shootout-list mailing list
> Shootout-list@lists.alioth.debian.org
> http://lists.alioth.debian.org/mailman/listinfo/shootout-list
>