[Shootout-list] Updates for Haskell benchmarks.

Stephen Weeks sweeks@sweeks.com
Tue, 6 Jul 2004 17:21:31 -0700


> I think we should revise the shootout to require that all languages
> present solutions in their most idiomatic form, so that
> single-assignment languages aren't force to jump through hoops to
> give the illusion of update.
...
> So, we need to do the following:
>
> 1.  Modify those language entries that seem to be unidiomatic.
> 2.  Possibly extend the size of the data sets required to run the tests 
> so that enough time is required to provide meaningful measurements.

I think this is a fine goal.  I wonder what should be done in the
interim, however, with tests where one implementation is known to be
not performing the same computation as other implementations.  For
example, for regexmatch, the ghc code performs a computation once that
other implementations perform 9000 times.  I think it is quite
confusing to leave the data as is with such a known discrepancy.
Hence, I view any of the following as better than the current state of
affairs, until the above goal is reached.

* Put a hack in the ghc code to force it to compute <N> times.
* Disable the ghc code for the test.
* Disable the test altogether.