[Shootout-list] CVS updates not visible on the website

[Einar Karttunen]

Isaac Gouy <igouy2@yahoo.com> writes:
> The immediate issue is that Brent is unavailable. 
> Let's hope he's enjoying some family time.

I was fearing that I was just doing something wrong.

> 1. I don't know how we can avoid someone looking at the code to check
> that it does indeed implement the benchmark, instead of returning the
> correct value from a print statement. There will always be a code
> inspection. Let's assume that whoever does CVS checkin takes that
> responsibility.

Trusting few people should be ok, but even verifying the code is quite
fast. 

> 2. We need computer time to run the benchmarks (Brent doesn't have a
> dedicated machine to run these benchmarks, does anyone else?)

A how fast machine do we need? 

> 4. We need to install new language implementations and update old
> implementations. That's always going to be time consuming. That's one
> of the issues with C# Mono and Fortran. 
> Figuring out how to build and run new language implmentations is always
> going to be time consuming.

True, this is going to get quite nasty. Of course we could require the
submitters to provide a script for building their favorite language...

btw I can do work on updating GHC to 6.4 when that gets released.
What else besides testing the build-process for all benchmarks and
providing commands to build it is needed?

> 1. Timestamp check for CVS content without measurement

I think that a single page listing benchmarks not yet measured would be
enough for me.

> Should we add alternatives rather than overwrite a program? Maybe the
> new version is slower on Brent's test machine!

Then the submitter will see that and hack an even better version in
future hopefully. I think this is a responsibility of the submitter.

- Einar Karttunen