[Shootout-list] Many tests cheat

[skaller]

On Wed, 2004-11-03 at 02:31, Isaac Gouy wrote:
 
> > The benchmark obviously must be changed so the IO speed isn't
> > relevant -- right now several benchmarks measure nothing
> > else.
> 
> IO speed isn't relevant because...?

.. the stated goals of the tests are to measure something
different. The 4K limit is there to try to elide I/O
speed as a factor, to create a level playing field,
but also to impose a constant memory requirement.

I have no problem at all with the motivation
for the constraint.

This can affect other things. If I can give an example,
currently Felix can tokenise a string very fast with
moderately simple code using the construction

	reglex p1 to p2 ...

This construction requires p1,p2 to be char* into memory.
A requirement with any kind of memory constraint forces
me to do buffer handling manually .. when I'm finished
parsing a particular block, I have to load another one
and continue on. This is not so easy -- I have to
preserve the automaton state, and I have to accept 
that the lexed string may be split across buffers.

There is no easy way to meet the first requirement at present,
I will actually *have* to modify the compiler to allow it.
Lexemes split across buffers are messy but doable.

So, the constraint is *good* in principle because in this
kind of case it shows a deficiency in Felix -- Ocaml
lexers use a buffer object which already handles this
transparently. Of course the Ocaml lexers will load
one or more blocks .. unless you look at the implementation
code and the test data you can't tell how many are buffered,
they may be 4K each, which means an Ocamllex solution
would fail the 4K buffer restriction. 

But Ocamllex would pass the constant memory constraint:
it only buffers enough to hold each lexeme in memory.
For small lexemes that's 2 buffers, for large ones
1 buffer more than required to hold the lexeme.

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net