[Shootout-list] Re: language shootout - python with psyco?

[Brent A. Fulgham]

TY Chan wrote:

>Brent,
>
>Thanks for reviving the shootout!
>
>It'd be interesting to see the timings for python with psyco. The latter is
>a JIT specializing compiler that, on occasion, massively speeds things up.
>For short programs, all one has to do is add two lines:
>
>import psyco
>psyco.full()
>
>The tradeoff is that it uses more memory than the original program.
>
>http://psyco.sourceforge.net/
>http://reypastor.hispalinux.es/~setepo/psyco/test.html
>
>The latter link has some partial shootout results.
>
>
>Best regards,
>Jeff
>  
>

I gave this a quick shot, but found there was little difference in the 
results because startup times are too significant:

Currently tests are done by firing off the command with a parameter.  
For python we byte-compile before running (excerpt from Makefile):

########################################
# Python
########################################
%.py: $(MB_SRCDIR)/%.python $(PYTHON)
        -rm $(TEST).pyo
        cp $< $@

.PRECIOUS: %.pyo

%.pyo: %.py
        $(PYTHON) -OO -c "from py_compile import compile; compile('$<')"

%.python_run: %.pyo
        @:

This just copies "test.python" from the source directory and calls it 
"test.py", then it builds "test.pyo" which it then runs.

So for Psyco I tried this:

########################################
# Psyco
########################################
%.py: $(MB_SRCDIR)/%.psyco $(PYTHON)
        -rm $(TEST).pyo
        cp $< $@

.PRECIOUS: %.pyo

%.pyo: %.py
        $(PYTHON) -OO -c "from py_compile import compile; compile('$<')"

%.psyco_run: %.pyo
        @:

(I.e., destroy any old build, copy "test.psyco" to "test.py", compile it and
then run the resulting test.pyo).

Each test was modified to add:

import psyco
psyco.full()

At the head of the test.

I think the compilation is not happening properly, so there is no 
performance boost.  Could profiled data from perhaps a test run be used 
to produce a valid psyco module that could then be loaded later and run 
in head-to-head competition with the others?

The page you site uses a special script that times the compiled 
function.  But this is not the way the current shootout is structured 
(though perhaps it should be).  To fairly rate psyco, we need to emit 
the JIT code into a module we can load at run-time.

Any ideas on how to restructure tests to deal with start-up times would 
be a big benefit.  There are plenty of "large" environments that suffer 
from startup times (e.g., Erlang, Poplog, Mozart/Oz) but are quick in 
real applications.

On the other hand, there are plenty of environments that do manage to 
provide quick executables (such as CMUCL) even in the presence of their 
large environments.

I'd appreciate any thoughts...

Thanks,

-Brent