[Pkg-exppsy-maintainers] Segfault example

Per B. Sederberg persed at princeton.edu
Thu Jan 17 13:34:36 UTC 2008 

On Jan 16, 2008 7:55 PM, Yaroslav Halchenko <debian at onerussian.com> wrote:
> cool... so I think now it is even more important to file a but against
> numpy ;-) good catch!
>

Yup, numpy will be the recipient.  I'm trying to get an example that
breaks it with numpy-generated data so that I don't have to provide
data to the folks who will try to fix the problem.

> As for confusion -- you hit a stone -- time to refactor... I think we
> might better have Classifier and Regressor (both childs of Machine). It
> gets convoluted at SVM interface a bit since SVMBase does everything but
> things might change ;)
> then Regressor's wouldn't have confusion matrix at all, but should have
> some error associated to keep training error (RMSE most common right?)
>

RMSE might be a bit harsh, but would be fine.  Another possibility is
correlation.

Any thoughts for how to deal with the SVMBase class, which does it all :)

Best,
P


>
> On Wed, 16 Jan 2008, Per B. Sederberg wrote:
>
> > Now here's a new turn!
>
> > If I change over to using the scipy.linalg.lstsq instead of
> > numpy.linalg.lstsq it works on my machine!
>
> > So, that really points to a numpy bug.
>
> > I'm switching the code in pymvpa and will test some more.
>
> > It also doesn't make sense to run a confusion matrix on the ridge
> > regression so I'm going to turn that off for that classifier.
>
> > Latro,
> > P
>
> > On Jan 16, 2008 5:49 PM, Yaroslav Halchenko <debian at onerussian.com> wrote:
>
> > > > I'll file that bug and see if anyone responds.
> > > yeah... buzz us again if nothing happens... we might do some bughunting
> > > party ;-)
>
> > > > Yes, my new laptop can run amd64 (Intel Core 2 Duo), but I didn't put
> > > > / in a logical drive, so it'll be a bit more difficult to do the
> > > > reinstallation.  I sure would like to start over, but I have a pretty
> > > > happy system right now and I'm reluctant to start fiddlin'.
> > > don't you have any free space left? I am not sure if you have LVM or
> > > not... if there is a partition with lots of space you can easily shrink
> > > it ;-)
>
> > > or another approach would simply be -- install amd64 kernel (instead of
> > > default i686 which comes with i686 arch) from amd64 architecture (just
> > > --force-architecture... and with 2.6.24 I think there would be no
> > > difference between the two so it would not be even needed to mangle that
> > > way) and install amd64 chroot anywhere where you have space. This way
> > > you will have fully functional amd64 environment
>
> > > > And yes, it's Debian (Lenny for the most part) and I don't plan on
> > > > going back to Ubuntu :).  I could use some more ammunition to convince
> > > > others in my lab, but at least I'm in Debian-land.
> > > that is great that you got Debian working and I hope it wasn't too much
> > > of hassle (besides some peculiarities).
> > > At least with Debian you will get the freshest releases of what we do
> > > for Debian ;-)
>
>
> > > > P
>
>
> > > > > On Wed, 16 Jan 2008, Per B. Sederberg wrote:
>
> > > > > > Here ya go:
>
> > > > > > $ dpkg -l refblas3 lapack3
> > > > > > ||/ Name                Version             Description
> > > > > > +++-===================-===================-======================================================
> > > > > > ii  lapack3             3.0.20000531a-6.1   library of linear algebra
> > > > > > routines 3 - shared version
> > > > > > ii  refblas3            1.2-8               Basic Linear Algebra
> > > > > > Subroutines 3, shared library
>
>
>
> > > > > > On Jan 16, 2008 5:17 PM, Yaroslav Halchenko <debian at onerussian.com> wrote:
> > > > > > > oppa - so what libraries of lapack/blas are used?
>
> > > > > > > although now it seems that the main possible factor is an architecture
> > > > > > > -- we have it failing on i386 but not on amd64... let me also give a
> > > > > > > spin on ia64
>
>
> > > > > > > On Wed, 16 Jan 2008, Per B. Sederberg wrote:
>
> > > > > > > > Here's mine:
>
> > > > > > > > $ uname -a
> > > > > > > > Linux flow 2.6.23-1-686 #1 SMP Fri Dec 21 13:57:07 UTC 2007 i686 GNU/Linux
>
> > > > > > > > $ dpkg -l python-numpy atlas3-base python2.4  libc6
> > > > > > > > Desired=Unknown/Install/Remove/Purge/Hold
> > > > > > > > | Status=Not/Installed/Config-files/Unpacked/Failed-config/Half-installed
> > > > > > > > |/ Err?=(none)/Hold/Reinst-required/X=both-problems (Status,Err: uppercase=bad)
> > > > > > > > ||/ Name                Version             Description
> > > > > > > > +++-===================-===================-======================================================
> > > > > > > > un  atlas3-base         <none>              (no description available)
> > > > > > > > ii  libc6               2.7-5               GNU C Library: Shared libraries
> > > > > > > > ii  python-numpy        1:1.0.4-5           Numerical Python adds a
> > > > > > > > fast array facility to the Pyt
> > > > > > > > ii  python2.4           2.4.4-7             An interactive high-level
> > > > > > > > object-oriented language (ve
>
>
> > > > > > > > P
>
> > > > > > > > On Jan 16, 2008 5:06 PM, Yaroslav Halchenko <debian at onerussian.com> wrote:
> > > > > > > > > What I meant is smth like
>
> > > > > > > > > $> uname -a
> > > > > > > > > Linux belka 2.6.22-2-686 #1 SMP Fri Aug 31 00:24:01 UTC 2007 i686
> > > > > > > > > GNU/Linux
>
> > > > > > > > > *$> dpkg -l python-numpy atlas3-base python2.4  libc6
> > > > > > > > > ii  atlas3-base    3.6.0-20.6     Automatically Tuned Linear Algebra Software,
> > > > > > > > > ii  libc6          2.7-2          GNU C Library: Shared libraries
> > > > > > > > > ii  python-numpy   1:1.0.3-1      Numerical Python adds a fast array facility
> > > > > > > > > ii  python2.4      2.4.4-5        An interactive high-level object-oriented la
>
> > > > > > > > > I will check it on amd64 box too...
>
>
> > > > > > > > > On Wed, 16 Jan 2008, Per B. Sederberg wrote:
>
> > > > > > > > > > Veeerrry interesting!
>
> > > > > > > > > > I, too, ran on a i386 machine, but Michael's was running amd64.  My
> > > > > > > > > > numpy version is:
>
> > > > > > > > > > $ python2.4
> > > > > > > > > > Python 2.4.4 (#2, Jan  3 2008, 13:36:28)
> > > > > > > > > > [GCC 4.2.3 20071123 (prerelease) (Debian 4.2.2-4)] on linux2
> > > > > > > > > > Type "help", "copyright", "credits" or "license" for more information.
> > > > > > > > > > >>> import numpy
> > > > > > > > > > >>> numpy.version.version
> > > > > > > > > > '1.0.4'
>
>
> > > > > > > > > > Thanks for trying it out.  I have yet to find anything on the web
> > > > > > > > > > about this, but will let you know if I do.
>
> > > > > > > > > > Thanks,
> > > > > > > > > > Per
>
>
> > > > > > > > > > On Jan 16, 2008 4:54 PM, Yaroslav Halchenko <debian at onerussian.com> wrote:
> > > > > > > > > > > Sorry I didn't follow up -- I did run and reproduced -- it puked many times and
> > > > > > > > > > > at the end with (which is I think what you got also). I was going to debug it a
> > > > > > > > > > > bit more but yet had a chance... will do later...  From Michael we should get
> > > > > > > > > > > versions of involved packages (numpy, blas, lapack) to see what makes the
> > > > > > > > > > > difference... Also - I ran it on i386 box...  what did you run it on?
>
> > > > > > > > > > > ==10429== Invalid read of size 4
> > > > > > > > > > > ==10429==    at 0x503DD6D: ATL_dcopy_xp1yp1aXbX (in /usr/lib/atlas/libblas.so.3.0)
> > > > > > > > > > > ==10429==  Address 0x5A8B5A0 is 8 bytes after a block of size 880,000 alloc'd
> > > > > > > > > > > ==10429==    at 0x40244B0: malloc (vg_replace_malloc.c:149)
> > > > > > > > > > > ==10429==    by 0x470FCA0: (within /usr/lib/python2.4/site-packages/numpy/core/multiarray.so)
> > > > > > > > > > > ==10429==    by 0x4711EF3: (within /usr/lib/python2.4/site-packages/numpy/core/multiarray.so)
> > > > > > > > > > > ==10429==    by 0x4712071: (within /usr/lib/python2.4/site-packages/numpy/core/multiarray.so)
> > > > > > > > > > > ==10429==    by 0x80B9C49: PyEval_EvalFrame (in /usr/bin/python2.4)
> > > > > > > > > > > ==10429==    by 0x80BB0E4: PyEval_EvalCodeEx (in /usr/bin/python2.4)
> > > > > > > > > > > ==10429==    by 0x80B9319: PyEval_EvalFrame (in /usr/bin/python2.4)
> > > > > > > > > > > ==10429==    by 0x80B99C3: PyEval_EvalFrame (in /usr/bin/python2.4)
> > > > > > > > > > > ==10429==    by 0x80B99C3: PyEval_EvalFrame (in /usr/bin/python2.4)
> > > > > > > > > > > ==10429==    by 0x80BB0E4: PyEval_EvalCodeEx (in /usr/bin/python2.4)
> > > > > > > > > > > ==10429==    by 0x80BB156: PyEval_EvalCode (in /usr/bin/python2.4)
> > > > > > > > > > > ==10429==    by 0x80DDF79: PyRun_FileExFlags (in /usr/bin/python2.4)
> > > > > > > > > > > --10429-- VALGRIND INTERNAL ERROR: Valgrind received a signal 11 (SIGSEGV) - exiting
> > > > > > > > > > > --10429-- si_code=1;  Faulting address: 0x4;  sp: 0x6250EDBC
>
> > > > > > > > > > > valgrind: the 'impossible' happened:
> > > > > > > > > > >    Killed by fatal signal
> > > > > > > > > > > ==10429==    at 0x3801FFEA: unlinkBlock (m_mallocfree.c:314)
> > > > > > > > > > > ==10429==    by 0x380208A0: vgPlain_arena_free (m_mallocfree.c:1162)
> > > > > > > > > > > ==10429==    by 0x38036501: vgPlain_cli_free (replacemalloc_core.c:108)
> > > > > > > > > > > ==10429==    by 0x38001A1B: die_and_free_mem (mc_malloc_wrappers.c:111)
> > > > > > > > > > > ==10429==    by 0x38036C82: do_client_request (scheduler.c:1158)
> > > > > > > > > > > ==10429==    by 0x380385A0: vgPlain_scheduler (scheduler.c:869)
> > > > > > > > > > > ==10429==    by 0x38058946: run_a_thread_NORETURN (syswrap-linux.c:87)
>
> > > > > > > > > > > sched status:
> > > > > > > > > > >   running_tid=1
>
> > > > > > > > > > > Thread 1: status = VgTs_Runnable
> > > > > > > > > > > ==10429==    at 0x40240CA: free (vg_replace_malloc.c:233)
> > > > > > > > > > > ==10429==    by 0x505495E: ATL_dmmJIK (in /usr/lib/atlas/libblas.so.3.0)
>
>
>
>
> > > > > > > > > > > On Wed, 16 Jan 2008, Per B. Sederberg wrote:
>
> > > > > > > > > > > > Hi Yarik:
>
> > > > > > > > > > > > Did you have the chance to run the ridge regression example code that
> > > > > > > > > > > > is breaking on my machine?  Michael ran it on his machine and could
> > > > > > > > > > > > not reproduce the error.
>
> > > > > > > > > > > > This is getting quite weird and I'm not sure where to go from here.
> > > > > > > > > > > > It breaks at the same point (returning from the _train call) for all
> > > > > > > > > > > > of my actual data.  The call to the numpy.linalg.lstsqr runs with no
> > > > > > > > > > > > error and returns the correct values while inside the _train call, but
> > > > > > > > > > > > all goes haywire upon the return from that method when the python
> > > > > > > > > > > > garbage collection kicks in.
>
> > > > > > > > > > > > Any thoughts on how to proceed are more than welcome from all.
>
> > > > > > > > > > > > Thanks,
> > > > > > > > > > > > Per
>
>
> > > > > > > > > > > --
> > > > > > > > > > > Yaroslav Halchenko
> > > > > > > > > > > Research Assistant, Psychology Department, Rutgers-Newark
> > > > > > > > > > > Student  Ph.D. @ CS Dept. NJIT
> > > > > > > > > > > Office: (973) 353-5440x263 | FWD: 82823 | Fax: (973) 353-1171
> > > > > > > > > > >         101 Warren Str, Smith Hall, Rm 4-105, Newark NJ 07102
> > > > > > > > > > > WWW:     http://www.linkedin.com/in/yarik
>
> > > > > > > > > > > _______________________________________________
> > > > > > > > > > > Pkg-exppsy-maintainers mailing list
> > > > > > > > > > > Pkg-exppsy-maintainers at lists.alioth.debian.org
> > > > > > > > > > > http://lists.alioth.debian.org/mailman/listinfo/pkg-exppsy-maintainers
>
>
>
> > > > > > > > > --
>
> > > > > > > > > Yaroslav Halchenko
> > > > > > > > > Research Assistant, Psychology Department, Rutgers-Newark
> > > > > > > > > Student  Ph.D. @ CS Dept. NJIT
> > > > > > > > > Office: (973) 353-5440x263 | FWD: 82823 | Fax: (973) 353-1171
> > > > > > > > >         101 Warren Str, Smith Hall, Rm 4-105, Newark NJ 07102
> > > > > > > > > WWW:     http://www.linkedin.com/in/yarik
>
>
>
> > > > > > > --
>
> > > > > > > Yaroslav Halchenko
> > > > > > > Research Assistant, Psychology Department, Rutgers-Newark
> > > > > > > Student  Ph.D. @ CS Dept. NJIT
> > > > > > > Office: (973) 353-5440x263 | FWD: 82823 | Fax: (973) 353-1171
> > > > > > >         101 Warren Str, Smith Hall, Rm 4-105, Newark NJ 07102
> > > > > > > WWW:     http://www.linkedin.com/in/yarik
>
>
>
> > > > > --
>
> > > > > Yaroslav Halchenko
> > > > > Research Assistant, Psychology Department, Rutgers-Newark
> > > > > Student  Ph.D. @ CS Dept. NJIT
> > > > > Office: (973) 353-5440x263 | FWD: 82823 | Fax: (973) 353-1171
> > > > >         101 Warren Str, Smith Hall, Rm 4-105, Newark NJ 07102
> > > > > WWW:     http://www.linkedin.com/in/yarik
>
>
>
> > > --
>
> > > Yaroslav Halchenko
> > > Research Assistant, Psychology Department, Rutgers-Newark
> > > Student  Ph.D. @ CS Dept. NJIT
> > > Office: (973) 353-5440x263 | FWD: 82823 | Fax: (973) 353-1171
> > >         101 Warren Str, Smith Hall, Rm 4-105, Newark NJ 07102
> > > WWW:     http://www.linkedin.com/in/yarik
>
> > > _______________________________________________
> > > Pkg-exppsy-maintainers mailing list
> > > Pkg-exppsy-maintainers at lists.alioth.debian.org
> > > http://lists.alioth.debian.org/mailman/listinfo/pkg-exppsy-maintainers
>
>
>
> --
>
> Yaroslav Halchenko
> Research Assistant, Psychology Department, Rutgers-Newark
> Student  Ph.D. @ CS Dept. NJIT
> Office: (973) 353-5440x263 | FWD: 82823 | Fax: (973) 353-1171
>         101 Warren Str, Smith Hall, Rm 4-105, Newark NJ 07102
> WWW:     http://www.linkedin.com/in/yarik
>
> _______________________________________________
> Pkg-exppsy-maintainers mailing list
> Pkg-exppsy-maintainers at lists.alioth.debian.org
> http://lists.alioth.debian.org/mailman/listinfo/pkg-exppsy-maintainers
>

More information about the Pkg-exppsy-maintainers mailing list