[Pkg-octave-devel] Bug#706376: Bug#706376: Bug#706376: Bug#706376: Bug#706376: Bug#706376: octave: sparse matrix n*2^16

[David Bateman]

On 06/19/2013 09:53 PM, Jordi Gutiérrez Hermoso wrote:
> I think you'll still need to do something about a more realistic
> situation of linear indexing with a logical matrix, which also ends up
> translating into linear indexing thanks to our underlying bug: the
> assumption that linear indexing works. In this case, there shouldn't
> be an error at all, like Ed suggested, since we have enough
> information in a logical matrix to avoid linear indexing.

Huh ? How can a logical matrix linear index overflow ? The matrix is
limited to 2^31-1 elements in any case and so can never overflow! I
presume you means "sparse logical matrix linear indexing". This is one
of the FIXMEs the the sparse code that has existed for a long while and
is documented in ov-bool-sparse.h

<quote>
  // FIXME Adapt idx_vector to allow sparse logical indexing!!
  idx_vector index_vector (void) const
    { return idx_vector (bool_array_value ()); }
</quote>

and in the projects page (as copied from the old PROJECTS file)

<quote>
Sparse logical indexing in idx_vector class so that something like
'a=sprandn(1e6,1e6,1e-6); a(a<1) = 0' won't cause a memory overflow.
</quote>

At the time I wrote the above line, it seemed a major effort to do this
correct. It seems that Yaroslav added some code to treat sparse logical
index vectors but didn't connect it to the idx_vector method of
octave_sparse_bool_matrix. It should be used as it'll improve the speed
of the creation of idx_vector, but it doesn't help as the underlying
data type of idx_vector in this case is Array<octave_idx_type> and so a
single index (even if its a sparse logical matrix) can result in an
overflow. This will take major reworking of the idx_vector class and the
uses of single index vectors in Sparse<T>. In any case this throws and
error at this point and so won't silently fail

> It looks good. Can you push it? 

I'd like to add some tests first and see if any other bugs have turned
up after this change. For example the changes use made to sprand and
sprandn 2 years ago to call randperm also overflows. At the moment I'm
getting 791 failed tests with "make check" is that normal ?

David