[Pkg-octave-devel] Unit test bug in octave-signal

[Oliver Heimlich]

On 13.11.2017 16:26, Sébastien Villemot wrote:
> On Mon, Nov 13, 2017 at 04:04:38PM +0100, Sébastien Villemot wrote:
> 
>> This has already been said several times on this list, but it is important to
>> remember that it is normal to get slightly different numerical results on i386.
>> The reason is that in i386, numerical computations are done with the x87 FPU,
>> which internally stores double-floats with extra precision (80-bits), which
>> means that when several consecutive mathematical operations are done within FPU
>> registers, they do not conform to the IEEE 754 standard for double floats.
>> On the contrary, amd64 uses SSE2 registers, which are 64-bit wide, and
>> therefore conformant to the IEEE standard.
> 
> And I forgot to add that the easy workaround is to add the -ffloat-store option
> to gcc. With this option, the result of every floating point operation is
> copied back into memory (and read again) before moving to the next operation.
> Therefore, every intermediate result gets truncated to 64-bit double float. As
> a consequence, one gets exactly the same results as on amd64. But this has an
> obvious performance cost (many additional memory read/writes, as many as there
> are intermediate results). So one has to ponder carefully before using this option.


I could simply reproduce the numeric errors in a VirtualBox VM with
Debian 32-bit.  However, using “-ffloat-store” didn't fix it.  The only
way to make it work was to use either “-mfpmath=sse -msse” or
“-mpfmath=sse -msse2”.

Also, I noticed that the function crlibm_init, which is called, sets FPU
flags to use double, not double exteded.  However, it doesn't help
either.  Do you know another trick to disable 80-bit arithmetic?


If not, I'd suggest to always enforce SSE(2) on x86.  That means, the
program will not run on certain processors.  If this could be detected
at run-time, everything is fine:
 * x86 CPUs with SSE(2) can use crlibm with correct results
 * x86 CPUs without SSE(2) would not use crlibm at all, the interval
package's algorithms will fall back to use mpfr with correct results


What do you think?

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://lists.alioth.debian.org/pipermail/pkg-octave-devel/attachments/20171204/93f7c587/attachment.sig>