Bugs with uint64() and associated binary and bitwise operations (library functions only?) around %inf
Reported by Brian Miller
BUG DESCRIPTION:
----------------
Apparent rounding discrepancies for elementwise vs non-elementwise binary operations on uint64 type, especially on large (>~2^52)
Bugs in uint64 and bitwise operations on uint64 type.
ERROR LOG:
----------
No errors
HOW TO REPRODUCE THE BUG:
-------------------------
--> uint64(%inf)./uint64(2) // ans=9223372036854775807, "correct", ignores remainder
--> uint64(%inf)/uint64(2) // ans=9223372036854775808, rounds up??
--> uint64(%inf).*uint64(2) // ans=18446744073709551614, "correct", trailing zero and discards overflow,
--> uint64(%inf)*uint64(2) // ans=9223372036854775808, buggy!
--> uint64(18446744073709551615) //ans=9223372036854775808, bug, possibly Win7 specific?
Bitwise operations with function calls:
--> bitand(uint64(%inf),uint64(%inf)) //ans=9223372036854775808, bug
--> bitand(uint64(%inf),uint64(2)) //ans=0, bug
--> bitand(uint64(2).^63-1,uint64(2).^63-1) //ans=0, bug, and just to check...
--> uint64(2).^63-1 //ans=9223372036854775807 is fine??
Similar errors for bitxor(), but bitor() seemed to work fine, as did the symbolic binary operations & and |.
OTHER INFORMATION:
------------------
In regards / vs ./ problem, it appears related to double conversions. If you examine first
--> (uint64(2)^(2:63)-1)'
which is all correct, then
--> (uint64(2)^(2:63)-1)'/uint64(2)
you can see the "rounding up" discrepancy kicks in at 2^53-1.