JuliaMath / FixedPointNumbers.jl

fixed point types for julia

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

`rem(::BigFloat, ::Type{<:Normed})` may return an incorrect value

kimikage opened this issue · comments

This isn't a fatal issue in practice, but I ran into it in testing multiplication code.

julia> BigFloat(0x0_01234567_89abcdef) % N63f1 |> reinterpret
0x0123456789abcdef

julia> BigFloat(0x1_01234567_89abcdef) % N63f1 |> reinterpret
0xffffffffffffffff

The cause is the unsafe_trunc.

julia> r = 0x1_01234567_89abcdef;

julia> bf = BigFloat(r)
1.8528729602926038511e+19

julia> bi = BigInt(bf)
18528729602926038511

julia> unsafe_trunc(UInt64, bf)
0xffffffffffffffff

julia> unsafe_trunc(UInt64, bi)
0x0123456789abcdef

Perhaps it would be better to convert via BigInt.

FYI, Fixed is not directly affected by this issue, but there is a problem with Fixed's rem in the first place.
(The same kind of fix as PR #170 needs to be applied.)

rem(x::Real, ::Type{Fixed{T,f}}) where {T,f} = Fixed{T,f}(rem(Integer(trunc(x)),T)<<f + rem(Integer(round(rem(x,1)*(one(widen1(T))<<f))),T),0)