Approx 1 ulp imprecise in rem_pio2

[Max Graey]

Example:

sin(1e90 * M_PI);

Expect: 0.7352269418766969
Actual: 0.7352269418766968

I guess it comes from right_ helper. If shifter decrease by 1 result will be correct (or correct just compare to musl/libc?)

Btw I checked this argument in Julia which also used similar approach for argument reduction:
Julia playground

println(sin(1e90 * pi))
> 0.7352269418766968

So I'm wondering which result is correct

[Chen-Pang He]

Both of them are 'faithful' (error < 1 ulp) because

• expected - actual = 1 ulp
• actual < sin(1e90 * pi) < expected

a = 0.7352269418766968
b = 0.7352269418766969
nextfloat(a) == b
> true

c = sin(big(1e90 * pi))
> 0.7352269418766968505970386886904097559371310898766280200559523340248368213051653

a < c < b
> true

Indeed, the expected answer is the closest representation ('correct', error <= 0.5 ulp), but current result is still conforming.

# b - a == 1 ulp because b == nextfloat(a)

Float64(b - c) / (b - a)
> 0.485203106315901 # (b - c) in ulps

Float64(a - c) / (b - a)
> -0.514796893684099 # (a - c) in ulps

Correctly rounding transcendental functions is generally infeasible due to Table Maker's Dilemma. That's why metallic only promises faithful rounding like other libcs.

[Max Graey]

Thanks for explanation!