marcandrysco / Errol

Binary floating-point to decimal string conversion algorithm.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Incorrectly rounded result for 10.1178131103515625

tbkka opened this issue · comments

In my tests, Errol4 currently returns 10117813110351563 for this value.

I think it should be 10117813110351562 (rounding the last digit even).

Another example: 14.6767120361328125

I believe this should produce 14676712036132812, not 14676712036132813 as Errol4 currently does.

A third example, following the same pattern:

Input:    15.1475372314453125
Errol4:   15.147537231445313
Expected: 15.147537231445312

I can confirm that this bug is a bug, and is still present in current code (as of commit 5364de4).

For anyone wondering why this is actually a bug, 10.1178131103515625 is a decimal number that happens to be exactly representable as a 64-bit float. Therefore, it is exactly halfway between the two optimal decimal string representations 10.117813110351562 and 10.117813110351563. By the "best selection" criterion as defined in the comment for commit 43600ce on March 2, 2016:

In the event that two optimal values are equal distant from the
floating point number, the even value is chosen (round to even).

the even representation should have been output.

The Errol tests confirm that this bug is present in errol4 for all of the example values given by @tbkka.

~/Repositories/Errol/test $ ./errol-test --fuzz4 1 --lower 10.1178131103515625 --upper 10.1178131103515625
Fuzzing Errol4... 0k/0k 0.00%Conversion failed. Expected 0.10117813110351562e2. Actual 0.10117813110351563e2. Read as 1.01178131103515625e+01.
Fuzzing Errol4 done on 1 numbers, 0 failures (0.000%), 0 suboptimal (0.000%), 1 notbest (100.000%)

Errol3 produced proper output, at least for the example values given by @tbkka.