lolengine / lol

😂 Small multiple purpose C++ framework

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

real::erf() inaccuracy

samhocevar opened this issue · comments

The current real::erf() implementation uses a Taylor series that does not converge and thus has precision issues.

  • The affected range is [7,19].
  • The approximate error is 10^(-n²/1.13).

That is to say, erf(8) will have a precision error of 10^(-8²/1.13) = 2.3e-57, e.g. the 57th decimal will be inaccurate.

Hey Sam