Proposal: specify max_precision in struct?
giddie opened this issue · comments
Addressing some thoughts from #202 (comment), I wonder what you think about the feasibility of (optionally) storing a max_precision field in the Decimal struct? The would mean that when creating Decimals, libraries have the option of specifying physical limits (e.g. numerics coming from Postgresql). The lowest max_precision could then be used for each Decimal operation.
This would solve the problem of operations on Decimals coming from a PostgreSQL numeric column (via Ecto) using a precision many orders of magnitude lower than the column would support for storing the result.
If the use case is for PostgreSQL then I don't think it's a good idea. If postgrex would set the precision to 131072, then running an operation such as sqrt on a number from postgrex could be very expensive.
But isn't that kind of the point of Decimal? If you wanted fast operations, you'd use a float. Decimals are precise, but slow. To me, I would expect that Decimal operations would be as correct as possible by default, and maybe offer speed improvements as advanced configuration. In this case, I'd expect that if sqrt is slow for a given input, the option to specify a reduced precision would be great, but I wouldn't expect it to be the default.
If the default values are slow enough that would a security risk since it's a vector for DDOS attacks. I would prefer that the defaults are safe with an option to explicitly increase the precision to be more accurate.
Isn't that an issue for any calculation involving large numbers? Integer arithmetic in Elixir is arbitrary-precision by default, but the ** operator is still happy to calculate 123456 ** 123456 at full precision, even though it takes a considerable length of time. I don't think we can reasonably mitigate DDOS at the data layer - it's an app-level concern.
This library which is based on IEEE 754 does have those performance guarantees by default though and I don't feel comfortable changing that. The fact that other data types can have slow operations is not really relevant.
Since you can solve this issue by explicitly setting the precision you need before doing any operations I don't think we should go forward this.