ericmj / decimal

Arbitrary precision decimal arithmetic

Home Page:https://hexdocs.pm/decimal/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Signals when applying different scale results in inexact result

LostKobrakai opened this issue · comments

I'm working with an encoding, where the scale (and precision) is hardcoded. I'm using Decimal.Context.with/2 to verify the provided decimal value is within those specs. That works great for precision (using Decimal.apply_context), but Decimal.round doesn't provide any signals if the rounding changed the value. I guess this kinda makes sense given the value was explicitly rounded, but on the other hand there's also no alternative to change the scale.

Edit:

As a workaround I'm currently using this:

decimal
|> Decimal.normalize()
|> tap(fn decimal ->
  if Decimal.scale(decimal) > scale do
    Decimal.Context.get()
    |> Map.update!(:flags, &[:inexact | &1])
    |> Decimal.Context.set()
  end
end)
|> Decimal.round(scale)

How come you can't use Decimal.apply_context/1?

I'd love that, but I don't see a scale setting on Decimal.Context. I'm already using it for the precision part after having adjusted the scale.

Oh, I see. Then comparing against Decimal.scale/1 is the correct approach. We shouldn't emit signals when Decimal.round/2 rounds.

Would you be open to making scale part of Decimal.Context (e.g. given a PR)? E.g. consider 40.01 / 10 = 4.001. Given the context of precision 4 / scale 2, this would also produce a problematic value per the scale, but not per the precision.

I don't think we should add scale because it's not part of the specification we built this library from.

Flags for precision is required because it's the only way to know we lost data during an operation. Scale can always be checked after the operation using `Decimal.scale/1.