can some result have a higher precision?
Freed-Wu opened this issue · comments
wzy commented
such as
20.41
20.42
20.40
20.38
20.41
20.42
20.39
20.37
20.40
20.42
SmplStdCol
0.0
in fact
the result is 0.017511900715419.
Christian Brabandt commented
On Mo, 31 Aug 2020, wzy wrote:
such as
```
20.41
20.42
20.40
20.38
20.41
20.42
20.39
20.37
20.40
20.42
```
`SmplStdCol`
0.0
in fact
the result is 0.017511900715419.
Okay, the whole calculations was a bit of a mess, because too many
places have to handle string and non-floats. Let's refactor all this to
always handle floats.
I think it does so now correctly, please check.
Best,
Christian
--
Betet der Bauer in der Kirche laut, hat er des Nachbarn Schwein geklaut.