Investigate whether we need to test negative axes in the error metric
nfrechette opened this issue · comments
In the error metric, we only test the positive axes in local space before we transform them. Would there be a benefit in testing 6 axes instead of 3 when scale is present? Due to floating point inaccuracies and how negative scale is handled by QVV transforms, perhaps it can contribute to the error.
Testing 6 axes reduced the memory footprint slightly (less than 1%) with no accuracy benefit.
Although qvv_mul has special handling for negative scale, qvv_mul_point3 does not. It shouldn't contribute significantly one way or the other. Since testing 3 axes is faster than 6 (no SOA error metric right now), the original 3 axis test will remain.