[Magneto] Handle infinite samples with constant memory usage
kitlith opened this issue · comments
the first thing magneto does with the input sample is first expand each sample into a 10 item row/column, treat the input as a 10xN matrix, and multiply it by it's own transpose, resulting in a 10x10 matrix.
expanding out the math, it turns out that taking each sample as a 10x1 matrix, multiplying by its own transpose, then summing each resulting matrix together results in the same matrix, except for floating point imprecision. I'm not sure whether the result is more or less precise, but according to my testing it's fairly close.
https://blog.demofox.org/2017/01/02/incremental-least-squares-surface-and-hyper-volume-fitting/ suggests we can do it with even less space than a full 10x10 matrix due to the structure of the input, 19 elements instead of 100. nevertheless, even if we don't go that far, we can still save quite a bit of memory. (currently, for mpu9250 at least, we save 3*300 elements, which gets expanded to 10*300 elements, before we end up with the 10x10 matrix of 100 elements.)
However, since we'd be doing the calculations on the fly, we lose the current method of outlier rejection. If we wanted to keep outlier rejection, we'd need to find a way to do it 'online', as samples are streaming in.
Ah. Turns out we're currently not rejecting anything anyway, as nxsrej = 0. So that's not actually really a blocker.
Apparently, 19 values is an underestimate based on a simpler form of the problem. meh. i don't think it's a big deal.