Flatline anomalies in regular period scalar data are not identified
rhyolight opened this issue · comments
Common flatline anomalies are not detected in many cases by HTM algorithms.
The first report of this is from HTM Forum. See:
Later, it was also reported against HTM.Java in a private message. I have taken all the data from that private message from Parag_Goyal and put it here in this report.
First and foremost, download the flatline-anomaly.zip file to get a reproduction of this issue. It contains data and a nupic program to replicate. See also the attached output from this program (output MT.xlsx) with Excel charts showing anomaly scores and anomaly likelihoods.
This phenomenon has been reported in NuPIC and HTM.Java, so we may assume it is an algorithmic issue. It could be something to do with how the anomaly scores are calculated, or how anomaly likelihoods are calculated.
This is an open issue, we are aware of it, but are not prioritizing it for work at this time. But we want to report that it exists and someone might be able to figure out what's wrong.
One last note: Eventually we would like to add the dataset in flatline-anomaly.zip to NAB. It is a good dataset that represents a very common anomaly in streaming scalar data. We plan to do this by adding a staging area for new data sets in NAB so we can publish them with our next versioned release.
I should mention this is a known issue:
This is not just a flat line issue. If you look at this data set : non-flatline-data.csv.zip data line in mid is not flat. Metrics vary between 1 to 3 but still, the anomaly is not detected
Anomaly region in data:
The overall analyzer output:
Correct, not just a flatline issue, and you can see why in this line of code I quoted above:
if metricDistribution["variance"] < 1.5e-5: