Giters

khundman / telemanom

A framework for using LSTMs to detect anomalies in multivariate time series data. Includes spacecraft anomaly data and experiments from the Mars Science Laboratory and SMAP missions.

Home Page:https://arxiv.org/abs/1802.04431

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Normalisation of test and training data

plankthom opened this issue · comments

commented

First, many thanks for your insightful work ...

I however have an issue when loading the training and test data sets: in most cases they seem to be normalised to [-1,1] independently, and I was wondering whether this would not make the trained model inaccurate.

Eg. a density plot for the channel E4 :
distplot-E-4

Or did i miss something else?

commented

Thanks for the comment, I'm looking into this. This was an issue I had found before releasing the data and thought was corrected. If there are other suspicious channels you have noticed it would be helpful if you could provide them.

This one looks like what you have described, but there are instances where channel behavior can change abruptly due to commanding. I will follow up.

commented

@plankthom Following up on this - it is an error that won't be corrected. Unfortunately I no longer have access to the raw data and therefore can't rescale. I don't think it is material to the results or methods (in a sense it actually demonstrates the robustness of the overall approach). Thanks again for the note.

commented

Is it possible to provide the list of sequences that are confirmed to have this issue, so we can exclude them from our experiments?