Question: REFIT Dataset - house1
fdelcab opened this issue · comments
Is there a reason why the house1 dataset wasn't used to build your model? I noticed that when resampling to 1 minute, 12.64% of the data is missing. Could this be the explanation?
Additionally, regarding the data ingested by the model, did you consider the entire dataset for seq-to-seq/point processing, regardless of whether the appliance was not consuming any power for a significant portion of the time? Or did you subset the dataset to isolate the period when the appliance is actively in use and then did the seq-to-seq/point processing?
Hi, these are good questions indeed. I cannot remember the details, but I am sure that the data should not be excluded just because some data points are missing; indeed, missing data may be helpful for training a DNN model as it would actually acts as noise which would be helpful for generalisation.
For your second question, each appliance was trained using a subset of the houses; I think that was because those excluded houses do not have that appliance. This is a research question: can you use the entire dataset to train an appliance? This means the training data include those houses which do not have that appliance. I don't have the answer for this question.
For the house which was included in the training set, I am sure we were not only using the period when the appliance was actively in use.
Thank you for your reply,
I have other question regarding this subject:
- When you say that missing data might be good for generalization as noise, should we do some random noisy imputation of that missing values? Or should we give the input as it is with NaN values?
Extra question:
2. Did you create a model per appliance, or you use one model to predict all the appliances?
- NaN values are replaced by 0.00, which means I don't impute missing values
- One model per appliance