Issue in running demo
Su-deep opened this issue · comments
Hi,
While running the demo on jupyter notebook I ran in the following issue, while executing the codeblock:
imputer.batch_generate_samples(m= 5)
print("Original value:", original_value)
imputed_vals = []
for dataset in imputer.output_list:
imputed_vals.append(pd.DataFrame(scaler.inverse_transform(dataset),
columns= dataset.columns).iloc[50, 0])
print("Imputed values:")
print(imputed_vals)
print("Imputation mean:", np.mean(imputed_vals))
print("Standard deviation of the imputation mean:", np.std(imputed_vals))
Can you please help me fix this. Thanks
INFO:tensorflow:Restoring parameters from tmp/MIDAS
Model restored.
InvalidIndexError Traceback (most recent call last)
in ()
----> 1 imputer.batch_generate_samples(m= 5)
2
3 print("Original value:", original_value)
4 imputed_vals = []
5 for dataset in imputer.output_list:
~/git/PycharmProjects/untitled/midas.py in batch_generate_samples(self, m, b_size, verbose)
910 columns= self.imputation_target.columns)
911 output_df = self.imputation_target.copy()
--> 912 output_df[np.invert(self.na_matrix.values)] = y_out[np.invert(self.na_matrix.values)]
913 self.output_list.append(output_df)
914 return self
~/git/PycharmProjects/untitled/venv/lib/python3.5/site-packages/pandas/core/frame.py in setitem(self, key, value)
3109
3110 if isinstance(key, DataFrame) or getattr(key, 'ndim', None) == 2:
-> 3111 self._setitem_frame(key, value)
3112 elif isinstance(key, (Series, np.ndarray, list, Index)):
3113 self._setitem_array(key, value)
~/git/PycharmProjects/untitled/venv/lib/python3.5/site-packages/pandas/core/frame.py in _setitem_frame(self, key, value)
3158 self._check_inplace_setting(value)
3159 self._check_setitem_copy()
-> 3160 self._where(-key, value, inplace=True)
3161
3162 def _ensure_valid_index(self, value):
~/git/PycharmProjects/untitled/venv/lib/python3.5/site-packages/pandas/core/generic.py in _where(self, cond, other, inplace, axis, level, errors, try_cast)
7543 not all(other._get_axis(i).equals(ax)
7544 for i, ax in enumerate(self.axes))):
-> 7545 raise InvalidIndexError
7546
7547 # slice me out of the other
InvalidIndexError:
Hi there,
I apologise for things like this. Often I'm writing MIDAS in coffee breaks or at midnight, so sometimes I forget to test things. (Unfortunately, this is in addition to my actual job and studies!) The recent updates broke a whole lot, but everything was working on my hope laptop which has an older development environment. You're actually not the first to report this bug, so it wasn't until I checked it on my work laptop that it emerged that it must be the older libraries.
I ran the new code on the latest libraries on two computers. It should work now.
The documentation is actually WIP right now, as is the paper. There's a brief 'how-to' available on the repository, but this hasn't integrated the new features I added over the last 3 weekends. For now, the code should be stable enough, and once I've finished performance testing, I'll rewrite the docs and make a proper CRAN-style vignette, as well as an expanded tutorial on how to use the variational component. Myself and Ranjit aren't CS people per se, and MIDAS has only really been unveiled at one event.
I could point you in the direction of the papers underlying the implementation in the meantime? Alternatively, I'm happy to open a dialogue as to how MIDAS works, either here or by email. I understand this is less useful to you than a paper, but I promise you that this will change soon!
Alex