Pandas dataframe used where numpy array is expected in in ArrayUtils.band_df
CyanBC opened this issue · comments
When a mask is not used for ArrayUtils there are instances where the input array calling that function is not properly the expected numpy array. The outcome is an ellipsis error due to the difference in indexing between pandas and numpy.
[ ... , bn ]
in Lyzenga method guide you have:
x_train, x_test, y_train, y_test = train_test_split( df[imrds.band_names],df.depth,train_size=20000,random_state=5)
which returns pandas dataframes for x_train, x_test, y_train, y_test
Moving further along these are passed to ArrayUtils.band_df as pandas dataframes.
traindf = ArrayUtils.band_df( x_train )
Even though the function expects:
imarr : np.array or np.ma.MaskedArray
This was probably not noticed because in testing ArrayUtils.band_df a mask was always used, which would have run ArrayUtils.equalize_band_masks which doesn’t have indexing problems and returns:
tuple of N np.ma.MaskedArray
The error you get is:
KeyError: (Ellipsis, 0)
because you cannot subset pandas dataframe with [ ... , ]
my quick-fix was to do
x_train = x_train.as_matrix() x_test = x_test.as_matrix() y_train = y_train.as_matrix() y_test = y_test.as_matrix()
which seems to work, but I don't know if it will cause issues later on.
Thanks for letting me know. I'll try to look into it when I can. I'm having a lot of trouble trying to do anything to support OpticalRS at the moment. I've got too much going on with other projects, and I don't have any funding to support work on OpticalRS. I'm doing what I can in my free time, but finding that I don't really have any free time. It's great to know that people are using the software though, and that may help motivate me to find a way to get back to it properly. Please do let me know if you find OpticalRS useful, and please cite it if you can. If I can demonstrate that people are using it, it'll be easier to try and get some funding going to support it. Thanks again.