TypeError: csv_to_numpy() got an unexpected keyword argument 'datapath'
Amyloida opened this issue · comments
Hi, when trying to turn .csv to .npy I get this error:
TypeError: csv_to_numpy() got an unexpected keyword argument 'datapath'
I have tried to just run "vame.csv_to_numpy(config)" without the paths to the files
And I get: Your data is now ine right format and you can call vame.create_trainset()
But when I try to create the training set, I get: ValueError: need at least one array to concatenate
In my VAME folder I can also not see any .npy files
Any idea what's the issue here and how to solve it? I'm grateful for any input.
I haven't been able to find this particular issue being reported/fixed previously. If I've missed it, please let know.
Hi!
Can you confirm that in your config.yaml under videos: all your video names are listed and that they correspond to the .csv file names?
The argument datapath is indeed not anymore part of the latest version in this function.
I just checked the config file and the video names don't seem to be listed:
Project path and videos
project_path: C:\Users\jrb21105\Desktop\VAME-mst\PS-mst-Aug9-2022
video_sets: []
When I specified the video paths in config = vame.init_new_project( ... ) I didn't get any error tho?
That is causing the problem. Please check out our workflow guide.
The crucial part is that the video names are in your config.yaml and that the video names are identical to the csv names.
Example:
video_sets:
- video-1
- video-2
- etc
The vame.init_new_project( ... ) function does not throw an error when it can't find the videos and still creates the project. I would recommend to re-create your project and make sure that the specified video paths are correct and the yaml file gets as well as the folder structure is created correctly. Moreover I would suggest to use simpler video names but this might be just a preference of style.
I hope this helps and you manage to setup your VAME project.
Thank you so much!
I've created a new project and pre-defining the videos (as described here: https://guillermo-hidalgo-gadea.github.io/Seminar-ComputationalEthology/Part1VAME.html) worked! video names are in the config file and all the steps work until I'm training the model. Here I get an error after 86 epochs:
Any idea what's the problem here?
Perfect!
This is a numerical issue and I would recommend to set the kmeans_lambda parameter within your config.yaml to 0, as the model learns a better embedding without this (see our latest preprint update where we tested this in the supplemental section).
I've set the kmeans_lambda to 0 in the config file but still ended up with the same error (at epoch 103 this time)
Thank you for reporting this, I think I will need to update the code so that it does not go at all into this function when kmeans_lambda is set to 0. For now, you could just go ahead into the rnn_vae.py script and set kmeans_loss = 0 in line 126.
If the error still persists for some reason ~103 epochs could be enough to try the next steps in VAME and go forward for now.
I've set the kmeans_loss to 0 in the rnn_vae script as well as the config file but unfortunately I still get the same error during training
If you get the same error, then the script still tries to get call the cluster_loss function. Best would be to delete it completely for now. Make sure to save the script afterwards and re-import everything.
Closing this for now.