Can't work well on Colab
yuya-0411 opened this issue · comments
I find a related issue learnables/learn2learn#310 (comment)
this happens when Google drive limits the number of downloads of a file. Rsyncing a local copy works, else wait a few hours for the limit to reset.
And the possible solution learnables/learn2learn#310 (comment)
A temporal solution:
1 Download the saved models directly from the google drive link I provide in the github page with your web browser.
2 Move them to the folder the network look for the saved model in the collab and rename it based on the code.
3 Load the state dict again
@williamyang1991 to avoid google drive issues, would also suggest hosting the models on the huggingface hub
Some of the benefits of sharing your models through the Hub would be:
- versioning, commit history and diffs
- repos provide useful metadata about their tasks, languages, metrics, etc that make them discoverable
- multiple features from TensorBoard visualizations, PapersWithCode integration, and more
- wider reach of your work to the ecosystem
you can learn more here: https://huggingface.co/docs/hub/models-uploading
Hi @AK391 ,
Yes, Liming (SoraEndless) has just told me about it :)
We plan to share our models on the huggingface just like DualStyleGAN.
However, we're extremely busy next week. So once we have spare time, we will work on it.
@williamyang1991 @AK391
Thank you for sharing the ways to solve this issue.
I hope that sharing will work well.
Before it, I will try what @williamyang1991 suggested first.
thank you
Along with your suggestion, I understand the number of files which will be downloaded is the cause for this problem.
so, I did the below instead of using my drive.
def get_download_model_command(file_id, file_name):
""" Get wget download command for downloading the desired model and save to directory ../checkpoint/. """
current_directory = os.getcwd()
save_path = MODEL_DIR
if not os.path.exists(save_path):
os.makedirs(save_path)
url = r"""wget --load-cookies /tmp/cookies.txt "https://docs.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate 'https://docs.google.com/uc?export=download&id={FILE_ID}' -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\1\n/p')&id={FILE_ID}" -O {SAVE_PATH}/{FILE_NAME} && rm -rf /tmp/cookies.txt""".format(FILE_ID=file_id, FILE_NAME=file_name, SAVE_PATH=save_path)
return url
MODEL_PATHS = {
"encoder": {"id": "1NgI4mPkboYvYw3MWcdUaQhkr0OWgs9ej", "name": "encoder.pt"},
"faceparsing": {"id": "1jY0mTjVB8njDh6e0LP_2UxuRK3MnjoIR", "name": "faceparsing.pth"},
# "arcane_exstyle": {"id": "1TC67wRJkdmNRZTqYMUEFkrhWRKKZW40c", "name": "exstyle_code.npy"},
# "caricature_exstyle": {"id": "1xr9sx_WmRYJ4qHGTtdVQCSxSo4HP3-ip", "name": "exstyle_code.npy"},
# "cartoon_exstyle": {"id": "1BuCeLk3ASZcoHlbfT28qNru4r5f-hErr", "name": "exstyle_code.npy"},
"pixar_exstyle": {"id": "1yTaKuSrL7I0i0RYEEK5XD6GI-y5iNUbj", "name": "exstyle_code.npy"},
# "arcane000": {"id": "1pF4fJ8acmawMsjjXo4HXRIOXeZR8jLVh", "name": "generator.pt"},
# "arcane077": {"id": "16rLTF2oC0ZeurnM6hjrfrc8BxtW8P8Qf", "name": "generator.pt"},
# "caricature039": {"id": "1C1E4WEoDWzl0nAxR9okKffFmlMOENbeF", "name": "generator.pt"},
# "caricature068": {"id": "1B1ko1x8fX2aJ4BYCL12AnknVAi3qQc8W", "name": "generator.pt"},
# "cartoon026": {"id": "1YJYODh_vEyUrL0q02okjcicpJhdYY8An", "name": "generator.pt"},
# "cartoon299": {"id": "101qMUMfcI2qDxEbfCBt5mOg2aSqdTaIt", "name": "generator.pt"},
"pixar052": {"id": "16j_l1x0DD0PjwO8YdplAk69sh3-v95rr", "name": "generator.pt"},
# "cartoon": {"id": "11s0hwhZWTLacMAzZH4OU-o3Qkp54h30J", "name": "generator.pt"},
}
I wanted to try pixar-like transformation, so I changed the menu for downloading a little bit.
I succeeded.
it's interesting!!!
thank you
Which lines to change in the colab to say that it will fetch the data in my personal google drive instead of yours?
Which lines to change in the colab to say that it will fetch the data in my personal google drive instead of yours?
@zast57 for example, if the file in your google driver is shared by the link https://drive.google.com/file/d/1YJYODh_vEyUrL0q02okjcicpJhdYY8An/view?usp=sharing (this is cartoon26 model in my drive)
Then you just need to change the id of this line of the colab into 1YJYODh_vEyUrL0q02okjcicpJhdYY8An
"cartoon026": {"id": "1YJYODh_vEyUrL0q02okjcicpJhdYY8An", "name": "generator.pt"},
for example,
if your link is https://drive.google.com/file/d/ABCDEFG/view?usp=sharing
Then the line should be changed into
"cartoon026": {"id": "ABCDEFG", "name": "generator.pt"},
@williamyang1991 Ok :). It s perfect Thank you :)