Large file management recommendation

Question

Large file management recommendation

daviddwlee84 opened this issue 5 years ago · comments

I found that you said the MNIST dataset was exceeded the maximum storage of the Github limitation.
Maybe you can try to use git-lfs to manage that file then you won't need to keep the file in the different place!

Dod-o · Answer 1 · Wed Dec 05 2018 19:37:20 GMT+0800 (China Standard Time)

I found that you said the MNIST dataset was exceeded the maximum storage of the Github limitation.
Maybe you can try to use git-lfs to manage that file then you won't need to keep the file in the different place!

I actually tried,but when I clone it I only got 1kb of dataset.
I will try again, thank you.

David Lee · Answer 2 · Thu Dec 06 2018 20:53:43 GMT+0800 (China Standard Time)

Maybe use git lfs pull to fetch lfs files. (reference)
And it said that

Git will only fetch LFS files that match patterns in your .gitattributes

Maybe another reason is that the file doesn't list in .gitattributes. Good luck :)

Dod-o · Answer 3 · Sat Dec 08 2018 22:33:37 GMT+0800 (China Standard Time)

I was going to try it today, but I suddenly felt that if the dataSet was too large, it maybe very inconvenient to clone.
So I think it's better to maintain the data size, don't you think?
Best wishes