Sufficient Colab tier/GPU for the "original" notebook?

Question

Sufficient Colab tier/GPU for the "original" notebook?

olaviinha opened this issue a year ago · comments

I'm using the "original" notebook. At first it ran just fine and dandy for a few days, but since 2 days ago or so it's giving me almost always "CUDA OUT OF MEMORY". Other times it doesn't, which makes it even weirder, and I can't find anything that would have changed within the past 2 days.

I'm on Colab Pro tier (i.e. T4), didn't at any point have Premium GPU selected. Is Premium GPU required/recommended, or have you any idea what might be causing this behaviour or how to prevent it? I'm out of ideas myself...

Thanks for the great work btw!

camenduru · Answer 1 · Thu Apr 06 2023 05:41:55 GMT+0800 (China Standard Time)

Hi @olaviinha 👋 do you have this line in your colab

!pip install -q torch==1.13.1+cu116 torchvision==0.14.1+cu116 torchaudio==0.13.1 torchtext==0.14.1 torchdata==0.5.1 --extra-index-url https://download.pytorch.org/whl/cu116 -U

O. Inha · Answer 2 · Thu Apr 06 2023 05:53:46 GMT+0800 (China Standard Time)

I do, yes, and I think it started working a little better after those updates. Now CUDA OOM is no longer default result, but still happens a lot...

UPDATE: Ok I see Colab has changed GPU class to GPU type, which now defaults to better than T4 when you have compute units. I didn't know this, just assumed I had T4 all along (as Colab used to work). I think I ran out of compute units when it started complaining CUDA OOM, so probably just dropped to T4 from a better GPU at that time – that would explain the behaviour.

Anyway it would be great if somebody came up with a solution to run it smoothly on T4!