CPU ram offload
NicolasMejiaPetit opened this issue · comments
Nicolas Mejia Petit commented
How can we offload the weights to CPU ram as opposed to the disk?
many people have for example
32-128gb of ram
And a 3090
With the layers offloaded to cpu ram we would get full pcei speeds, and not be bottle necked by m.2 speeds.