Can you run it on CPU only?
amitbar05 opened this issue · comments
Amit commented
Can you run it on the CPU only?
And if so, where can I find how much ram is needed?
github-actions commented
Hello there, thanks for opening your first issue. We welcome you to the FauxPilot community!
Brendan Dolan-Gavitt commented
Hi – running it on CPU only should be possible using the Python backend. The RAM requirements will be similar to the GPU VRAM requirements. However, I should warn you that it will be a lot slower...
nzlov commented
When run CPU, should set cpu num use triton env in docker-compose.yaml
OMP_NUM_THREADS: 10
OPENBLAS_NUM_THREADS: 10
MKL_NUM_THREADS: 10
VECLIB_MAXIMUM_THREADS: 10
NUMEXPR_NUM_THREADS: 10
Abhimanyu Saharan commented
How about threadripper 3990X with 128GB RAM?
Michael McMahon commented
TurboPilot based on the concept of FauxPilot is built to run with CPU only and 4GiB of RAM.