fauxpilot / fauxpilot

FauxPilot - an open-source alternative to GitHub Copilot server

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Can you run it on CPU only?

amitbar05 opened this issue · comments

commented

Can you run it on the CPU only?

And if so, where can I find how much ram is needed?

Hello there, thanks for opening your first issue. We welcome you to the FauxPilot community!

Hi – running it on CPU only should be possible using the Python backend. The RAM requirements will be similar to the GPU VRAM requirements. However, I should warn you that it will be a lot slower...

commented

When run CPU, should set cpu num use triton env in docker-compose.yaml

OMP_NUM_THREADS: 10
OPENBLAS_NUM_THREADS: 10
MKL_NUM_THREADS: 10
VECLIB_MAXIMUM_THREADS: 10
NUMEXPR_NUM_THREADS: 10  

How about threadripper 3990X with 128GB RAM?

TurboPilot based on the concept of FauxPilot is built to run with CPU only and 4GiB of RAM.