feat: Support `--main-gpu` in the ggml plugin
hydai opened this issue · comments
Summary
When using the ggml plugin in a multiple-GPU machine, it's important to specify the main GPU. The default value is using GPU 0 now.
Details
- Support
--main-gpu
, see https://github.com/ggerganov/llama.cpp/tree/master/examples/main#additional-options
Appendix
No response