[Request]: koboldcpp-cu12 (new version using CUDA 12)
Spacellary opened this issue · comments
Spacellary commented
Criteria
- I have searched all issues/PRs to ensure it has not already been reported or fixed.
- English interface (or at least English documentation)
- Alternative, forked or prerelease version of an existing package
- Fairly standard install (e.g. no elaborate pre/post install scripts)
Name
koboldcpp-cu12
Description
"A simple one-file way to run various GGML models like LLAMA, ALPACA, VICUNA. This version uses the newer CUDA 12 binaries. If you have a newer NVIDIA GPU and don't mind larger files, you may get increased speeds by using this new version."
Homepage
https://github.com/lostruins/koboldcpp
Download Link(s)
https://github.com/LostRuins/koboldcpp/releases/download/{latest-tag}/koboldcpp_cu12.exe
Eg.: https://github.com/LostRuins/koboldcpp/releases/download/v1.66/koboldcpp_cu12.exe
Some Indication of Popularity/Repute
Over 3.9k stars
282 forks
Same regular install as the standard koboldcpp
, just a new executable with newer cuda binaries.
Corresponding package
extras/koboldcpp