Giters

rustformers / llm

[Unmaintained, see README] An ecosystem of Rust libraries for working with large language models

Home Page:https://docs.rs/llm/latest/llm/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

AMD ROCm support with HIPBLAS

xangelix opened this issue · comments

commented

ggerganov/llama.cpp#1087 was merged recently and should make its way into GGML shortly. I think (hopefully) it should require only minimal changes on this side.

commented

rustformers uses llama.cpp as it's ggml source, feel free to create an PR including this change, seams like you only need to adjust the build.rs of the ggml-sys create. I wont be able to test this as i don't own a amd gpu.