PotatoSpudowski / fastLLaMa

fastLLaMa: An experimental high-performance framework for running Decoder-only LLMs with 4-bit quantization in Python using a C/C++ backend.

Home Page:https://potatospudowski.github.io/fastLLaMa/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Should fastLLaMa support more than just Python?

PotatoSpudowski opened this issue · comments

Hi, Would it be interesting to get fastLLaMa to support more than just Python?

Screenshot 2023-03-24 at 11 19 08 AM

Thoughts @amitsingh19975 @raldebsi ? 🤔

That would be cool since the concept of bridging is nice, but in pretty sure by now others have already begun work on other bridges so maybe it would be better to make a new repo that combines all the works?

yes! Lets do this. we need to create a folder called bindings with one for each language below.

@dahlej We have created a folder called interfaces where new languages can be added!

List of languages to consider

  • Rust
  • NodeJS/WebAssembly
  • Java
  • Go

If anyone else feels like we need to support any other languages please do request it!