LLMFarm is an iOS and MacOS app to work with large language models (LLM). It allows you to load different LLMs with certain parameters.
Based on ggml and llama.cpp by Georgi Gerganov.
Also used sources from:
- rwkv.cpp by saharNooby.
- Mia by byroneverson.
- MacOS (13+)
- iOS (16+)
- Various inferences
- Metal for llama inference (MacOS and iOS) (dont work on intel Mac)
- Model setting templates
- Sampling from llama.cpp for other inference
- classifier-free guidance sampling from llama.cpp
- Other tokenizers support
- Restore context state (now only chat history)
- Metal for other inference
Models must be quantized by ggml after #154. LLMFarm work in sandbox mode and put added models to the "models" directory.
When creating a chat, a JSON file is generated in which you can specify additional inference options. The chat files are located in the "chats" directory. You can see all inference options here.
You can download some of the supported models here.
llmfarm_core has been moved to a separate repository. To build llmfarm, you need to clone this repository recursively:
git clone --recurse-submodules https://github.com/guinmoon/LLMFarmThe number of open models is continuously growing. One of the advantages of using such models is the possibility of preserving their original content without censorship. However, the disadvantage may be the irrelevance of the information contained in them. You can also get answers to questions from various industries, for example, there are models that specialize in medical terms or programming. In addition, with the help of these models, you can create stories, songs, music and play quests etc...