remixer-dec / llama.sh

No-messing-around sh client for llama.cpp's server

llama.sh

No-messing-around sh client for llama.cpp's server

NOTE: Default config optimized for Mixtral-Instruct-8x7B

Depends on

• sh
• curl
• jq

Setup

Add these scripts to your $PATH. Configuration is done by editing the top of the llama.sh file.

Not sure how to set up llama.cpp? Check out Mozilla's llamafile!

Usage

llama.sh [--options] "prompt"
echo "prompt" | llama.sh [--options]
echo "prompt" | llama.sh [--options] "system prompt"

llama.sh --help

Demo (python_chat.sh)