af913337456 / pygpt4all

Official supported Python bindings for llama.cpp + gpt4all

PyGPT4All

Official Python CPU inference for GPT4All language models based on llama.cpp and ggml

NB: Under active development

Installation

pip install pygpt4all

Usage

GPT4All model

Download a GPT4All model from https://the-eye.eu/public/AI/models/nomic-ai/gpt4all/

from pygpt4all.models.gpt4all import GPT4All

def new_text_callback(text):
    print(text, end="")

model = GPT4All('./models/ggml-gpt4all-j.bin')
model.generate("Once upon a time, ", n_predict=55, new_text_callback=new_text_callback)

GPT4All-J model

Download the GPT4All-J model from https://gpt4all.io/models/ggml-gpt4all-j-v1.3-groovy.bin

from pygpt4all.models.gpt4all_j import GPT4All_J

def new_text_callback(text):
    print(text, end="")

model = GPT4All_J('./models/ggml-gpt4all-j.bin')
model.generate("Once upon a time, ", n_predict=55, new_text_callback=new_text_callback)

License

This project is licensed under the MIT License.