alexrozanski / LlamaChat

Chat with your favourite LLaMA models in a native macOS app

Home Page:https://llamachat.app

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Add Metal/GPU support for running model inference

singularitti opened this issue · comments

I am no expert in this, but it seems to be running on CPUs, which could cause severe heat generation.

@singularitti adding support for this in llama.swift to start with (see alexrozanski/llama.swift#8). this will be coming to LlamaChat v2 which is still a WIP!