joone / loz

Loz is a command-line tool that enables your preferred LLM to execute system commands and utilize Unix pipes, integrating AI capabilities with other Unix tools.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Model configuration does not work (Error: Model llama2 not found)

sammcj opened this issue · comments

commented

When configuring the Ollama model to use loz does not seem to actually save the change and crashes out:

samm-mbp ~ $ ollama pull tinyllama:1.1b-chat-v1-fp16
pulling manifest
pulling 659305c3a0fd... 100% ...
▕███████████████████████████████████████████████████████████████████████████████████████▏  2.2 GB
verifying sha256 digest
writing manifest
removing any unused layers
success
samm-mbp ~ $ loz config model tinyllama:1.1b-chat-v1-fp16
llama-node/lib/ollama.js:80
            throw new Error(`Model ${model} not found.`);
                  ^

Error: Model llama2 not found.
    at Ollama.setModel (loz/node_modules/ollama-node/lib/ollama.js:80:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Node.js v20.9.0
samm-mbp ~ [1] $ loz
Loz: a simple CLI for LLM
  api: ollama
  model: llama2

> config model tinyllama:1.1b-chat-v1-fp16
The model has been updated to 'tinyllama:1.1b-chat-v1-fp16'

> write a fake commit message about cats
loz/node_modules/ollama-node/lib/ollama.js:80
            throw new Error(`Model ${model} not found.`);
                  ^

Error: Model llama2 not found.
    at Ollama.setModel (loz/node_modules/ollama-node/lib/ollama.js:80:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Node.js v20.9.0

On the Ollama server side I see Loz hitting two API endpoints:

ollama serve
time=2024-02-15T07:40:31.676+11:00 level=INFO source=images.go:863 msg="total blobs: 60"
time=2024-02-15T07:40:31.679+11:00 level=INFO source=images.go:870 msg="total unused blobs removed: 0"
time=2024-02-15T07:40:31.681+11:00 level=INFO source=routes.go:999 msg="Listening on 127.0.0.1:11434 (version 0.1.24)"
time=2024-02-15T07:40:31.681+11:00 level=INFO source=payload_common.go:106 msg="Extracting dynamic libraries..."
time=2024-02-15T07:40:31.700+11:00 level=INFO source=payload_common.go:145 msg="Dynamic LLM libraries [metal]"
[GIN] 2024/02/15 - 07:40:39 | 200 |      29.458µs |       127.0.0.1 | GET      "/api/version"
[GIN] 2024/02/15 - 07:40:47 | 200 |   13.131041ms |       127.0.0.1 | GET      "/api/tags"

Loz currently supports only Llama2 and CodeLlama. I will consider supporting any LLM models that Ollama supports.

commented

Oh really, llama2 is getting pretty far behind other models these days, perhaps it could just support loading from the Ollama model list?

It is hard-coded. Okay, I will fix it.

This bug has been resolved in the latest release. Please let me know if the issue reoccurs.