Working with ollama or llama.cpp

[Pavel Pavlenko]

With the publication of codellama, it became possible to run LLM on a local machine using ollama or llama.cpp.
How to configure your extension to work with local codellama?

[Mishig]

hello @region23

yes it is possible to use a local model. What you'd need to do is:

1. Serve the model locally at some endpoint
2. And change the settings accordingly

change it to your local endpoint

and make sure to update the prompt template

[Pavel Pavlenko]

HF Code Error: code - 400; msg - Bad Request

[Pavel Pavlenko]

curl to API is working

[Luc Georges]

For now ollama's API is not supported, it's on the todo list though!

cf huggingface/llm-ls#17

[Luc Georges]

Also created an issue for llama.cpp : huggingface/llm-ls#28

[github-actions]

This issue is stale because it has been open for 30 days with no activity.

[Flavio Del Grosso]

+1

[github-actions]

This issue is stale because it has been open for 30 days with no activity.

[jefffortune]

Is there a timeline for when feat: Add adaptors for ollama and openai #117 might be merged?

[Luc Georges]

Finishing the last touches of fixes on llm-ls and testing everything works as expected for 0.5.0 and we should be good to go for a release. I'd say either I find some time this week-end or next week :)