Custom Functions & Retrival of Documents, Notes... with PrivateGPT or Anything LLM

[Rodrigo Garcia Saura]

Hi! I'm trying to get a chatbot that can do custom functions and get context from Notes. PrivateGPT & Anything LLM seem like two good options as they let you upload data to a vector database and then retrive it.

I need help setting something with these characteristics in place. If you like this idea please say something on the comments or up vote this pls! Help make this possible 🙏🏼

[Jens Leinenbach]

PrivateGPT may work out of the box as it follows the OpenAI API standard, but without the custom API features.

The project provides an API offering all the primitives required to build private, context-aware AI applications. It follows and extends the OpenAI API standard, and supports both normal and streaming responses.

[Rodrigo Garcia Saura]

I tried PRIVATEGPT with all possible combinations without success. I cant get another thing other than "detail": "Not Found", so it's reaching the server, but in an incompatible way or I'm doing something wrong.... 😭

Any help / ideas would be apreciated! @jekalmin @jleinenbach 😄 🚀

[Jens Leinenbach]

When I search for the error on Google, I find several possible causes. One possibility could be that the base URL is not configured correctly.
Did you try http://100.126.37.110:8001/api/v1 as base URL?

[Rodrigo Garcia Saura]

As I understand it combines this two fields. but I also did try several combinations without success. Apreciate quick response!

[Jens Leinenbach]

Even if you check "Skip Authentication", you still have to enter an API key value. This is a bit confusing.
So as you set it to "-", the api_key variable may still be filled with this value and you need to trust the software not to set an API key.

According to the error message, your error happens in helpers.py: "validate authentication".

This is the code:

async def validate_authentication(
    hass: HomeAssistant,
    api_key: str,
    base_url: str,
    api_version: str,
    organization: str = None,
    skip_authentication=False,
) -> None:
    if skip_authentication:
        return

    if is_azure(base_url):
        client = AsyncAzureOpenAI(
            api_key=api_key,
            azure_endpoint=base_url,
            api_version=api_version,
            organization=organization,
        )
    else:
        client = AsyncOpenAI(
            api_key=api_key, base_url=base_url, organization=organization
        )

    await client.models.list(timeout=10)

This function should do nothing if you check "Skip Authentication", so I guess you didn't check it it when this exception was logged.
What I can see is that if it's not Azure, the api_version seems to be igored.

At this point, I'd use this base URL:
http://100.126.37.110:8001/api/v1

On the first look, I didn't find a way how to create an API key as I would try it with a valid API key next:
I guess you installed an OpenAI Compatible Server as an LLM Backend:
https://docs.privategpt.dev/manual/advanced-setup/llm-backends
https://docs.vllm.ai/en/latest/getting_started/quickstart.html#openai-compatible-server