[v0.8.3] serper, serpstack API, local web search not working

[Adhish Thite]

Context

I have serper.dev API key, serpstack API key and I have put it correctly in my .env.local file.

Issue

However, even if I enable Web Search, it still does not reach out to those APIs, and shows me "an error occured" no the Web Search part.

I don't see calls reaching Serper and SerpStack as well.

It was working for a bit on v0.8.2, but then it stopped working there as well. Now, for v.0.8.3, it's not working at all. Am I missing something? I have tried using either of those APIs too, but it still does not work.

Please help.

[Adhish Thite]

cc @nsarrazin

[Nathan Sarrazin]

Oh USE_LOCAL_WEBSEARCH will attempt to search the web by fetching google search directly and parsing the resulting page. It will override other websearch APIs. It's a bit more unstable than using an API, try removing this flag and let me know if that works. (it was meant more as a backup for users that can't/ don't want to use an API)

Also we don't support using multiple search queries at the same time so if you set both serpapi and serper, the priority will be given to serper with our current setup.

[Adhish Thite]

@nsarrazin

Thanks! I disabled the local web search and kept only Serper.

I see, but here's what's happening now. Even though Serper received the query, the UI shows an error.

Is there something I am missing?

[Nathan Sarrazin]

Can you expand the web search to see where the error lies ?

[Adhish Thite]

Similar experience with SerpStack:

Seems like the API is being hit, but it's not being considered in the LLM response.

[Adhish Thite]

@nsarrazin here you go

[Nathan Sarrazin]

Do you have a custom TEXT_EMBEDDING_MODELS maybe ? Looks like an error with embedding generation

[Adhish Thite]

@nsarrazin Yes, I have this custom embedding:

TEXT_EMBEDDING_MODELS=`[
    {
        "name":"azure-openai-text-embedding-3-large",
        "displayName":"Azure OpenAI Text Embedding 3 Large",
        "description":"Azure OpenAI Text Embedding 3 Large model hosted on Azure. This model is capable of generating embeddings for text data and can be used for a variety of tasks such as semantic search, clustering, and more.",
        "chunkCharLength":8191,
        "endpoints":[
            {
                "type":"openai",
                "baseURL":"https://MYDEP-dev.openai.azure.com/openai/deployments/text-embedding-3-large",
                "defaultHeaders":{
                    "api-key":"xxxx"
                },
                "defaultQuery":{
                    "api-version":"2024-02-01"
                }
            }
        ]
    }
]`

[Adhish Thite]

@nsarrazin Is there a way to check if the custom text embedding model is set up correctly? So that this error can be caught before.

[Nathan Sarrazin]

It's kind of hard to test right now, you should probably add a bunch of logs in src/lib/server/embeddingEndpoints/openai/embeddingEndpoints.ts . I haven't really used the openai embeddings yet so it could be just broken, I'll investigate.

In the meantime the default embedding shouldn't require any external API and just runs on the CPU, if that helps 😅

[Adhish Thite]

@nsarrazin This is working! I think I will have to re-check the OpenAI embeddings. I will have around 1000+ users once I deploy, so an API based embedding approach was desirable.

[Nathan Sarrazin]

For sure!

You can also deploy text-embeddings-inference, either self-hosted or using an Inference Endpoint. This is what we use for HuggingChat (config) and it handles the load pretty well.

But we of course have a goal of supporting a wide variety of endpoints so i'll try to see what the issue is with the OpenAI endpoint type. 🤔

[Adhish Thite]

Thanks so much! @nsarrazin Closing this issue, but please try to fix the OpenAI issue. I am taking a look at it myself too.

[Adhish Thite]

@nsarrazin Please take a look: #1077