[Bug]: Dimension of Ollama embedding is 1536 while the expected dimension is 4096

Question

[Bug]: Dimension of Ollama embedding is 1536 while the expected dimension is 4096

binhaoIsUsed opened this issue 3 months ago · comments

What happened?

When I'm going to chat with newly created brain via Ollama, get an error on "different vector dimensions". The expected dimension of embedding query is 4096. (chatgpt3.5-turbo-0125 is OK.)

Relevant log output

backend-core  | 2024-03-17 08:57:39,590:INFO - HTTP Request: GET http://host.docker.internal:54321/rest/v1/brains?select=%2A&brain_id=eq.b1c76842-fa7a-416e-b6a0-3f84fef9752f "HTTP/1.1 200 OK"
backend-core  | 2024-03-17 08:57:39,596:INFO - HTTP Request: GET http://host.docker.internal:54321/rest/v1/integrations_user?select=%2A&brain_id=eq.b1c76842-fa7a-416e-b6a0-3f84fef9752f&user_id=eq.39418e3b-0258-4452-af60-7acfcc1263ff "HTTP/1.1 200 OK"
backend-core  | 2024-03-17 08:57:39,601:INFO - HTTP Request: GET http://host.docker.internal:54321/rest/v1/integrations?select=%2A&id=eq.b37a2275-61b3-460b-b4ab-94dfdf3642fb "HTTP/1.1 200 OK"
backend-core  | 2024-03-17 08:57:39,606:INFO - HTTP Request: GET http://host.docker.internal:54321/rest/v1/brains_users?select=id%3Abrain_id%2C%20rights%2C%20brains%20%28id%3A%20brain_id%2C%20status%2C%20name%2C%20brain_type%2C%20description%29&user_id=eq.39418e3b-0258-4452-af60-7acfcc1263ff&brain_id=eq.b1c76842-fa7a-416e-b6a0-3f84fef9752f "HTTP/1.1 200 OK"
backend-core  | 2024-03-17 08:57:39,611:INFO - HTTP Request: GET http://host.docker.internal:54321/rest/v1/brains?select=%2A&brain_id=eq.b1c76842-fa7a-416e-b6a0-3f84fef9752f "HTTP/1.1 200 OK"
backend-core  | 2024-03-17 08:57:39,617:INFO - HTTP Request: GET http://host.docker.internal:54321/rest/v1/integrations_user?select=%2A&brain_id=eq.b1c76842-fa7a-416e-b6a0-3f84fef9752f&user_id=eq.39418e3b-0258-4452-af60-7acfcc1263ff "HTTP/1.1 200 OK"
backend-core  | 2024-03-17 08:57:39,621:INFO - HTTP Request: GET http://host.docker.internal:54321/rest/v1/integrations?select=%2A&id=eq.b37a2275-61b3-460b-b4ab-94dfdf3642fb "HTTP/1.1 200 OK"
backend-core  | INFO:     172.21.0.1:37134 - "GET /brains/b1c76842-fa7a-416e-b6a0-3f84fef9752f/ HTTP/1.1" 200 OK
backend-core  | INFO:     172.21.0.1:37134 - "OPTIONS /chat/6604e220-9b7d-4b82-b33f-fbd7038ec291/question/stream?brain_id=b1c76842-fa7a-416e-b6a0-3f84fef9752f HTTP/1.1" 200 OK
backend-core  | 2024-03-17 08:57:41,384:INFO - HTTP Request: GET http://host.docker.internal:54321/rest/v1/brains?select=%2A&brain_id=eq.b1c76842-fa7a-416e-b6a0-3f84fef9752f "HTTP/1.1 200 OK"
backend-core  | 2024-03-17 08:57:41,389:INFO - HTTP Request: GET http://host.docker.internal:54321/rest/v1/integrations_user?select=%2A&brain_id=eq.b1c76842-fa7a-416e-b6a0-3f84fef9752f&user_id=eq.39418e3b-0258-4452-af60-7acfcc1263ff "HTTP/1.1 200 OK"
backend-core  | 2024-03-17 08:57:41,392:INFO - HTTP Request: GET http://host.docker.internal:54321/rest/v1/integrations?select=%2A&id=eq.b37a2275-61b3-460b-b4ab-94dfdf3642fb "HTTP/1.1 200 OK"
backend-core  | 2024-03-17 08:57:41,395:INFO - HTTP Request: GET http://host.docker.internal:54321/rest/v1/brains_users?select=id%3Abrain_id%2C%20rights%2C%20brains%20%28id%3A%20brain_id%2C%20status%2C%20name%2C%20brain_type%2C%20description%29&user_id=eq.39418e3b-0258-4452-af60-7acfcc1263ff&brain_id=eq.b1c76842-fa7a-416e-b6a0-3f84fef9752f "HTTP/1.1 200 OK"
backend-core  | [INFO] modules.chat.controller.chat_routes [chat_routes.py:228]: Creating question for chat 6604e220-9b7d-4b82-b33f-fbd7038ec291 with brain b1c76842-fa7a-416e-b6a0-3f84fef9752f of type <class 'uuid.UUID'>
backend-core  | 2024-03-17 08:57:41,399:INFO - HTTP Request: GET http://host.docker.internal:54321/rest/v1/brains?select=%2A&brain_id=eq.b1c76842-fa7a-416e-b6a0-3f84fef9752f "HTTP/1.1 200 OK"
backend-core  | 2024-03-17 08:57:41,403:INFO - HTTP Request: GET http://host.docker.internal:54321/rest/v1/integrations_user?select=%2A&brain_id=eq.b1c76842-fa7a-416e-b6a0-3f84fef9752f&user_id=eq.39418e3b-0258-4452-af60-7acfcc1263ff "HTTP/1.1 200 OK"
backend-core  | 2024-03-17 08:57:41,406:INFO - HTTP Request: GET http://host.docker.internal:54321/rest/v1/integrations?select=%2A&id=eq.b37a2275-61b3-460b-b4ab-94dfdf3642fb "HTTP/1.1 200 OK"
backend-core  | 2024-03-17 08:57:41,409:INFO - HTTP Request: GET http://host.docker.internal:54321/rest/v1/brains_users?select=id%3Abrain_id%2C%20rights%2C%20brains%20%28id%3A%20brain_id%2C%20status%2C%20name%2C%20brain_type%2C%20description%29&user_id=eq.39418e3b-0258-4452-af60-7acfcc1263ff&brain_id=eq.b1c76842-fa7a-416e-b6a0-3f84fef9752f "HTTP/1.1 200 OK"
backend-core  | 2024-03-17 08:57:41,418:INFO - HTTP Request: GET http://host.docker.internal:54321/rest/v1/chat_history?select=%2A&chat_id=eq.6604e220-9b7d-4b82-b33f-fbd7038ec291&order=message_time "HTTP/1.1 200 OK"
backend-core  | 2024-03-17 08:57:41,427:INFO - HTTP Request: GET http://host.docker.internal:54321/rest/v1/brains?select=id%3Abrain_id%2C%20name%2C%20%2A&brain_id=eq.b1c76842-fa7a-416e-b6a0-3f84fef9752f "HTTP/1.1 200 OK"
backend-core  | 2024-03-17 08:57:41,601:INFO - HTTP Request: POST http://host.docker.internal:54321/rest/v1/rpc/match_brain "HTTP/1.1 400 Bad Request"
backend-core  | INFO:     172.21.0.1:37134 - "POST /chat/6604e220-9b7d-4b82-b33f-fbd7038ec291/question/stream?brain_id=b1c76842-fa7a-416e-b6a0-3f84fef9752f HTTP/1.1" 500 Internal Server Error
backend-core  | ERROR:    Exception in ASGI application
backend-core  | Traceback (most recent call last):
backend-core  |   File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py", line 408, in run_asgi
backend-core  |     result = await app(  # type: ignore[func-returns-value]
backend-core  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend-core  |   File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__
backend-core  |     return await self.app(scope, receive, send)
backend-core  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend-core  |   File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
backend-core  |     await super().__call__(scope, receive, send)
backend-core  |   File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
backend-core  |     await self.middleware_stack(scope, receive, send)
backend-core  |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
backend-core  |     raise exc
backend-core  |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
backend-core  |     await self.app(scope, receive, _send)
backend-core  |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__
backend-core  |     await self.simple_response(scope, receive, send, request_headers=headers)
backend-core  |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response
backend-core  |     await self.app(scope, receive, send)
backend-core  |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
backend-core  |     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
backend-core  |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
backend-core  |     raise exc
backend-core  |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
backend-core  |     await app(scope, receive, sender)
backend-core  |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 758, in __call__
backend-core  |     await self.middleware_stack(scope, receive, send)
backend-core  |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 778, in app
backend-core  |     await route.handle(scope, receive, send)
backend-core  |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 299, in handle
backend-core  |     await self.app(scope, receive, send)
backend-core  |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 79, in app
backend-core  |     await wrap_app_handling_exceptions(app, request)(scope, receive, send)
backend-core  |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
backend-core  |     raise exc
backend-core  |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
backend-core  |     await app(scope, receive, sender)
backend-core  |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 74, in app
backend-core  |     response = await func(request)
backend-core  |                ^^^^^^^^^^^^^^^^^^^
backend-core  |   File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app
backend-core  |     raw_response = await run_endpoint_function(
backend-core  |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend-core  |   File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
backend-core  |     return await dependant.call(**values)
backend-core  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend-core  |   File "/code/modules/chat/controller/chat_routes.py", line 232, in create_stream_question_handler
backend-core  |     gpt_answer_generator = get_answer_generator(
backend-core  |                            ^^^^^^^^^^^^^^^^^^^^^
backend-core  |   File "/code/modules/chat/controller/chat_routes.py", line 76, in get_answer_generator
backend-core  |     brain, metadata_brain = brain_service.find_brain_from_question(
backend-core  |                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend-core  |   File "/code/modules/brain/service/brain_service.py", line 109, in find_brain_from_question
backend-core  |     list_brains = vector_store.find_brain_closest_query(user.id, question)
backend-core  |                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend-core  |   File "/code/vectorstore/supabase.py", line 54, in find_brain_closest_query
backend-core  |     ).execute()
backend-core  |       ^^^^^^^^^
backend-core  |   File "/usr/local/lib/python3.11/site-packages/postgrest/_sync/request_builder.py", line 127, in execute
backend-core  |     raise APIError(r.json())
backend-core  | postgrest.exceptions.APIError: {'code': '22000', 'details': None, 'hint': None, 'message': 'different vector dimensions 1536 and 4096'}

Twitter / LinkedIn details

No response

dosubot · Answer 1 · Sun Mar 17 2024 17:07:16 GMT+0800 (China Standard Time)

Hey @binhaoIsUsed, nice to see you back! Diving into another challenge, I see? Let's crack this one together.

The error you're encountering is likely due to a mismatch between the vector dimensions expected by the find_brain_closest_query function and the actual dimensions of the vectors being generated by the Embeddings instance used in CustomSupabaseVectorStore.

The find_brain_closest_query function is trying to execute an RPC call with the query_embedding parameter, which is expected to be a vector of a certain dimension. The error message suggests that the function is expecting vectors of dimension 4096, but the vectors being generated by the Embeddings instance are of dimension 1536.

To resolve this issue, you need to ensure that the vectors generated by the Embeddings instance match the expected dimensions. This could involve:

Checking the configuration of the Embeddings instance to ensure it's set to generate vectors of the correct size (4096 in this case).
If the Embeddings instance is correct and cannot be changed, you may need to adjust the find_brain_closest_query function to accept vectors of size 1536.
Alternatively, if the find_brain_closest_query function cannot be changed, you might need to find a way to convert or adapt the 1536-dimensional vectors to the expected 4096-dimensional format, though this could potentially lead to loss of information or accuracy.

Unfortunately, without more information about the specific implementation of the Embeddings class used in CustomSupabaseVectorStore, it's difficult to provide a more specific solution. If you could provide more details about this, it would be helpful in diagnosing and resolving the issue.

Sources

backend/vectorstore/supabase.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

Stan Girard · Answer 2 · Mon Mar 18 2024 08:42:02 GMT+0800 (China Standard Time)

Yeah sorry about that the seeding of the database makes it so it only accepts OpenAI.

Could you delete all lines in the brain_vectors table on localhost:54323 ? And try again the procedure ?

Mirza Ismail Baig · Answer 3 · Mon Mar 18 2024 13:41:50 GMT+0800 (China Standard Time)

Would like to fix this bug....

Stan Girard · Answer 4 · Tue Mar 19 2024 00:17:45 GMT+0800 (China Standard Time)

Please go ahead :)

…

On Sun, Mar 17, 2024 at 22:42:11, Mirza Ismail Baig < ***@***.*** > wrote: Would like to fix this bug.... — Reply to this email directly, view it on GitHub ( #2356 (comment) ) , or unsubscribe ( https://github.com/notifications/unsubscribe-auth/AEVUW3B7QR2XPQFQNFNFHV3YYZ5DHAVCNFSM6AAAAABE2BK4ISVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBSHE3TEMJSHE ). You are receiving this because you commented. Message ID: <QuivrHQ/quivr/issues/2356/2002972129 @ github. com>

Marc Tanis · Answer 5 · Wed Apr 24 2024 02:39:04 GMT+0800 (China Standard Time)

Add a brain but getting this error even though the brains_vector table is empty