Trying to run it dockerized and getting "HTTPConnectionPool(host='localhost', port=11434): Max retries exceeded with url"
BenBatsir opened this issue · comments
It works perfectly when running it locally (without docker), but when I tried to import a doc via UI (inside a container), I'm getting the bellow:
Generating embeddings: 0%| | 0/23 [00:00<?, ?it/s]Traceback (most recent call last):
privategpt-private-gpt-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/urllib3/connection.py", line 174, in _new_conn
privategpt-private-gpt-1 | conn = connection.create_connection(
privategpt-private-gpt-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/urllib3/util/connection.py", line 95, in create_connection
privategpt-private-gpt-1 | raise err
privategpt-private-gpt-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection
privategpt-private-gpt-1 | sock.connect(sa)
privategpt-private-gpt-1 | ConnectionRefusedError: [Errno 111] Connection refused
privategpt-private-gpt-1 |
privategpt-private-gpt-1 | During handling of the above exception, another exception occurred:
privategpt-private-gpt-1 |
privategpt-private-gpt-1 | Traceback (most recent call last):
privategpt-private-gpt-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 715, in urlopen
privategpt-private-gpt-1 | httplib_response = self._make_request(
privategpt-private-gpt-1 | ^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 416, in _make_request
privategpt-private-gpt-1 | conn.request(method, url, **httplib_request_kw)
privategpt-private-gpt-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/urllib3/connection.py", line 244, in request
privategpt-private-gpt-1 | super(HTTPConnection, self).request(method, url, body=body, headers=headers)
privategpt-private-gpt-1 | File "/usr/local/lib/python3.11/http/client.py", line 1286, in request
privategpt-private-gpt-1 | self._send_request(method, url, body, headers, encode_chunked)
privategpt-private-gpt-1 | File "/usr/local/lib/python3.11/http/client.py", line 1332, in _send_request
privategpt-private-gpt-1 | self.endheaders(body, encode_chunked=encode_chunked)
privategpt-private-gpt-1 | File "/usr/local/lib/python3.11/http/client.py", line 1281, in endheaders
privategpt-private-gpt-1 | self._send_output(message_body, encode_chunked=encode_chunked)
privategpt-private-gpt-1 | File "/usr/local/lib/python3.11/http/client.py", line 1041, in _send_output
privategpt-private-gpt-1 | self.send(msg)
privategpt-private-gpt-1 | File "/usr/local/lib/python3.11/http/client.py", line 979, in send
privategpt-private-gpt-1 | self.connect()
privategpt-private-gpt-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/urllib3/connection.py", line 205, in connect
privategpt-private-gpt-1 | conn = self._new_conn()
privategpt-private-gpt-1 | ^^^^^^^^^^^^^^^^
privategpt-private-gpt-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/urllib3/connection.py", line 186, in _new_conn
privategpt-private-gpt-1 | raise NewConnectionError(
privategpt-private-gpt-1 | urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0xffff609ddf10>: Failed to establish a new connection: [Errno 111] Connection refused
privategpt-private-gpt-1 |
privategpt-private-gpt-1 | During handling of the above exception, another exception occurred:
privategpt-private-gpt-1 |
privategpt-private-gpt-1 | Traceback (most recent call last):
privategpt-private-gpt-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
privategpt-private-gpt-1 | resp = conn.urlopen(
privategpt-private-gpt-1 | ^^^^^^^^^^^^^
privategpt-private-gpt-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 799, in urlopen
privategpt-private-gpt-1 | retries = retries.increment(
privategpt-private-gpt-1 | ^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/urllib3/util/retry.py", line 592, in increment
privategpt-private-gpt-1 | raise MaxRetryError(_pool, url, error or ResponseError(cause))
privategpt-private-gpt-1 | urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=11434): Max retries exceeded with url: /api/embeddings (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0xffff609ddf10>: Failed to establish a new connection: [Errno 111] Connection refused'))
privategpt-private-gpt-1 |
privategpt-private-gpt-1 | During handling of the above exception, another exception occurred:
privategpt-private-gpt-1 |
privategpt-private-gpt-1 | Traceback (most recent call last):
privategpt-private-gpt-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/queueing.py", line 495, in call_prediction
privategpt-private-gpt-1 | output = await route_utils.call_process_api(
privategpt-private-gpt-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/route_utils.py", line 235, in call_process_api
privategpt-private-gpt-1 | output = await app.get_blocks().process_api(
privategpt-private-gpt-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/blocks.py", line 1627, in process_api
privategpt-private-gpt-1 | result = await self.call_function(
privategpt-private-gpt-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/blocks.py", line 1173, in call_function
privategpt-private-gpt-1 | prediction = await anyio.to_thread.run_sync(
privategpt-private-gpt-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/anyio/to_thread.py", line 33, in run_sync
privategpt-private-gpt-1 | return await get_asynclib().run_sync_in_worker_thread(
privategpt-private-gpt-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
privategpt-private-gpt-1 | return await future
privategpt-private-gpt-1 | ^^^^^^^^^^^^
privategpt-private-gpt-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 807, in run
privategpt-private-gpt-1 | result = context.run(func, *args)
privategpt-private-gpt-1 | ^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/utils.py", line 690, in wrapper
privategpt-private-gpt-1 | response = f(*args, **kwargs)
privategpt-private-gpt-1 | ^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1 | File "/home/worker/app/private_gpt/ui/ui.py", line 252, in _upload_file
privategpt-private-gpt-1 | self._ingest_service.bulk_ingest([(str(path.name), path) for path in paths])
privategpt-private-gpt-1 | File "/home/worker/app/private_gpt/server/ingest/ingest_service.py", line 87, in bulk_ingest
privategpt-private-gpt-1 | documents = self.ingest_component.bulk_ingest(files)
privategpt-private-gpt-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1 | File "/home/worker/app/private_gpt/components/ingest/ingest_component.py", line 135, in bulk_ingest
privategpt-private-gpt-1 | saved_documents.extend(self._save_docs(documents))
privategpt-private-gpt-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1 | File "/home/worker/app/private_gpt/components/ingest/ingest_component.py", line 142, in _save_docs
privategpt-private-gpt-1 | self._index.insert(document, show_progress=True)
privategpt-private-gpt-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/llama_index/core/indices/base.py", line 225, in insert
privategpt-private-gpt-1 | nodes = run_transformations(
privategpt-private-gpt-1 | ^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/llama_index/core/ingestion/pipeline.py", line 127, in run_transformations
privategpt-private-gpt-1 | nodes = transform(nodes, **kwargs)
privategpt-private-gpt-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/llama_index/core/base/embeddings/base.py", line 334, in __call__
privategpt-private-gpt-1 | embeddings = self.get_text_embedding_batch(
privategpt-private-gpt-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/llama_index/core/base/embeddings/base.py", line 255, in get_text_embedding_batch
privategpt-private-gpt-1 | embeddings = self._get_text_embeddings(cur_batch)
privategpt-private-gpt-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/llama_index/embeddings/ollama/base.py", line 64, in _get_text_embeddings
privategpt-private-gpt-1 | embeddings = self.get_general_text_embedding(text)
privategpt-private-gpt-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/llama_index/embeddings/ollama/base.py", line 89, in get_general_text_embedding
privategpt-private-gpt-1 | response = requests.post(
privategpt-private-gpt-1 | ^^^^^^^^^^^^^^
privategpt-private-gpt-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/requests/api.py", line 115, in post
privategpt-private-gpt-1 | return request("post", url, data=data, json=json, **kwargs)
privategpt-private-gpt-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/requests/api.py", line 59, in request
privategpt-private-gpt-1 | return session.request(method=method, url=url, **kwargs)
privategpt-private-gpt-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
privategpt-private-gpt-1 | resp = self.send(prep, **send_kwargs)
privategpt-private-gpt-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
privategpt-private-gpt-1 | r = adapter.send(request, **kwargs)
privategpt-private-gpt-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
privategpt-private-gpt-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/requests/adapters.py", line 519, in send
privategpt-private-gpt-1 | raise ConnectionError(e, request=request)
privategpt-private-gpt-1 | requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=11434): Max retries exceeded with url: /api/embeddings (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0xffff609ddf10>: Failed to establish a new connection: [Errno 111] Connection refused'))
I'm pretty sure this has nothing to do with private-gpt but with networking shenanigans related to containers.
I've pretty much tried to dockerize it according to the docs
Is Ollama running from inside the container too? (localhost)
Is Ollama running from inside the container too? (localhost)
no
That's the issue. From the container localhost IS the container, therefore, it will never reach the Ollama instance.
That's the issue. From the container
localhostIS the container, therefore, it will never reach the Ollama instance.
it makes sense, but then again I'm surprised how come I need to change anything given I'm simply following the documentation.
Maybe I need to expose port 11434 to this container ?
Maybe I need to expose port 11434 to this container ?
No, that is pointless.
What you need to do instead is to replace occurrences of localhost:11434 with <host-ip>:11434. This way, the container will be able to connect to it.
Maybe I need to expose port 11434 to this container ?
No, that is pointless.
What you need to do instead is to replace occurrences of
localhost:11434with<host-ip>:11434. This way, the container will be able to connect to it.
Not sure I understand where exactly.
In the code itself?
private-gpt uses YAML files for its configuration.
The ollama address to connect to is hardcoded in these configuration files:
private-gpt/settings-ollama.yaml
Line 16 in c1802e7
What you need to do is to create your custom YAML file and use it in Docker (via a directory binding?).
AFAIK and since there's no environment variable to change that Ollama URL, you don't have other choice.
private-gptuses YAML files for its configuration. Theollamaaddress to connect to is hardcoded in these configuration files:private-gpt/settings-ollama.yaml
Line 16 in c1802e7
What you need to do is to create your custom YAML file and use it in Docker (via a directory binding?).
AFAIK and since there's no environment variable to change that Ollama URL, you don't have other choice.
ok... so localhost should be change to what? 127.0.0.1 ? something else?
I guess your machine has a local IP address ? Then use it.
I guess your machine has a local IP address ? Then use it.
I ended up changing it to the container name:
ollama:
llm_model: mistral
embedding_model: nomic-embed-text
api_base: http://privategpt-ollama-1:11434
embedding_api_base: http://privategpt-ollama-1:11434
Now I'm having a different issue:
ValueError: Ollama call failed with status code 404. Details: model 'nomic-embed-text' not found, try pulling it first
That's very good news. That means you succeeded to connect to Ollama.
Now you need to read and hopefully tackle the next issue.
To fix it, just do: ollama pull nomic-embed-text
That's very good news. That means you succeeded to connect to Ollama.
Now you need to read and hopefully tackle the next issue.
To fix it, just do:
ollama pull nomic-embed-text
it worked!
so should I add it in Dockerfile.external as a step ?
I have no knowledge in Dockerfile.external, but very curious to learn how you fixed the issue. It would be nice if you could document it here too for future reference.
I have no knowledge in
Dockerfile.external, but very curious to learn how you fixed the issue. It would be nice if you could document it here too for future reference.
actually this docker file belongs to the private-gpt image, so I'll need to figure this out somehow, but I will document it once I'll find a suitable solution.
Thanks a lot for your help
@BenBatsir You can't add this line to Dockerfile.external, as it is something you need to run on the ollama container.
In this situation, I have three ideas on how to fix it:
- Modify the command in
docker-composeand replace it with something like:
ollama pull nomic-embed-text && ollama pull mistral && ollama serve. - Open a bash terminal on the container and pull both models.
- Create a custom version of Ollama with the downloaded model.
Anyway, we will talk internally about this problem and try to fix a solution to prevent further related problems.