run_localGPT.py fail: python run_localGPT.py --device_type cpu
chujb opened this issue · comments
Sytem OS：windows 11 + intel cpu
I deploy the localGPT in the Window PC，but when run the command of "python run_localGPT.py --device_type cpu"，I am getting issue like：
////////// python ingest.py --device_type cpu
(localGPT) D:\dev-llama\meta-llama\localGPT>python ingest.py --device_type cpu
2024-01-16 01:33:57,136 - INFO - ingest.py:144 - Loading documents from D:\dev-llama\meta-llama\localGPT/SOURCE_DOCUMENTS
Importing: constitution.pdf
D:\dev-llama\meta-llama\localGPT/SOURCE_DOCUMENTS\constitution.pdf loaded.
2024-01-16 01:34:07,972 - INFO - ingest.py:153 - Loaded 1 documents from D:\dev-llama\meta-llama\localGPT/SOURCE_DOCUMENTS
2024-01-16 01:34:07,972 - INFO - ingest.py:154 - Split into 72 chunks of text
2024-01-16 01:34:08,822 - INFO - SentenceTransformer.py:66 - Load pretrained SentenceTransformer: hkunlp/instructor-large
load INSTRUCTOR_Transformer
max_seq_length 512
(localGPT) D:\dev-llama\meta-llama\localGPT>
////////// Excecute run_localGPT.py
(localGPT) D:\dev-llama\meta-llama\localGPT>python run_localGPT.py --device_type cpu
2024-01-16 00:55:44,272 - INFO - run_localGPT.py:241 - Running on: cpu
2024-01-16 00:55:44,272 - INFO - run_localGPT.py:242 - Display Source Documents set to: False
2024-01-16 00:55:44,272 - INFO - run_localGPT.py:243 - Use history set to: False
2024-01-16 00:55:44,855 - INFO - SentenceTransformer.py:66 - Load pretrained SentenceTransformer: hkunlp/instructor-large
load INSTRUCTOR_Transformer
max_seq_length 512
2024-01-16 00:55:46,427 - INFO - run_localGPT.py:59 - Loading Model: TheBloke/Llama-2-7b-Chat-GGUF, on: cpu
2024-01-16 00:55:46,427 - INFO - run_localGPT.py:60 - This action can take a few minutes!
2024-01-16 00:55:46,427 - INFO - load_models.py:38 - Using Llamacpp for GGUF/GGML quantized models
Traceback (most recent call last):
File "D:\dev-llama\meta-llama\localGPT\localGPT\Lib\site-packages\langchain\llms\llamacpp.py", line 149, in validate_environment
from llama_cpp import Llama
ModuleNotFoundError: No module named 'llama_cpp'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "D:\dev-llama\meta-llama\localGPT\run_localGPT.py", line 282, in
main()
File "D:\dev-llama\meta-llama\localGPT\localGPT\Lib\site-packages\click\core.py", line 1157, in call
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\dev-llama\meta-llama\localGPT\localGPT\Lib\site-packages\click\core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "D:\dev-llama\meta-llama\localGPT\localGPT\Lib\site-packages\click\core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\dev-llama\meta-llama\localGPT\localGPT\Lib\site-packages\click\core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\dev-llama\meta-llama\localGPT\run_localGPT.py", line 249, in main
qa = retrieval_qa_pipline(device_type, use_history, promptTemplate_type=model_type)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\dev-llama\meta-llama\localGPT\run_localGPT.py", line 138, in retrieval_qa_pipline
llm = load_model(device_type, model_id=MODEL_ID, model_basename=MODEL_BASENAME, LOGGING=logging)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\dev-llama\meta-llama\localGPT\run_localGPT.py", line 64, in load_model
llm = load_quantized_model_gguf_ggml(model_id, model_basename, device_type, LOGGING)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\dev-llama\meta-llama\localGPT\load_models.py", line 56, in load_quantized_model_gguf_ggml
return LlamaCpp(**kwargs)
^^^^^^^^^^^^^^^^^^
File "D:\dev-llama\meta-llama\localGPT\localGPT\Lib\site-packages\langchain\load\serializable.py", line 74, in init
super().init(**kwargs)
File "pydantic\main.py", line 339, in pydantic.main.BaseModel.init
File "pydantic\main.py", line 1102, in pydantic.main.validate_model
File "D:\dev-llama\meta-llama\localGPT\localGPT\Lib\site-packages\langchain\llms\llamacpp.py", line 153, in validate_environment
raise ImportError(
ImportError: Could not import llama-cpp-python library. Please install the llama-cpp-python library to use this embedding model: pip install llama-cpp-python
////////// pip install llama-cpp-python
(localGPT) D:\dev-llama\meta-llama\localGPT>pip install llama-cpp-python
Collecting llama-cpp-python
Using cached llama_cpp_python-0.2.28.tar.gz (9.4 MB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: typing-extensions>=4.5.0 in d:\dev-llama\meta-llama\localgpt\localgpt\lib\site-packages (from llama-cpp-python) (4.9.0)
Requirement already satisfied: numpy>=1.20.0 in d:\dev-llama\meta-llama\localgpt\localgpt\lib\site-packages (from llama-cpp-python) (1.26.3)
Collecting diskcache>=5.6.1 (from llama-cpp-python)
Using cached diskcache-5.6.3-py3-none-any.whl.metadata (20 kB)
Using cached diskcache-5.6.3-py3-none-any.whl (45 kB)
Building wheels for collected packages: llama-cpp-python
Building wheel for llama-cpp-python (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building wheel for llama-cpp-python (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [9 lines of output]
*** scikit-build-core 0.7.1 using CMake 3.27.0 (wheel)
*** Configuring CMake...
2024-01-16 00:59:39,932 - scikit_build_core - WARNING - Can't find a Python library, got libdir=None, ldlibrary=None, multiarch=None, masd=None
loading initial cache file C:\Users\妤氭睙鍖梊AppData\Local\Temp\tmpuyv5k66v\build\CMakeInit.txt
-- Building for: Visual Studio 17 2022
-- The C compiler identification is MSVC 19.38.33134.0
-- The CXX compiler identification is MSVC 19.38.33134.0
***** CMake configuration failed**
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for llama-cpp-python
Failed to build llama-cpp-python
ERROR: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based projects
there are two problems：
1、C:\Users\妤氭睙鍖梊AppData\Local\Temp\tmpuyv5k66v\build\CMakeInit.txt
This path is not exist, why is this error reported?
2、ERROR: Failed building wheel for llama-cpp-python
I search from the google, but have no usefull information to solve the problem. Has anyone meet the same problems and solved it?
How to fix this issue help me out, thanks very much!