entity_graph 0 <graphml xmlns="http://graphml.graphdrawing.or... 🚀 create_summarized_entities entity_graph 0 <graphml xmlns="http://graphml.graphdrawing.or... ❌ create_base_entity_graph None ⠦ GraphRAG Indexer ├── Loading Input (text) - 4 files loaded (0 filtered) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00 0:00:00 ├── create_base_text_units ├── create_base_extracted_entities ├── create_summarized_entities └── create_base_entity_graph ❌ Errors occurred during the pipeline run, see logs for more details.

Question

entity_graph 0 <graphml xmlns="http://graphml.graphdrawing.or... 🚀 create_summarized_entities entity_graph 0 <graphml xmlns="http://graphml.graphdrawing.or... ❌ create_base_entity_graph None ⠦ GraphRAG Indexer ├── Loading Input (text) - 4 files loaded (0 filtered) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00 0:00:00 ├── create_base_text_units ├── create_base_extracted_entities ├── create_summarized_entities └── create_base_entity_graph ❌ Errors occurred during the pipeline run, see logs for more details.

CarolVim opened this issue 4 months ago · comments

                                   entity_graph

0 <graphml xmlns="http://graphml.graphdrawing.or...
🚀 create_summarized_entities
entity_graph
0 <graphml xmlns="http://graphml.graphdrawing.or...
❌ create_base_entity_graph
None
⠦ GraphRAG Indexer
├── Loading Input (text) - 4 files loaded (0 filtered) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00 0:00:00
├── create_base_text_units
├── create_base_extracted_entities
├── create_summarized_entities
└── create_base_entity_graph
❌ Errors occurred during the pipeline run, see logs for more details.

REVANTH PHANI SAI MEDUKONDURU · Answer 1 · Wed Jul 10 2024 14:20:53 GMT+0800 (China Standard Time)

@CarolVim Paste the full error you have faced from logs file (indexing-engine.log from outputs/reports folder).

Beckett · Answer 2 · Wed Jul 10 2024 17:50:01 GMT+0800 (China Standard Time)

I ran into this as well. The issue for me, which could be yours as well, was when I copied over the settings.yaml to the ./ragtest dir (mv settings.yaml ./ragtest), it reverted back to the original non-local version somehow. If you just manually copy/paste or add the ollama setting.yaml to the ./ragtest dir then it should work just fine

chanchen · Answer 3 · Wed Jul 10 2024 20:15:09 GMT+0800 (China Standard Time)

I ran into this as well. The issue for me, which could be yours as well, was when I copied over the settings.yaml to the ./ragtest dir (mv settings.yaml ./ragtest), it reverted back to the original non-local version somehow. If you just manually copy/paste or add the ollama setting.yaml to the ./ragtest dir then it should work just fine

thanks for your suggestions,but it didn't work.

chanchen · Answer 4 · Wed Jul 10 2024 20:19:45 GMT+0800 (China Standard Time)

d

i check the log the content might be that:4:04:34,932 root ERROR error extracting graph
Traceback (most recent call last):
File "/Users/chanchen/Desktop/ollama_graphrag/graphrag-local-ollama/graphrag/index/graph/extractors/graph/graph_extractor.py", line 118, in call
result = await self._process_document(text, prompt_variables)
File "/Users/chanchen/Desktop/ollama_graphrag/graphrag-local-ollama/graphrag/index/graph/extractors/graph/graph_extractor.py", line 146, in _process_document
response = await self._llm(
File "/Users/chanchen/Desktop/ollama_graphrag/graphrag-local-ollama/graphrag/llm/openai/json_parsing_llm.py", line 34, in call
result = await self._delegate(input, **kwargs)
File "/Users/chanchen/Desktop/ollama_graphrag/graphrag-local-ollama/graphrag/llm/openai/openai_token_replacing_llm.py", line 37, in call
return await self._delegate(input, **kwargs)
File "/Users/chanchen/Desktop/ollama_graphrag/graphrag-local-ollama/graphrag/llm/openai/openai_history_tracking_llm.py", line 33, in call
output = await self._delegate(input, **kwargs)
File "/Users/chanchen/Desktop/ollama_graphrag/graphrag-local-ollama/graphrag/llm/base/caching_llm.py", line 104, in call
result = await self._delegate(input, **kwargs)
File "/Users/chanchen/Desktop/ollama_graphrag/graphrag-local-ollama/graphrag/llm/base/rate_limiting_llm.py", line 177, in call
result, start = await execute_with_retry()
File "/Users/chanchen/Desktop/ollama_graphrag/graphrag-local-ollama/graphrag/llm/base/rate_limiting_llm.py", line 159, in execute_with_retry
async for attempt in retryer:
File "/Users/chanchen/miniforge3/lib/python3.10/site-packages/tenacity/_asyncio.py", line 71, in anext
do = self.iter(retry_state=self._retry_state)
File "/Users/chanchen/miniforge3/lib/python3.10/site-packages/tenacity/init.py", line 325, in iter
raise retry_exc.reraise()
File "/Users/chanchen/miniforge3/lib/python3.10/site-packages/tenacity/init.py", line 158, in reraise
raise self.last_attempt.result()
File "/Users/chanchen/miniforge3/lib/python3.10/concurrent/futures/_base.py", line 451, in result
return self.__get_result()
File "/Users/chanchen/miniforge3/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
raise self._exception
File "/Users/chanchen/Desktop/ollama_graphrag/graphrag-local-ollama/graphrag/llm/base/rate_limiting_llm.py", line 165, in execute_with_retry
return await do_attempt(), start
File "/Users/chanchen/Desktop/ollama_graphrag/graphrag-local-ollama/graphrag/llm/base/rate_limiting_llm.py", line 147, in do_attempt
return await self._delegate(input, **kwargs)
File "/Users/chanchen/Desktop/ollama_graphrag/graphrag-local-ollama/graphrag/llm/base/base_llm.py", line 49, in call
return await self._invoke(input, **kwargs)
File "/Users/chanchen/Desktop/ollama_graphrag/graphrag-local-ollama/graphrag/llm/base/base_llm.py", line 53, in _invoke
output = await self._execute_llm(input, **kwargs)
File "/Users/chanchen/Desktop/ollama_graphrag/graphrag-local-ollama/graphrag/llm/openai/openai_chat_llm.py", line 55, in _execute_llm
completion = await self.client.chat.completions.create(
File "/Users/chanchen/miniforge3/lib/python3.10/site-packages/openai/resources/chat/completions.py", line 1289, in create
return await self._post(
File "/Users/chanchen/miniforge3/lib/python3.10/site-packages/openai/_base_client.py", line 1816, in post
return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
File "/Users/chanchen/miniforge3/lib/python3.10/site-packages/openai/_base_client.py", line 1514, in request
return await self._request(
File "/Users/chanchen/miniforge3/lib/python3.10/site-packages/openai/_base_client.py", line 1610, in _request
raise self._make_status_error_from_response(err.response) from None
openai.InternalServerError: Error code: 502
14:04:34,932 graphrag.index.reporting.file_workflow_callbacks INFO Entity Extraction Error details={'doc_index': 0, 'text': 'block, where filters (or kernels) slide over the input data to produce feature maps. These filters are trained to recognize various patterns such as edges, textures, and shapes. Following convolutional layers, pooling layers (such as max pooling) are used to reduce the spatial dimensions of the feature maps, thereby decreasing the computational load and controlling overfitting. The fully connected layers, typically at the end of the network, take the high-level filtered and pooled features and perform the final classification or regression task. CNNs also often incorporate activation functions like ReLU (Rectified Linear Unit) and regularization techniques such as dropout to enhance performance and prevent overfitting.\n\nApplications of Convolutional Neural Networks\nConvolutional Neural Networks have revolutionized various applications across multiple domains. In computer vision, CNNs are the backbone of image classification models like AlexNet, VGGNet, and ResNet, which have achieved state-of-the-art performance on benchmark datasets such as ImageNet. Object detection frameworks like YOLO (You Only Look Once) and Faster R-CNN leverage CNNs to identify and localize objects within images. In medical imaging, CNNs assist in diagnosing diseases by analyzing X-rays, MRIs, and CT scans. Beyond vision tasks, CNNs are employed in natural language processing for tasks like sentence classification and text generation when combined with other architectures. Additionally, CNNs are used in video analysis, facial recognition systems, and even in autonomous vehicles for tasks like lane detection and obstacle recognition'}
14:04:34,950 datashaper.workflow.workflow INFO executing verb snapshot
14:04:34,961 datashaper.workflow.workflow INFO executing verb merge_graphs
14:04:34,976 datashaper.workflow.workflow INFO executing verb snapshot_rows
14:04:34,978 graphrag.index.emit.parquet_table_emitter INFO emitting parquet table create_base_extracted_entities.parquet
14:04:35,325 graphrag.index.run INFO Running workflow: create_summarized_entities...
14:04:35,331 graphrag.index.run INFO dependencies for create_summarized_entities: ['create_base_extracted_entities']
14:04:35,333 graphrag.index.run INFO read table from storage: create_base_extracted_entities.parquet
14:04:35,360 datashaper.workflow.workflow INFO executing verb summarize_descriptions
14:04:35,379 datashaper.workflow.workflow INFO executing verb snapshot_rows
14:04:35,381 graphrag.index.emit.parquet_table_emitter INFO emitting parquet table create_summarized_entities.parquet
14:04:35,553 graphrag.index.run INFO Running workflow: create_base_entity_graph...
14:04:35,553 graphrag.index.run INFO dependencies for create_base_entity_graph: ['create_summarized_entities']
14:04:35,553 graphrag.index.run INFO read table from storage: create_summarized_entities.parquet
14:04:35,570 datashaper.workflow.workflow INFO executing verb cluster_graph
14:04:35,571 graphrag.index.verbs.graph.clustering.cluster_graph WARNING Graph has no nodes
14:04:35,575 datashaper.workflow.workflow ERROR Error executing verb "cluster_graph" in create_base_entity_graph: Columns must be same length as key
Traceback (most recent call last):
File "/Users/chanchen/miniforge3/lib/python3.10/site-packages/datashaper/workflow/workflow.py", line 410, in _execute_verb
result = node.verb.func(**verb_args)
File "/Users/chanchen/Desktop/ollama_graphrag/graphrag-local-ollama/graphrag/index/verbs/graph/clustering/cluster_graph.py", line 102, in cluster_graph
output_df[[level_to, to]] = pd.DataFrame(
File "/Users/chanchen/miniforge3/lib/python3.10/site-packages/pandas/core/frame.py", line 4299, in setitem
self._setitem_array(key, value)
File "/Users/chanchen/miniforge3/lib/python3.10/site-packages/pandas/core/frame.py", line 4341, in _setitem_array
check_key_length(self.columns, key, value)
File "/Users/chanchen/miniforge3/lib/python3.10/site-packages/pandas/core/indexers/utils.py", line 390, in check_key_length
raise ValueError("Columns must be same length as key")
ValueError: Columns must be same length as key
14:04:35,580 graphrag.index.reporting.file_workflow_callbacks INFO Error executing verb "cluster_graph" in create_base_entity_graph: Columns must be same length as key details=None
14:04:35,580 graphrag.index.run ERROR error running workflow create_base_entity_graph
Traceback (most recent call last):
File "/Users/chanchen/Desktop/ollama_graphrag/graphrag-local-ollama/graphrag/index/run.py", line 323, in run_pipeline
result = await workflow.run(context, callbacks)
File "/Users/chanchen/miniforge3/lib/python3.10/site-packages/datashaper/workflow/workflow.py", line 369, in run
timing = await self._execute_verb(node, context, callbacks)
File "/Users/chanchen/miniforge3/lib/python3.10/site-packages/datashaper/workflow/workflow.py", line 410, in _execute_verb
result = node.verb.func(**verb_args)
File "/Users/chanchen/Desktop/ollama_graphrag/graphrag-local-ollama/graphrag/index/verbs/graph/clustering/cluster_graph.py", line 102, in cluster_graph
output_df[[level_to, to]] = pd.DataFrame(
File "/Users/chanchen/miniforge3/lib/python3.10/site-packages/pandas/core/frame.py", line 4299, in setitem
self._setitem_array(key, value)
File "/Users/chanchen/miniforge3/lib/python3.10/site-packages/pandas/core/frame.py", line 4341, in _setitem_array
check_key_length(self.columns, key, value)
File "/Users/chanchen/miniforge3/lib/python3.10/site-packages/pandas/core/indexers/utils.py", line 390, in check_key_length
raise ValueError("Columns must be same length as key")
ValueError: Columns must be same length as key
14:04:35,580 graphrag.index.reporting.file_workflow_callbacks INFO Error running pipeline! details=None

Patrick Wood · Answer 5 · Sat Jul 13 2024 04:41:13 GMT+0800 (China Standard Time)

I get the same error (and same log report). I am using llama3 and mxbai-embed-large:

graphrag-local-ollama: diff settings.yaml testrag
7c7
<   model: mistral
---
>   model: llama3:8b-instruct-q8_0
34c34
<     model: nomic_embed_text
---
>     model: mxbai-embed-large:latest

Patrick Wood · Answer 6 · Sat Jul 13 2024 05:19:08 GMT+0800 (China Standard Time)

So I tried mistral, and it ran to completion. When I tried gemma2, it ran further, but got an error that I also get with microsoft/graphrag:

...
17:15:18,108 graphrag.index.run ERROR error running workflow create_final_community_reports
Traceback (most recent call last):
  File "/mnt/nvme/src/ai/graphrag-local-ollama/graphrag/index/run.py", line 323, in run_pipeline
    result = await workflow.run(context, callbacks)
  File "/home/pwood/.cache/pypoetry/virtualenvs/graphrag-PFIj19e_-py3.10/lib/python3.10/site-packages/datashaper/workflow/workflow.py", line 369, in run
    timing = await self._execute_verb(node, context, callbacks)
  File "/home/pwood/.cache/pypoetry/virtualenvs/graphrag-PFIj19e_-py3.10/lib/python3.10/site-packages/datashaper/workflow/workflow.py", line 410, in _execute_verb
    result = node.verb.func(**verb_args)
  File "/home/pwood/.cache/pypoetry/virtualenvs/graphrag-PFIj19e_-py3.10/lib/python3.10/site-packages/datashaper/engine/verbs/window.py", line 73, in window
    window = __window_function_map[window_operation](input_table[column])
  File "/home/pwood/.cache/pypoetry/virtualenvs/graphrag-PFIj19e_-py3.10/lib/python3.10/site-packages/pandas/core/frame.py", line 4102, in __getitem__
    indexer = self.columns.get_loc(key)
  File "/home/pwood/.cache/pypoetry/virtualenvs/graphrag-PFIj19e_-py3.10/lib/python3.10/site-packages/pandas/core/indexes/range.py", line 417, in get_loc
    raise KeyError(key)
KeyError: 'community'
17:15:18,108 graphrag.index.reporting.file_workflow_callbacks INFO Error running pipeline! details=None

EDIT: also runs to completion with phi3 3.8b and phi3 14b.

chanchen · Answer 7 · Sun Jul 14 2024 19:56:43 GMT+0800 (China Standard Time)

所以我尝试了 mistral，它运行完成。当我尝试 gemma2 时，它继续运行，但出现了一个错误，我使用 microsoft/graphrag 时也遇到了这个错误：

...
17:15:18,108 graphrag.index.run ERROR error running workflow create_final_community_reports
Traceback (most recent call last):
  File "/mnt/nvme/src/ai/graphrag-local-ollama/graphrag/index/run.py", line 323, in run_pipeline
    result = await workflow.run(context, callbacks)
  File "/home/pwood/.cache/pypoetry/virtualenvs/graphrag-PFIj19e_-py3.10/lib/python3.10/site-packages/datashaper/workflow/workflow.py", line 369, in run
    timing = await self._execute_verb(node, context, callbacks)
  File "/home/pwood/.cache/pypoetry/virtualenvs/graphrag-PFIj19e_-py3.10/lib/python3.10/site-packages/datashaper/workflow/workflow.py", line 410, in _execute_verb
    result = node.verb.func(**verb_args)
  File "/home/pwood/.cache/pypoetry/virtualenvs/graphrag-PFIj19e_-py3.10/lib/python3.10/site-packages/datashaper/engine/verbs/window.py", line 73, in window
    window = __window_function_map[window_operation](input_table[column])
  File "/home/pwood/.cache/pypoetry/virtualenvs/graphrag-PFIj19e_-py3.10/lib/python3.10/site-packages/pandas/core/frame.py", line 4102, in __getitem__
    indexer = self.columns.get_loc(key)
  File "/home/pwood/.cache/pypoetry/virtualenvs/graphrag-PFIj19e_-py3.10/lib/python3.10/site-packages/pandas/core/indexes/range.py", line 417, in get_loc
    raise KeyError(key)
KeyError: 'community'
17:15:18,108 graphrag.index.reporting.file_workflow_callbacks INFO Error running pipeline! details=None

编辑：也可以使用 phi3 3.8b 和 phi3 14b 完成运行。

thank you your examples ,i will try it again.

happyrenxiaozhao · Answer 8 · Mon Jul 15 2024 20:47:17 GMT+0800 (China Standard Time)

hi, I have the same problem. Have you solved it?

chanchen · Answer 9 · Tue Jul 16 2024 00:35:47 GMT+0800 (China Standard Time)

it did'nt work.it made me sad.

chanchen · Answer 10 · Tue Jul 16 2024 00:38:02 GMT+0800 (China Standard Time)

I ran into this as well. The issue for me, which could be yours as well, was when I copied over the settings.yaml to the ./ragtest dir (mv settings.yaml ./ragtest), it reverted back to the original non-local version somehow. If you just manually copy/paste or add the ollama setting.yaml to the ./ragtest dir then it should work just fine

i saw you new project about adjust ollama and graphrag.i ran it on my macbook m1.but i unfortunately not ran it well.

happyrenxiaozhao · Answer 11 · Tue Jul 16 2024 09:47:50 GMT+0800 (China Standard Time)

it did'nt work.it made me sad.

hi, My settings are as follows：model:mistral:latest api_base: http://localhost:11434/v1
it works. The step Verb entity_extract has finished 21%. You can try it.

happyrenxiaozhao · Answer 12 · Tue Jul 16 2024 14:44:47 GMT+0800 (China Standard Time)

it did'nt work.it made me sad.

hi, My settings are as follows：model:mistral:latest api_base: http://localhost:11434/v1 it works. The step Verb entity_extract has finished 21%. You can try it.

sorry, it will stay at 21%, and fail. Very unfortunate!

雨飞 · Answer 13 · Tue Jul 16 2024 16:08:23 GMT+0800 (China Standard Time)

I also get this error with milkey/m3e embedding model.
My settings are as follows:
api_base: http://localhost:11434/v1
model: milkey/m3e

When I change the embedding model to nomic-embed-text, the error still exists.

happyrenxiaozhao · Answer 14 · Wed Jul 17 2024 16:12:49 GMT+0800 (China Standard Time)

@CarolVim hi, good news! i can run it successfully by looking the error logs and ollama Background server.
You should change the tool that loads the embedding model(nomic), like LM studio or xinference, I use the LM studio.

zqcrafts · Answer 15 · Thu Jul 18 2024 17:55:42 GMT+0800 (China Standard Time)

@CarolVim hi, good news! i can run it successfully by looking the error logs and ollama Background server. You should change the tool that loads the embedding model(nomic), like LM studio or xinference, I use the LM studio.

Thanks for your suggestion. How to change it to LM studio?

happyrenxiaozhao · Answer 16 · Thu Jul 18 2024 20:27:29 GMT+0800 (China Standard Time)

@CarolVim hi, good news! i can run it successfully by looking the error logs and ollama Background server. You should change the tool that loads the embedding model(nomic), like LM studio or xinference, I use the LM studio.

Thanks for your suggestion. How to change it to LM studio?
you first download this LM studio, and search the nomic-embed-text as your embedding model on the LM studio, start the server! Change the settings like as follows:(I download the nomic-embed-text-v1.5.Q5_K_M.gguf )
model: nomic-ai/nomic-embed-text-v1.5-GGUF/nomic-embed-text-v1.5.Q5_K_M.gguf
api_base: http://localhost:1234/v1
I hope this work for you!

zqcrafts · Answer 17 · Thu Jul 18 2024 21:37:57 GMT+0800 (China Standard Time)

Thanks for your prompt answer, but it still not works. Is there anything else that needs to be modified? My yaml is following：

encoding_model: cl100k_base
skip_workflows: []
llm:
  api_key: ${GRAPHRAG_API_KEY}
  type: openai_chat # or azure_openai_chat
  model: mistral
  model_supports_json: true # recommended if this is available for your model.
  # max_tokens: 4000
  # request_timeout: 180.0
  api_base: http://localhost:11434/v1
  # api_version: 2024-02-15-preview
  # organization: <organization_id>
  # deployment_name: <azure_model_deployment_name>
  # tokens_per_minute: 150_000 # set a leaky bucket throttle
  # requests_per_minute: 10_000 # set a leaky bucket throttle
  # max_retries: 10
  # max_retry_wait: 10.0
  # sleep_on_rate_limit_recommendation: true # whether to sleep when azure suggests wait-times
  # concurrent_requests: 25 # the number of parallel inflight requests that may be made

parallelization:
  stagger: 0.3
  # num_threads: 50 # the number of threads to use for parallel processing

async_mode: threaded # or asyncio

embeddings:
  ## parallelization: override the global parallelization settings for embeddings
  async_mode: threaded # or asyncio
  llm:
    api_key: lm-studio # ${GRAPHRAG_API_KEY}
    type: openai_embedding # or azure_openai_embedding
    model: nomic-ai/nomic-embed-text-v1.5-GGUF # /nomic-embed-text-v1.5.Q5_K_M.gguf # nomic_embed_text
    api_base: http://localhost:1234/v1  # http://localhost:11434/api
    # api_version: 2024-02-15-preview
    # organization: <organization_id>
    # deployment_name: <azure_model_deployment_name>
    # tokens_per_minute: 150_000 # set a leaky bucket throttle
    # requests_per_minute: 10_000 # set a leaky bucket throttle
    # max_retries: 10
    # max_retry_wait: 10.0
    # sleep_on_rate_limit_recommendation: true # whether to sleep when azure suggests wait-times
    # concurrent_requests: 25 # the number of parallel inflight requests that may be made
    # batch_size: 16 # the number of documents to send in a single request
    # batch_max_tokens: 8191 # the maximum number of tokens to send in a single request
    # target: required # or optional

happyrenxiaozhao · Answer 18 · Mon Jul 22 2024 19:08:53 GMT+0800 (China Standard Time)

@zqcrafts sorry, I forgot to reply to you. I use the qwen2: 7b , and you should check your log files.

zqcrafts · Answer 19 · Mon Jul 22 2024 19:35:21 GMT+0800 (China Standard Time)

@zqcrafts sorry, I forgot to reply to you. I use the qwen2: 7b , and you should check your log files.

Thank you. I reduced the input word size and found that it worked. The context length of the local model has a great influence.

huangbinghe · Answer 20 · Tue Jul 23 2024 19:41:57 GMT+0800 (China Standard Time)

@zqcrafts sorry, I forgot to reply to you. I use the qwen2: 7b , and you should check your log files.

I use qwen2, it is successfully

yakejiao · Answer 21 · Wed Jul 24 2024 17:29:50 GMT+0800 (China Standard Time)

please check .env file, modify
GRAPHRAG_API_KEY=<sk-***********>
to
GRAPHRAG_API_KEY=sk-***********

xuhe · Answer 22 · Tue Jul 30 2024 22:10:18 GMT+0800 (China Standard Time)

Thanks for the help from the comments above
I use qwen2 in ollama and use nomic-embed-text:latest in llama.cpp(Using llama.cpp may help those who use SSH and port forwarding. Using the LM Studio GUI is not very convenient.), it works.

use ./llama-server -ngl 100 -m ./models/nomic-embed-text-v1.5/nomic-embed-text-v1.5.Q5_K_M.gguf --embedding --port 11433 to start llama.cpp embedding model server (you can get .gguf model in hf https://hf-mirror.com/nomic-ai/nomic-embed-text-v1.5-GGUF)

encoding_model: cl100k_base
skip_workflows: []
llm:
  api_key: ${GRAPHRAG_API_KEY}
  type: openai_chat # or azure_openai_chat
  model: qwen2
  model_supports_json: true # recommended if this is available for your model.
  # max_tokens: 4096
  # request_timeout: 180.0
  api_base: http://localhost:11434/v1
  # api_version: 2024-02-15-preview
  # organization: <organization_id>
  # deployment_name: <azure_model_deployment_name>
  # tokens_per_minute: 150_000 # set a leaky bucket throttle
  # requests_per_minute: 10_000 # set a leaky bucket throttle
  max_retries: 3
  # max_retry_wait: 10.0
  # sleep_on_rate_limit_recommendation: true # whether to sleep when azure suggests wait-times
  concurrent_requests: 25 # the number of parallel inflight requests that may be made

parallelization:
  stagger: 0.3
  num_threads: 50 # the number of threads to use for parallel processing

async_mode: threaded # or asyncio

embeddings:
  ## parallelization: override the global parallelization settings for embeddings
  async_mode: threaded # or asyncio
  llm:
    api_key: ${GRAPHRAG_API_KEY}
    type: openai_embedding # or azure_openai_embedding
    model: nomic-embed-text:latest
    api_base: http://localhost:11433
    # api_version: 2024-02-15-preview
    # organization: <organization_id>
    # deployment_name: <azure_model_deployment_name>
    # tokens_per_minute: 150_000 # set a leaky bucket throttle
    # requests_per_minute: 10_000 # set a leaky bucket throttle
    max_retries: 3
    # max_retry_wait: 10.0
    # sleep_on_rate_limit_recommendation: true # whether to sleep when azure suggests wait-times
    concurrent_requests: 25 # the number of parallel inflight requests that may be made
    #batch_size: 1 # the number of documents to send in a single request
    #batch_max_tokens: 4000 # the maximum number of tokens to send in a single request
    # target: required # or optional

chunks:
  size: 512
  overlap: 64
  group_by_columns: [id] # by default, we don't allow chunks to cross documents

input:
  type: file # or blob
  file_type: text # or csv
  base_dir: "input"
  file_encoding: utf-8
  file_pattern: ".*\\.txt$"

cache:
  type: file # or blob
  base_dir: "cache"
  # connection_string: <azure_blob_storage_connection_string>
  # container_name: <azure_blob_storage_container_name>

storage:
  type: file # or blob
  base_dir: "output/${timestamp}/artifacts"
  # connection_string: <azure_blob_storage_connection_string>
  # container_name: <azure_blob_storage_container_name>

reporting:
  type: file # or console, blob
  base_dir: "output/${timestamp}/reports"
  # connection_string: <azure_blob_storage_connection_string>
  # container_name: <azure_blob_storage_container_name>

entity_extraction:
  ## llm: override the global llm settings for this task
  ## parallelization: override the global parallelization settings for this task
  ## async_mode: override the global async_mode settings for this task
  prompt: "prompts/entity_extraction.txt"
  entity_types: [organization, person, geo, event]
  max_gleanings: 0

summarize_descriptions:
  ## llm: override the global llm settings for this task
  ## parallelization: override the global parallelization settings for this task
  ## async_mode: override the global async_mode settings for this task
  prompt: "prompts/summarize_descriptions.txt"
  max_length: 500

claim_extraction:
  ## llm: override the global llm settings for this task
  ## parallelization: override the global parallelization settings for this task
  ## async_mode: override the global async_mode settings for this task
  # enabled: true
  prompt: "prompts/claim_extraction.txt"
  description: "Any claims or facts that could be relevant to information discovery."
  max_gleanings: 0

community_reports:
  ## llm: override the global llm settings for this task
  ## parallelization: override the global parallelization settings for this task
  ## async_mode: override the global async_mode settings for this task
  prompt: "prompts/community_report.txt"
  max_length: 2000
  max_input_length: 4000

cluster_graph:
  max_cluster_size: 10

embed_graph:
  enabled: false # if true, will generate node2vec embeddings for nodes
  # num_walks: 10
  # walk_length: 40
  # window_size: 2
  # iterations: 3
  # random_seed: 597832

umap:
  enabled: false # if true, will generate UMAP embeddings for nodes

snapshots:
  graphml: false
  raw_entities: false
  top_level_nodes: false

local_search:
  # text_unit_prop: 0.5
  # community_prop: 0.1
  # conversation_history_max_turns: 5
  # top_k_mapped_entities: 10
  # top_k_relationships: 10
  # max_tokens: 12000

global_search:
  # max_tokens: 12000
  # data_max_tokens: 12000
  # map_max_tokens: 1000
  # reduce_max_tokens: 2000
  # concurrency: 32

this is my setting.yaml

jwc19890114 · Answer 23 · Fri Aug 16 2024 17:05:31 GMT+0800 (China Standard Time)

@CarolVim hi, good news! i can run it successfully by looking the error logs and ollama Background server. You should change the tool that loads the embedding model(nomic), like LM studio or xinference, I use the LM studio.

Thanks for your suggestion. How to change it to LM studio?
you first download this LM studio, and search the nomic-embed-text as your embedding model on the LM studio, start the server! Change the settings like as follows:(I download the nomic-embed-text-v1.5.Q5_K_M.gguf )
model: nomic-ai/nomic-embed-text-v1.5-GGUF/nomic-embed-text-v1.5.Q5_K_M.gguf
api_base: http://localhost:1234/v1
I hope this work for you!

could you pls check my question, I have tried the LM-studio, but it not work

Grewizard11 · Answer 24 · Thu Aug 29 2024 17:50:51 GMT+0800 (China Standard Time)

Thanks for the help from the comments above I use qwen2 in ollama and use nomic-embed-text:latest in llama.cpp(Using llama.cpp may help those who use SSH and port forwarding. Using the LM Studio GUI is not very convenient.), it works. use ./llama-server -ngl 100 -m ./models/nomic-embed-text-v1.5/nomic-embed-text-v1.5.Q5_K_M.gguf --embedding --port 11433 to start llama.cpp embedding model server (you can get .gguf model in hf https://hf-mirror.com/nomic-ai/nomic-embed-text-v1.5-GGUF)

encoding_model: cl100k_base
skip_workflows: []
llm:
  api_key: ${GRAPHRAG_API_KEY}
  type: openai_chat # or azure_openai_chat
  model: qwen2
  model_supports_json: true # recommended if this is available for your model.
  # max_tokens: 4096
  # request_timeout: 180.0
  api_base: http://localhost:11434/v1
  # api_version: 2024-02-15-preview
  # organization: <organization_id>
  # deployment_name: <azure_model_deployment_name>
  # tokens_per_minute: 150_000 # set a leaky bucket throttle
  # requests_per_minute: 10_000 # set a leaky bucket throttle
  max_retries: 3
  # max_retry_wait: 10.0
  # sleep_on_rate_limit_recommendation: true # whether to sleep when azure suggests wait-times
  concurrent_requests: 25 # the number of parallel inflight requests that may be made

parallelization:
  stagger: 0.3
  num_threads: 50 # the number of threads to use for parallel processing

async_mode: threaded # or asyncio

embeddings:
  ## parallelization: override the global parallelization settings for embeddings
  async_mode: threaded # or asyncio
  llm:
    api_key: ${GRAPHRAG_API_KEY}
    type: openai_embedding # or azure_openai_embedding
    model: nomic-embed-text:latest
    api_base: http://localhost:11433
    # api_version: 2024-02-15-preview
    # organization: <organization_id>
    # deployment_name: <azure_model_deployment_name>
    # tokens_per_minute: 150_000 # set a leaky bucket throttle
    # requests_per_minute: 10_000 # set a leaky bucket throttle
    max_retries: 3
    # max_retry_wait: 10.0
    # sleep_on_rate_limit_recommendation: true # whether to sleep when azure suggests wait-times
    concurrent_requests: 25 # the number of parallel inflight requests that may be made
    #batch_size: 1 # the number of documents to send in a single request
    #batch_max_tokens: 4000 # the maximum number of tokens to send in a single request
    # target: required # or optional

chunks:
  size: 512
  overlap: 64
  group_by_columns: [id] # by default, we don't allow chunks to cross documents

input:
  type: file # or blob
  file_type: text # or csv
  base_dir: "input"
  file_encoding: utf-8
  file_pattern: ".*\\.txt$"

cache:
  type: file # or blob
  base_dir: "cache"
  # connection_string: <azure_blob_storage_connection_string>
  # container_name: <azure_blob_storage_container_name>

storage:
  type: file # or blob
  base_dir: "output/${timestamp}/artifacts"
  # connection_string: <azure_blob_storage_connection_string>
  # container_name: <azure_blob_storage_container_name>

reporting:
  type: file # or console, blob
  base_dir: "output/${timestamp}/reports"
  # connection_string: <azure_blob_storage_connection_string>
  # container_name: <azure_blob_storage_container_name>

entity_extraction:
  ## llm: override the global llm settings for this task
  ## parallelization: override the global parallelization settings for this task
  ## async_mode: override the global async_mode settings for this task
  prompt: "prompts/entity_extraction.txt"
  entity_types: [organization, person, geo, event]
  max_gleanings: 0

summarize_descriptions:
  ## llm: override the global llm settings for this task
  ## parallelization: override the global parallelization settings for this task
  ## async_mode: override the global async_mode settings for this task
  prompt: "prompts/summarize_descriptions.txt"
  max_length: 500

claim_extraction:
  ## llm: override the global llm settings for this task
  ## parallelization: override the global parallelization settings for this task
  ## async_mode: override the global async_mode settings for this task
  # enabled: true
  prompt: "prompts/claim_extraction.txt"
  description: "Any claims or facts that could be relevant to information discovery."
  max_gleanings: 0

community_reports:
  ## llm: override the global llm settings for this task
  ## parallelization: override the global parallelization settings for this task
  ## async_mode: override the global async_mode settings for this task
  prompt: "prompts/community_report.txt"
  max_length: 2000
  max_input_length: 4000

cluster_graph:
  max_cluster_size: 10

embed_graph:
  enabled: false # if true, will generate node2vec embeddings for nodes
  # num_walks: 10
  # walk_length: 40
  # window_size: 2
  # iterations: 3
  # random_seed: 597832

umap:
  enabled: false # if true, will generate UMAP embeddings for nodes

snapshots:
  graphml: false
  raw_entities: false
  top_level_nodes: false

local_search:
  # text_unit_prop: 0.5
  # community_prop: 0.1
  # conversation_history_max_turns: 5
  # top_k_mapped_entities: 10
  # top_k_relationships: 10
  # max_tokens: 12000

global_search:
  # max_tokens: 12000
  # data_max_tokens: 12000
  # map_max_tokens: 1000
  # reduce_max_tokens: 2000
  # concurrency: 32

this is my setting.yaml

我也用qwen2，你的setting起作用了，感谢老铁

MichaelPanwenjie · Answer 25 · Thu Sep 05 2024 02:35:30 GMT+0800 (China Standard Time)

it works？ i use the same setting as below，it still do not fixed

Mostafa Ghadimi · Answer 26 · Tue Sep 10 2024 04:42:33 GMT+0800 (China Standard Time)

openai.InternalServerError: Error code: 502

The above error code says that you are experiencing connectivity issue to your LLM model. If you are using local LLMs, make sure that you can connect to it using curl requests.

One possible issue is that using proxy make issue with the connectivity of your model. So make sure you deactivated the proxy for a while and test it agian.

Here is my configuration:

llm:
  api_key: ${GRAPHRAG_API_KEY}
  type: openai_chat
  model: llama3.1
  model_supports_json: true
  api_base: http://localhost:11434/v1 # this line!

embeddings:
  async_mode: threaded
  llm:
    api_key: ${GRAPHRAG_API_KEY}
    type: openai_embedding
    model: nomic-ai/nomic-embed-text-v1.5-GGUF/nomic-embed-text-v1.5.Q8_0.gguf
    api_base: http://127.0.0.1:1234/v1