RuntimeErrror ipex_duplicate_import_error
idkSeth opened this issue · comments
Describe the bug
When trying to use IPEX LLM with LlamaIndex by following their example, a usage error occurs as IPEX is already imported.
How to reproduce
Follow the LlamaIndex example by importing IpexLLM and loading a model.
# Transform a string into input zephyr-specific input
def completion_to_prompt(completion):
return f"<|system|>\n</s>\n<|user|>\n{completion}</s>\n<|assistant|>\n"
# Transform a list of chat messages into zephyr-specific input
def messages_to_prompt(messages):
prompt = ""
for message in messages:
if message.role == "system":
prompt += f"<|system|>\n{message.content}</s>\n"
elif message.role == "user":
prompt += f"<|user|>\n{message.content}</s>\n"
elif message.role == "assistant":
prompt += f"<|assistant|>\n{message.content}</s>\n"
# ensure we start with a system prompt, insert blank if needed
if not prompt.startswith("<|system|>\n"):
prompt = "<|system|>\n</s>\n" + prompt
# add final assistant prompt
prompt = prompt + "<|assistant|>\n"
return prompt
from llama_index.llms.ipex_llm import IpexLLM
llm = IpexLLM.from_model_id(
model_name="HuggingFaceH4/zephyr-7b-alpha",
tokenizer_name="HuggingFaceH4/zephyr-7b-alpha",
context_window=512,
max_new_tokens=128,
generate_kwargs={"do_sample": False},
completion_to_prompt=completion_to_prompt,
messages_to_prompt=messages_to_prompt,
device_map="xpu",
)
Screenshots
If applicable, add screenshots to help explain the problem
Environment information
Attached output of env-check.sh. I have an iGPU that is not being detected but works(unrelated).
Additional context
Replacing ipex_importer.py with the version from BigDL fixes this issue.
Setting BIGDL_CHECK_DUPLICATE_IMPORT=0 in env can also disable this duplicate checker. For example,
export BIGDL_CHECK_DUPLICATE_IMPORT=0