h2oai / h2o-llmstudio

H2O LLM Studio - a framework and no-code GUI for fine-tuning LLMs. Documentation: https://h2oai.github.io/h2o-llmstudio/

Home Page:https://gpt-gm.h2o.ai

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[CODE IMPROVEMENT] Flash_attn installation may be wrong if the wheel is cached

pascal-pfeiffer opened this issue · comments

🔧 Proposed code refactoring

The flash_attn installation command in the Makefile is not fail proof if the wheel is cached and build with a different cuda version:

RuntimeError: Failed to import transformers.models.mistral.modeling_mistral because of the following error (look up to see its traceback):
libcudart.so.12: cannot open shared object file: No such file or directory

Motivation

Make installation fail proof