XueFuzhao / OpenMoE

A family of open-sourced Mixture-of-Experts (MoE) Large Language Models

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

ValueError: Cannot find backend for cpu

duterscmy opened this issue · comments

when inference using cpu on colab, raise this error both on chat-7b and base model;
x = LlamaActCombine.apply(x_gate, x) File "/usr/local/lib/python3.10/dist-packages/torch/autograd/function.py", line 539, in apply return super().apply(*args, **kwargs) # type: ignore[misc] File "/usr/local/lib/python3.10/dist-packages/torch/cuda/amp/autocast_mode.py", line 113, in decorate_fwd return fwd(*args, **kwargs) File "/content/ColossalAI/colossalai/kernel/triton/llama_act_combine_kernel.py", line 150, in forward _llama_act_combine_forward[(M,)](x_gate1, File "<string>", line 27, in _llama_act_combine_forward ValueError: Cannot find backend for cpu