How can load this model by torch_dtype=torch.bfloat16
A-Salty-Fish opened this issue · comments
open-compass / MixtralKit
A toolkit for inference and evaluation of 'mixtral-8x7b-32kseqlen' from Mistral AI
A toolkit for inference and evaluation of 'mixtral-8x7b-32kseqlen' from Mistral AI
A-Salty-Fish opened this issue · comments