Giters

mistralai / mistral-inference

Official inference library for Mistral models

Home Page:https://mistral.ai/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Mistral 7B v0.1 does not support optimum BetterTransformers for better and optimized Inference

KaifAhmad1 opened this issue · comments

commented

Raising issue: Facing GPU resource constraints with Mistral-7B-v0.1. Seeking optimizations for VRAM usage and inference performance. Considering alternative solutions due to BetterTransformers not being supported. Open to collaboration on resolving this.