Mistral 7B v0.1 does not support optimum BetterTransformers for better and optimized Inference
KaifAhmad1 opened this issue · comments
Mohd Kaif commented
Raising issue: Facing GPU resource constraints with Mistral-7B-v0.1. Seeking optimizations for VRAM usage and inference performance. Considering alternative solutions due to BetterTransformers not being supported. Open to collaboration on resolving this.