How to run LLama8b/70b using FP8

[Vishal Goklani]

Are the instructions available to converting to FP8?

I'd like to try converting both the 8B and 70B to FP8 and compare.

Thank you!

[Aritra Roy Gosthipaty]

Hey @vgoklani

As far as I understand it, this notebook does just what you are asking for. 🤗