Giters

mlcommons / inference

Reference implementations of MLPerf™ inference benchmarks

Home Page:https://mlcommons.org/en/groups/inference

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Question: LLama-70B - Calibration and Quantization

rgandikota opened this issue · comments

commented

  1. In the Orca dataset, there is a Calibration pkl file which is generated after the preprocessing step.
    Can this be used as the calibration dataset or should we use the cnn-daily dataset which has been used in other language benchmarks.

  2. The current README file(https://github.com/mlcommons/inference/blob/e39003a9c4c89a2215db0ca57ad7a57b16f9a785/language/llama2-70b/README.md) doesn't give any pointers towards calibration or quantization. We are submitting for the first time and would appreciate any help we can get.