Evaluation code for Falcon models

[Abdelrahman Shaker]

Hi Authors,

Thank you for sharing the evaluation code for llama models. Could you please release the code for the evaluation of Falcon models?

Best regards,
Abdelrahman.

[Egiazarian Vage]

Hello!
To run Falcon for perplexity calculation use code in https://github.com/Vahe1994/SpQR/blob/main/main_for_rw.py , it is similar to main.py with some additional code to allow run falcon.
As for LM_eval - you can use lm_eval_main.py and provide falcon model in the args: --model hf-causal --model_args pretrained=tiiuae/falcon-40b.
Feel free to report any problem that you encounter.

P.s. We are working on combining scripts for Falcon and LLama and deduplicating code, see PR #7 .We will add more explicit instruction how to run Falcon in ReadMe.