MT-bench evaluation

[ATP-1010]

Thanks for your valuable work in MoE!
When I try to evaluate the openmoe-8b-chat model by MT-bench with this command:
python gen_model_answer.py --model-path OrionZheng/openmoe-8b-chat --model-id moe-8b-chat --dtype bfloat16.

It throws this error: AssertionError: assert max_length <= self.sin.shape[0]

It can be solved by setting the max_new_token to 512 (The default is 1024). But I can only get a 2.0 score on MT-bench.

Would you happen to have any suggestions?