hpcaitech / EnergonAI

Large-scale model inference.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

num_beams for beam search

shammmmmless opened this issue · comments

Hi, I want to use num_beams for generate, but PipelineModel can't. Can you support num_beams?

Best wishes.

commented

Since we are working on imporving a large-scale model demo recently, the requirement might be delayed for one or two weeks.
You can use beam search in transformer and add it to your code following this.
Also, welcome you to contribute your beam code usage to the project.