A simple question regarding to the paper.
w32zhong opened this issue · comments
Not sure if here is the right place to ask, but I want to check my understanding here.
In the end of section 4.1, I quote here:
EAGLE generates a tree-structured draft. To enhance efficiency, we implement tree attention,
enabling the creation of a draft tree with a depth of m through m forward passes,
thereby encompassing more than m tokens.
Should it be "enabling the creation of a draft tree with a depth of m through 1 forward passes"? Should the point of draft tree be completing a tree of guess predictions in one forward pass?
Thanks!
Sorry, ignore my stupid question here. I guess it is because that section I am reading is the drafting process, not the verification process. Issue closed.