Ability to await the result of infer
Lordxan opened this issue · comments
Lordxan commented
I was wondering is it possible to use this library without callbacks?
Dan Forbes commented
No, this is not possible at this time. I believe the goal of this library is to leave "higher-level" abstractions such as that up to the user. For now, you'd probably want to use probably want to use use a combination of InferenceSession::feed_prompt
and InferenceSession::infer_next_token
to implement this. I'm going to close this Issue, but please feel free to re-open it or open another one if you have more questions.