Stream option
rasbt opened this issue · comments
Add a stream=True option to generate
Hey @rasbt, are you looking for contributions to this enhancement? If so, can you share your ideas on where and how to start with streaming so I can contribute?
Thanks for your interest in this. This was requested by a colleague for a downstream application, but I haven't had a chance to think this through in terms of how this would look like (am currently traveling and haven't had a chance to follow up). I added it as a note for now but will probably have to implement it in tandem with the application.
@rasbt Got it, thank you.