support incremental parsing with streams
Zane-XY opened this issue · comments
Zane XiaoYe commented
It's now common to parse Markdown text streams from LLMs, and these inputs are in token streams. However, when a client needs to incrementally render these tokens as they receive them, they must accumulate the prefix and parse them as a whole repeatedly. This process can be very inefficient. It would be beneficial if the parser could support input streams and incrementally parse tokens and trigger the event listeners for the node creation.