Giters

guidance-ai / guidance

A guidance language for controlling large language models.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

`ModelStream` should be a drop-in replacement for `Model` or their API differences should be documented

hudson-ai opened this issue · comments

commented

ModelStreams (returned from Model.stream()) seem to be drop-in replacements for Model objects, at least when it comes to appending grammars. But they do not support:

  • __getitem__
  • __str__
  • role blocks (fail silently, similar to #916)

They should either be brought in line with ModelStream being a drop-in replacement, or additional documentation about the cases where this is not the case (and maybe warnings/exceptions should be raised?)

commented

For the role blocks, I made a temporary workaround at #943 (I didn't see this issue was mentioned before, sorry!)