[FeatureRequest] Add streaming support to openAI compatible api.
s-kostyaev opened this issue · comments
For now api not fully compatible with openAI one, it lacks streaming support. I would like to use it with streaming support.This one https://github.com/ml-explore/mlx-examples/blob/main/llms/mlx_lm/SERVER.md supports it and has MIT license.
Yes, the server is not fully implemented yet - good suggestion!