lhenault / simpleAI

An easy way to host your own AI API and expose alternative models, while being compatible with "open" AI clients.

Home Page:https://pypi.org/project/simple-ai-server/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Incompatiable with Ray

Nintorac opened this issue · comments

Hey,

Trying to use ray serve in the backend to facilitate autobatching and I ran across this error. They seem to work fine together so I suggest dropping the minimum requirement of for gRPC to 1.49.1.. or even 1.42.0 unless you know of a reason why not?

Because no versions of ray match >2.4.0,<3.0.0
 and ray (2.4.0) depends on grpcio (>=1.42.0,<=1.49.1), ray (>=2.4.0,<3.0.0) requires grpcio (>=1.42.0,<=1.49.1).
And because simple-ai-server (0.2.1) depends on grpcio (>=1.51.3,<2.0.0)
 and no versions of simple-ai-server match >0.2.1,<0.3.0, ray (>=2.4.0,<3.0.0) is incompatible with simple-ai-server (>=0.2.1,<0.3.0).
So, because simple-rwkv depends on both simple-ai-server (^0.2.1) and ray (^2.4.0), version solving failed.

Thanks for the PR, please let me know how it's going with ray and auto batching, that looks promising.