Giters

Lightning-Universe / stable-diffusion-deploy

Learn to serve Stable Diffusion models on cloud infrastructure at scale. This Lightning App shows load-balancing, orchestrating, pre-provisioning, dynamic batching, GPU-inference, micro-services working together via the Lightning Apps framework.

Home Page:https://lightning.ai/muse

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

model doesn't work properly with concurrent requests

aniketmaurya opened this issue · comments

commented

model doesn't work properly with concurrent requests. And Gradio is not respected enable_queue=True when we directly access the REST API.

commented

We should definitely add our own queue in the app, just a Python queue, nothing fancy.