Model exports for Tritonserver

[Ivan Borko]

Hi! First of all, great repo and very useful. I've been using it with your model export (from huggingface) and everything works great.

But now I have to deploy a different version of Stable Diffusion with Triton server. I saw that you mentioned that you used this script: https://github.com/harishanand95/diffusers/blob/dml/examples/inference/save_onnx.py
for model export. That script works for fp32, but how you exported it for fp16? Have you used some kind of converter? Do you have an example of fp16 export?

Thanks :)

[Kamal Raj Kanakarajan]

Hi @iborko,

import onnx
import onnxconverter_common

new_onnx_model = onnxconverter_common.convert_float_to_float16_model_path("stable-diffusion-v1-4-onnx/unet/1/unet.onnx",keep_io_types=True)
onnx.save(new_onnx_model, 'stable-diffusion-v1-4-onnx/unet/1/unet_fp16.onnx')

[Ivan Borko]

Thank you very much!

[lolagiscard]

Hi @iborko

Can you give some details about your hardware and the versions of diffusers and transformers you have been testing to make this Repo running ?

Thanks

[Ivan Borko]

I tested this repository on both Amazon G4dn and G5 instances.

You can find exact package versions in my fork: https://github.com/velebit-ai/stable-diffusion-tritonserver

Version from this repo doesn't work anymore because many packages changed their latest versions (including pytorch, diffusers, ...)

[Kamal Raj Kanakarajan]

Please checkout v2

let me know any issues