Check for supported file formats when displaying available inference engines
dadmobile opened this issue · comments
Specifically MLX only supports some weight file formats (safetensors and nfz I think?). We currently only check architecture which means you sometimes get a "No safetensors for..." error when trying to run a model with MLX.
There are several possible ways to address:
- check weight file formats (currently there is a formats array in model gallery)
- use allow_patterns to see if there's a supported file type (lots of reasons this might not work)
- take advantage if we add MLX field to models that says minimum version required to support like transformers (separate issue)
- create some other way for a plugin to take a model and return if it's supported (kind of ugly but more flexible)