[FEATURE] Apple Silicon: Add support of the mlx-lm
qdrddr opened this issue Β· comments
qdrddr commented
π Feature
Hi, can you please consider adding Apple Silicon support?
mlx-lm makes it very easy to fine-tune LLM. That'll be great having UI for this framework from H2O LLM Studio.
pip install mlx-lm torch sentencepiece
https://medium.com/@anchen.li/fine-tune-llama3-with-function-calling-via-mlx-lm-5ebbee41558f
Motivation
Utilize powerful M-Ultra GPUs
qdrddr commented
To answer why:
- Powerful macs: MaxStudio or Mac Pro with M2 Ultra and 196GB RAM available for GPU with 60 cores.
- Apple is developing Apple Silicon Project ACDC designed specifically for server farms dedicated to AI processing.