h2oai / h2o-llmstudio

H2O LLM Studio - a framework and no-code GUI for fine-tuning LLMs. Documentation: https://docs.h2o.ai/h2o-llmstudio/

Home Page:https://h2o.ai

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[FEATURE] Apple Silicon: Add support of the mlx-lm

qdrddr opened this issue Β· comments

πŸš€ Feature

Hi, can you please consider adding Apple Silicon support?
mlx-lm makes it very easy to fine-tune LLM. That'll be great having UI for this framework from H2O LLM Studio.
pip install mlx-lm torch sentencepiece
https://medium.com/@anchen.li/fine-tune-llama3-with-function-calling-via-mlx-lm-5ebbee41558f

Motivation

Utilize powerful M-Ultra GPUs

To answer why:

  • Powerful macs: MaxStudio or Mac Pro with M2 Ultra and 196GB RAM available for GPU with 60 cores.
  • Apple is developing Apple Silicon Project ACDC designed specifically for server farms dedicated to AI processing.