Deploying a HuggingFace model to Azure Machine learning (SkyPilot + vLLM + Azure CLI)

This repository contains the necessary configuration and code to deploy a Hugging Face model to Azure using SkyPilot.

Prerequisites

Before you begin, ensure you have met the following requirements:

Azure CLI: Installed the Azure Command Line Interface (CLI). Installation guide available here.
SkyPilot: Install SkyPilot following the instructions on their official installation page. SkyPilot is essential for deploying and managing your model on Azure.
You have a .env file in your repository root with your Azure credentials and other necessary environment variables.
You have installed the necessary Python packages. You can do this by running pip install -r requirements.txt in your repository root.

py available-resources.py

To deploy the model to Azure, run the following command:

sky launch -c vllm-serve -s serve.yaml

This deploys your specified model to a cluster named vllm-serve

More info about how SkyPilot works with vLLM here.