How to deploy this and host it on SageMaker?
zomglawlz opened this issue · comments
Hi, I went through the the example and can get it working in SageMaker notebooks. The issue I'm running into is deploying KeyBERT as an endpoint.
Normally, I train a huggingface model, and create an endpoint configuration for the resulting model, and then create an endpoint from that.
But since KeyBERT doesn't product a model, I'm looking for guidance on how I can create create an endpoint on SageMaker. Thanks
Although I have not used Amazon SageMaker endpoints before, I imagine you would need to create a custom solution for the model. In that sense, it would expect it to be much different from creating endpoints with scikit-learn.
Looking at how scikit-learn models get deployed on sagemaker, it looks similar to the huggingface process.
https://sagemaker.readthedocs.io/en/stable/frameworks/sklearn/using_sklearn.html#deploy-a-scikit-learn-model
In this example, the scikit-learn model is also first fine-tuned, and an estimator is created as a result.
That estimator can then be deployed to an endpoint.
Can I create an estimator for KeyBERT like the scikit-learn example?
If you directly can use scikit-learn-compatible models, then it would be possible to wrap KeyBERT in class that follows their conventions. However, it does seem like sagemaker is using some internal models to do this which means you might have to check their internal code to see how they are exactly doing this.