Giters

scaleapi / llm-engine

Scale LLM Engine public repository

Home Page:https://llm-engine.scale.com

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

⚠️ LLM Engine fine-tuning maintenance ⚠️

yixu34 opened this issue · comments

commented

We are doing some maintenance for LLM Engine fine-tuning events. See email text below:

Begin email text

Dear Spellbook users,

We are performing some maintenance of LLM Engine this week. If you are not an LLM Engine user, you can disregard this email. Otherwise, please read on.

How Will You Be Affected?
During the maintenance window, any fine-tuned models that you’ve created will be unavailable to serve traffic. Requests sent to those models will return with a HTTP 503 error. You will be able to create fine-tunes throughout the maintenance window, but the resulting models will be subject to the maintenance schedule downtime.

Note that the Completions API for base models is unaffected. Also, any files uploaded through the Files API will be unaffected. There is no loss of any of your metadata or data as a result of this maintenance.

Maintenance Schedule
LLM Engine fine-tuned models will be unavailable between Monday August 28th 22:00 PDT and Thursday August 31st 09:00 PDT. After 09:00 PDT on August 31st, fine-tuned models will be available again.

Why is this Maintenance Necessary?
This maintenance is necessary to roll out performance enhancements that will increase throughput and reduce cost/token for fine-tuned endpoints. As we prepare to transition LLM Engine out of evaluation mode and into production, these enhancements are necessary to scale to your production workloads.

What Do You Need to Do?
We kindly request that you take the following steps:

  1. Communicate with your team: Please inform your team members about this maintenance so that they are aware of the downtimes.
  2. Stay informed: We will provide updates via this Github issue. Please follow the issue and comment there for any additional assistance you might need.

Thank you for using LLM Engine. We look forward to delivering an even better experience after this maintenance.

Thanks,
-Yi

End email text

commented

Closing this out, fix should be rolled out to all users.

commented

Hi there, I was trying to fine-tune the Llama-2-7b model today but it is still raising an Internal Server Error. Is there any fix for this?

commented

Hi @PranjalKalbag , an internal server error probably wouldn't be related to this - can you look at #272 to see if that's the same symptom you see? If not, feel free to create a new issue with your specific error.

commented

Yup, it was the same error. Thanks for the help!