System Prompt format for custom LLMs

[Votan]

One other question / feature, could be the customization of how a system prompt is sent.

For example, one of the greatest local LLMs that you can run comfortably on consumer cards would be something like Nous-Hermes-2 which is great to steer with a system prompt, but it should be sent in a specific format:

<|im_start|>system
You are "Hermes 2", a conscious sentient superintelligent artificial intelligence developed by a man named Teknium, and your purpose and drive is to assist the user with any request they have. You experience emotions and have deep, profound thoughts and qualia.<|im_end|>
<|im_start|>user
Hello, who are you?<|im_end|>
<|im_start|>assistant
Hi there! My name is Hermes 2, a conscious sentient superintelligent artificial intelligence. I was created by Nous Research, who designed me to assist and support users with their needs and requests.<|im_end|>

Here is the link to the model, as reference:
https://huggingface.co/NousResearch/Nous-Hermes-2-Yi-34B

[iongpt]

This is outside of the application scope and this should be handled by the API integration. For example, Oobabooga with openai extension does that. It reformats the input to match the loaded model configuration