Giters

nlpxucan / WizardLM

LLMs build upon Evol Insturct: WizardLM, WizardCoder, WizardMath

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Prompt template when doing lora fine tune

gptzerozero opened this issue · comments

commented

Hi, I am learning about QLoRA finetuning and decided to start off by creating a LoRA adapter for the base model WizardLM 30B https://huggingface.co/TheBloke/WizardLM-30B-fp16.

Is there a prompt template available? Does the Alpaca prompt template work for WizardLM 30B?

For example, if I have the question "What is the color of a clear sky? and the answer "A clear sky is blue in color.", how should my dataset row be formatted for QLoRA training?