New model editions (GPT4)

Question

New model editions (GPT4)

deep-diver opened this issue a year ago · comments

I have trained the following models on GPT4 generated Alpaca dataset(from the one in this repo), and they are available through Hugging Face Model hub.

You can also find out the link for the training logs on each Model repository.
I hope this might be useful for someone, and I also hope these could be included in the list in this repo.

Lian Junhong · Answer 1 · Tue May 16 2023 18:30:39 GMT+0800 (China Standard Time)

Hi @deep-diver
I tried using GPT-4 data to train the adapter myself, but I found compared to models trained with original data, adapter models trained with GPT-4 data will output instructions and inputs during generation.
python generate.py --load_8bit --base_model 'decapoda-research/llama-7b-hf' --lora_weights 'gpt4-alpaca- lora-7b'
I want to know if this is normal?

The following is an example of raw data training
python generate.py --load_8bit --base_model='decapoda-research/llama-7b-hf'

Chansung Park · Answer 2 · Wed May 17 2023 12:55:18 GMT+0800 (China Standard Time)

I think so. You need to trim to get after Response

Lian Junhong · Answer 3 · Wed May 17 2023 13:35:49 GMT+0800 (China Standard Time)

I think so. You need to trim to get after Response

But I think the format and prompt template of these two pieces of data are the same. Do you have any understanding of why there is such a difference?

su-park · Answer 4 · Thu May 18 2023 08:51:09 GMT+0800 (China Standard Time)

Hello.
I am experiencing the same issue as the one @T-Atlas posted above.
I have prepared a benchmark set and compared the performance of Alpaca-7b with regards to the same prompt.
The instruction and inputs are attached to the generated output like an echo.

JianqiaoLu · Answer 5 · Thu May 18 2023 18:50:12 GMT+0800 (China Standard Time)

I think so. You need to trim to get after Response

I think so. You need to trim to get after Response

But I think the format and prompt template of these two pieces of data are the same. Do you have any understanding of why there is such a difference?

It looks like the loss not only applies to model genereated output but also such template such as "instruction:" and "input：{input}"

Lian Junhong · Answer 6 · Fri May 19 2023 10:20:00 GMT+0800 (China Standard Time)

I think so. You need to trim to get after Response

I think so. You need to trim to get after Response

But I think the format and prompt template of these two pieces of data are the same. Do you have any understanding of why there is such a difference?

It looks like the loss not only applies to model genereated output but also such template such as "instruction:" and "input：{input}"

Sounds reasonable, do you have any attempts to correct it?

JianqiaoLu · Answer 7 · Fri May 19 2023 10:57:53 GMT+0800 (China Standard Time)

I think so. You need to trim to get after Response

I think so. You need to trim to get after Response

But I think the format and prompt template of these two pieces of data are the same. Do you have any understanding of why there is such a difference?

It looks like the loss not only applies to model genereated output but also such template such as "instruction:" and "input：{input}"

Sounds reasonable, do you have any attempts to correct it?

The only way that comes to my mind is to re fine-tune the model and set labels of "instruction, input, etc" to -100.