A program error in create_trigger.py for sentiment analysis task

[尹子豪]

When running the python file create_trigger.py (for sentiment analysis task) , the training process can run normally, but when the program step into the part of saving the template to json file, an KeyError rasied. I think the error is because that the the only two key for the dict is "[X]" and "[Y]", but the program tends to get the value of "label", so the error raised. Moreover, I think the fuction __call__ of class TriggerTemplatizer is used for load dataset, so the error raised when the same function is used to get the best template. So I don't know how to fix it, looking forward to your reply!

Traceback (most recent call last):
File "/home/yzh/remote_folder/Major_Code/autoprompt-master/autoprompt/create_trigger.py", line 527, in
run_model(args)
File "/home/yzh/remote_folder/Major_Code/autoprompt-master/autoprompt/create_trigger.py", line 448, in run_model
'obj_label': tokenizer.lama_y,
File "/home/yzh/remote_folder/Major_Code/autoprompt-master/autoprompt/utils.py", line 157, in __call__
label = format_kwargs[self._label_field]
KeyError: 'label'

The error code is shown in the following figure

[Taylor Shin]

Hi Jason,

Can you share the command you used?

I tried the following and it gave me no errors

python -m autoprompt.create_trigger --train data/sentiment_analysis/train.tsv --dev data/sentiment_analysis/dev.tsv --template '<s> {sentence} [T] [T] [T] [P] . </s>' --num-cand 10 --accumulation-steps 1 --model-name roberta-base --bsz 8 --eval-size 8 --iters 1 --label-map '{"0": ["Ġworse", "Ġincompetence", "ĠWorse", "Ġblamed", "Ġsucked"], "1": ["ĠCris", "Ġmarvelous", "Ġphilanthrop", "Ġvisionary", "Ġwonderful"]}'

[尹子豪]

Thanks a lot for you replying me about my questions, I'll try the command that you provided later! Thank you again！

[尹子豪]

Dear Taylorshin,
After using the command you provided, the "create_trigger. py" file can run normally. Thank you again for your reply to my question! By the way, after running "create_trigger. py", how can I evaluate the trigger words obtained from "create_trigger. py" for sentiment analysis task? Wishing for your reply!

[Taylor Shin]

To evaluate the trigger, you would do the following

• Append the trigger tokens to each sentence from the test set
• Pass in each prompt into the model of your choice
• Get the probabilities for each class label by marginalizing over the set of associated label tokens (i.e. label map argument from the command I posted above)
• Compute the average accuracy across the test set

Figure 1 of the paper has a good explanation of the process.

[尹子豪]

OK, thank you a lot to reply for me, I'll try to evluate the trigger by following your instructions, thanks a lot!