THUDM / GLM

GLM (General Language Model)

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Few-shot tests on GLM-10B

Vispstar-V opened this issue · comments

Hi, I'm trying to conduct a multiple-choice test on GLM-10B using few-shot prompting. I have tried to start a service using the following command:

bash scripts/generate_block.sh \
     config_tasks/model_blocklm_10B.sh

Then I input my few-shot prompts, such as:

Please answer this multiple-choice question.\nQuestion: Which instrument measures atmospheric pressure?\nOptions: A: Thermometer. B: Anemometer. C: Barometer. D: Hydrometer.\nAnswer: C.\n\nQuestion: Which gas is most commonly used in light bulbs?\nOptions: A: Nitrogen. B: Oxygen. C: Helium. D: Argon.\nAnswer: D.\n\nQuestion: What is the largest organ in the human body?\nOptions: A: Heart. B: Liver. C: Skin. D: Brain.\nAnswer: C.\n\nQuestion: What is the pH level of pure water?\nOptions: A: 5. B: 7. C: 9. D: 11.\nAnswer:[gMask]

But on most tests, the model fails to provide a correct option. I wonder if there is a flaw in my prompt design, or it is a common issue.
Thanks!