Responses get cut off in the middle.
nickorrnah opened this issue · comments
I often find that whenever I have a response around 2 paragraphs or so in length that the response just gets cut off and does not fully complete. Is there a config setting to increase max response length or some other way to get around this issue?
Increase your max_new_tokens in the config.yml file to like 1500 and that will sort out the problem
Thank you for the help! I hadn't thought of looking at the ctransformers documentation options.