limit db result size to not "overflow" context
nilsherzig opened this issue · comments
Is your feature request related to a problem? Please describe.
Sometimes we get more result text back than there is model context, especially if the context is set to a low number. This results in the llm not adhering to the requested answer structure. Which results in parsing errors.
Describe the solution you'd like
Limit the amount of context we use with db / websearch results.