SillyTavern / SillyTavern

LLM Frontend for Power Users.

Home Page:https://sillytavern.app

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[FEATURE_REQUEST] Extension Prompt Injection

Andybui1012 opened this issue · comments

Have you searched for similar requests?

Yes

Is your feature request related to a problem? If so, please describe.

Currently, the extension insert function like summary and objective only allow insert at the beginning or at some depth from the message being sent which interferes with context shift making the prompt processing to run again on the previous chat. This happens each message for depth and when updated for before/after story string.

Describe the solution you'd like

I think it is good to have another option added to stop the prompt processing from running over and over again on the same chat, which is simply to inject at a set position in chat directly as normal system message or during prompt combination like currently.

For example, for summary every 5 message, the summary can be injected at chat position 5, 10, 15 right after it is triggered. Added benefit of this is that the summary position would only contain info prior to it in the prompt.

Another nice thing to have to go along with this feature for summary would be to keep track of the previous summary position and run summary again when the previous summary is about to be cut off from context. This make sure the summary prompt is less likely made to run on same chat twice and also mean that the user don’t have to gauge the summary frequency anymore.

A draw back i could think of for this approach would be if the user change the message before a summary, then it could be harder to handle removing summary message from chat, instead of from memory like currently.

Describe alternatives you've considered

No response

Additional context

No response

Priority

Medium (Would be very useful)

Are you willing to test this on staging/unstable branch if this is implemented?

Yes