Chatbot Clinic

Developing virtual characters for use with large language models can be a process that more closely resembles alchemy than engineering. Endless tinkering around with character descriptions, followed by unscientific "tests" where a few chat messages are exchanged with the bot, and at last a semi-educated guess on whether the change has been an improvement or not.

Chatbot Clinic is an extension for text-generation-webui that automates and formalizes that process. It brings science to the art of creating chatbots.

How it works

1️⃣ Configure as many chatbots as you like

Each one can have its own context and generation parameters. This allows you to comparatively evaluate both character descriptions and parameter presets, or any combination thereof.

2️⃣ Chat with all of them simultaneously

When you send a message to the chat, you receive a reply from each of the configured bots. Clicking on your preferred reply continues the chat with that reply, and records this as a vote for the bot that wrote it.

The replies are presented in random order and without identifying the bots, to avoid any potential source of bias and ensure that your vote is based solely on the content of the replies.

3️⃣ See which one you voted for the most

Numbers talk, bullshit walks! The statistics view gives you a breakdown of how many votes you gave to each bot's replies, so you can make an informed decision on which bot is "best", without having to rely on any hunches.

Installation

Installing Chatbot Clinic follows the standard process for installing text-generation-webui extensions, namely:

Install and run text-generation-webui.
Navigate to the Session tab.
Paste the URL https://github.com/p-e-w/chatbot_clinic into the textbox labeled Install or update an extension and press Enter.
Restart text-generation-webui.
In the Session tab, check the entry chatbot_clinic under Available extensions, then click Save UI defaults to settings.yaml, then click Apply flags/extensions and restart.
Make sure you have installed and loaded a language model.
Navigate to the Chatbot Clinic tab to use the extension.

License

This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License along with this program. If not, see https://www.gnu.org/licenses/.

By contributing to this project, you agree to release your contributions under the same license.

p-e-w / chatbot_clinic