by Starlight Robotics
ARCANE Manual is a curated repository of AI Role-playing LLMs, chatbots, services, apps, interfaces, offline tools, and hardware tips. This manual provides insight for where to start, and reference links for seasoned developers.
Local:
- Hardware is needed to run LLMs locally
- LLM Deployment of your own LLM
- Chatbot Frameworks for public presence
- Specialist models - medicine, finance, law
- ⭐ Big Code Models Leaderboard - coding models
Online:
- Online services and apps for Role-play are available, paid and free
- Image generation models can be found on CivitAI
Advancements:
- Challenges and Limits that we face and need to be solved
- Awesome LLM-Powered Agent - source for papers, specifically [Human Interaction Simulation](Awesome LLM-Powered Agent)
Help:
- ARCANE Manual has a FAQ section
- To suggest updates and edits or just introduce yourself, you can contact me on Reddit or Discord
-
PygmalionAI - All models: HuggingFace, Most popular of which as of October 2023 is Mythalion 13B - A merge of Pygmalion-2 13B and MythoMax 13B
-
MythoMax 13B - by Gryphe. Uncensored.
-
Manticore-13B - Uncensored.
-
Mistral 7B and SynthIA - Uncensored, trained on Mistral
-
CalliopeDS - This is a Llama 2-based model consisting of a merge of several models using a weight-adjusted TIES merge (Resolving Interference When Merging Models): jondurbin/airoboros-l2-13b-2.2, elinas/chronos-13b-v2, NousResearch/Nous-Hermes-Llama2-13b, lemonilia/limarp-llama2-v2, PygmalionAI/pygmalion-2-13b.
Language-Vision models
- LLaVA - trained on LLaMA 2
- BakLLaVA-1 - fine-tuned on Mistral
- 🤗 Open LLM Leaderboard
- Another LLM Roleplay Rankings
- Ayumi's LLM Role Play & ERP Ranking (NSFW)
- Rankings site for online NSFW chatbots
- Wiki by r/LocalLLaMA
For awareness, the scores are NOT representative of quality of prose, performance across the entire context window (i.e. ability to effectively recall information accurately), stability (i.e. logical flow, coherence) and actual RP performance. There are many models that score well that perform very poorly with actual use For example, in October 2023, Mythomax scores worse, but is better than a majority of the models that score better than it.
Recommendations for levels as of October 2023:
7b: Mistral and its variants, especially Mistral-OpenOrca and SynthIA - Mistral-7B in some cases already outperforms GPT-3.5
13b: Mythomax and its variants, Athena
20b: Emerhyst, MLewd-remm-l2-chat
33b: Airoboros, Chronoboros
70b+: Xwin, Synthia 1.5 - 70B outperform GPT-3.5 and rival GPT4
Some of the UIs, like SillyTavern, also have APIs, that can be used for own coding projects and automation.
Details
KoboldCpp is an easy-to-use AI text-generation software for GGML models. It's a single self contained distributable from Concedo, that builds off llama.cpp, and adds a versatile Kobold API endpoint, additional format support, backward compatibility, as well as a fancy UI with persistent stories, editing tools, save formats, memory, world info, author's note, characters, scenarios and everything Kobold and Kobold Lite have to offer.Details
From providing user donated backends as workers, and providing an application interface for running all sort of models - it is PygmalionAI's pick for running models locally.Discord: KoboldAI
Website: KoboldAI
Details
The original frontend. It lacks the options for the user on their chatting experience, however it is still very usable.Oobabooga Text Generation WebUI - Front-End
Details
Oobabooga is a web interface, in the same look as Stable Diffusion Web UI by Automatic1111. It has more features, and compatibility compared to KoboldAI running models locally.Discord: Oobabooga
SillyTavern - Power-user Front-End
Details
An open-source frontend for running LLM models. Provides all the tools to customize your chatting experience. An extensive modification of TavernAI.Discord: SillyTavern
API Documentation for SillyTavern
Details
An open-source web based AI agnostic roleplay chat. The backend for chats: NovelAI, Horde, or other services. It can be ran locally, or use the production website.Discord: Agnaistic (link broken)
oterm - CLI UI
Sample prompts and character sheets by Botprompts
The format example:
Personality ( "cute" + "gentle" + "intelligent" + "etc" ) Loves ( "food" + "etc" )