radareorg / r2ai

local language model for radare2

Home Page:https://www.radare.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Plugin System / Infrastructure to decouple all the model/api uses

trufae opened this issue · comments

Right now the logic in interpreter.py is a bit overwhelming because we are handling so many models and growing. The plan would be to create a generic class that abstracts the exposed methods to split each model into separate files, so the user can add/remove more at its will.

The common attributes I see are the following:

  • local/remote (or api: llama, http, ...) right now all local are just "llama"
  • name (taken from huggingface)
  • prompt templater (maybe we should do that dynamically using the special tokens from the model instead of cooking them by hand)
  • category (coding, chat, uncensored, functionary...)
  • description (short text describing the usecases of it)
  • date (release date of the model)

the prompt templates right now are reused between models, so maybe we should decouple that from the plugin and just make the plugin specify which template use.

RFC