ludwig-ai / ludwig

Low-code framework for building custom LLMs, neural networks, and other AI models

Home Page:http://ludwig.ai

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Integrating new frameworks

yogeshhk opened this issue · comments

Is your feature request related to a problem? Please describe.
This feature request is related to brand new ask, such as integration of external libraries/frameworks.

Describe the use case
Ideas: Not sure if the following ideas have been already considered or not, but still let me enumerate them

  • integrating Crew AI (https://github.com/joaomdmoura/crewAI): This library seems to be written on top of Langchain, and provides ways to specify Agents and Tasks, which work together as a crew to complete the given tasks. This arrangements seems to be an apt candidate to become 'declarative'.
  • a larger task is to make Langchain itself 'declarative'. With introduction of LangChain Expression Language (LCEL) https://python.langchain.com/docs/expression_language/ it is becoming declarative. Just by providing, LLM info, tools info, Agents, prompts etc, in yaml, would it be possible to run Langchain in Ludwig? (sorry, as I am unaware of the internal architecture, if the callbacks need to be TensorFlow/PyTorch compatible or we can call any function)

Describe the solution you'd like
What would it take to integrate the above? Is Ludwig thin enough to integrate any new functionality, by specifying parameters in yaml and then having callbacks to get the task done?