Kerlinn / farfalle

open-source answer engine - run local or cloud models

Farfalle

Open-source AI-powered search engine. Run your own local LLM or use the cloud.

Demo answering questions with llama3 on my M1 Macbook Pro:

local-demo.mp4

💻 Live Demo

farfalle.dev (Cloud models only)

📖 Overview

• 🛠️ Tech Stack
• 🏃🏿‍♂️ Getting Started
• 🚀 Deploy

🛣️ Roadmap

• Add support for local LLMs through Ollama
• Docker deployment setup

🛠️ Tech Stack

• Frontend: Next.js
• Backend: FastAPI
• Search API: Tavily
• Logging: Logfire
• Rate Limiting: Redis
• Components: shadcn/ui

🏃🏿‍♂️ Getting Started

Prerequisites

• Docker
• Ollama
  • Download any of the supported models: llama3, mistral, gemma
  • Start ollama server ollama serve

Get API Keys

• Tavily
• OpenAI (Optional)
• Groq (Optional)

1. Clone the Repo

git clone [e-mail]:rashadphz/farfalle.git
cd farfalle

2. Add Environment Variables

touch .env

Add the following variables to the .env file:

Required

TAVILY_API_KEY=...

Optional

# Cloud Models
OPENAI_API_KEY=...
GROQ_API_KEY=...

# Rate Limit
RATE_LIMIT_ENABLED=
REDIS_URL=...

# Logging
LOGFIRE_TOKEN=...

Optional Variables (Pre-configured Defaults)

# API URL
NEXT_PUBLIC_API_URL=http://localhost:8000

# Local Models
NEXT_PUBLIC_LOCAL_MODE_ENABLED=true
ENABLE_LOCAL_MODELS=True

3. Run Containers

This requires Docker Compose version 2.22.0 or later.

docker-compose -f docker-compose.dev.yaml up -d

Visit http://localhost:3000 to view the app.

🚀 Deploy

Backend

After the backend is deployed, copy the web service URL to your clipboard. It should look something like: https://some-service-name.onrender.com.

Frontend

Use the copied backend URL in the NEXT_PUBLIC_API_URL environment variable when deploying with Vercel.

And you're done! 🥳