twai 🐦🤖

twai is a tool to scrape tweets and add a score to them using AI.

📢 Connect with us! Join our Telegram group for support and collaboration: t.me/igohub

🚀 Features

Scrape tweets from Twitter using a headless chrome browser
Add a score from 1 to 10 to each tweet using AI
Add a score using Elo rating system, comparing tweets to each other

📦 Installation

You can use the Golang binary to install twai:

go install github.com/igolaizola/twai/cmd/twai@latest

Or you can download the binary from the releases

📋 Requirements

Twitter cookie

You need to capture the cookie from Twitter website.

Go to https://x.com/home
Login if you are not already logged in
Open the developer tools (F12)
Go to the "Network" tab
Refresh the page
Click on the first request to https://x.com/home
Go to the "Request Headers"
Copy the "cookie" header and save it in a cookie.txt file

AI model

You need to have an AI model to score the tweets.

You can use a local model via Ollama.

Download and install Ollama tool
Run ollama run llama3 to download the model
Set the host to http://localhost:11434/v1 and the model to llama3

Or you can use an online model like OpenAI.

Get an API key from OpenAI
Set the model to gpt-3.5-turbo and the token to your API key

🕹️ Usage

You can launch the twai command without any arguments or simply double-click on the binary to start the web UI. The web UI will be available at the address printed in the console.

Alternatively, you can use the command-line interface to interact with the tool.

Scrape tweets

Scrape tweets from a user or from the home page:

twai scrape --config scrape.yaml

#scrape.yaml
debug: false #(bool): Debug mode
page: home #(string): Page to fetch (home or username)
n: 50 #(int): Number of tweets to fetch
followers: false #(bool): Fetch followers stats, it will take longer
output: scape.csv #(string): Output file
show-browser: false #(bool): Show browser
cookie-file: cookie.txt #(string): Cookie file

Add simple score

Add a score to the tweets:

twai score --config score.yaml

#score.yaml
debug: false #(bool): Debug mode
concurrency: 1 #(int): Number of concurrent requests to the AI
input: scrape.csv #(string): Input file (generated by scrape command)
output: score.csv #(string): Output file (csv)
prompt: "Rate the following tweet from 1 to 10 based on relevance, clarity, engagement, and impact. Only answer with a number." #(string): Prompt
model: llama3 #(string): AI model (e.g., llama3, gpt-3.5-turbo)
host: "http://localhost:11434/v1" #(string): AI endpoint host (not needed for openai)
token: "" #(string): Authorization token (required for openai)

Add Elo score

Add a score to the tweets using the Elo rating system:

twai elo --config elo.yaml

#elo.yaml
debug: false #(bool): Debug mode
concurrency: 1 #(int): Number of concurrent requests to the AI
input: scrape.csv #(string): Input file (generated by scrape command)
output: elo.csv #(string): Output file (csv)
iterations: 10 #(int): Number of iterations
prompt: "Which tweet is best? 1 or 2? Answer only with the number 1 or 2." #(string): Prompt
model: llama3 #(string): AI model (e.g., llama3, gpt-3.5-turbo)
host: "http://localhost:11434/v1" #(string): AI endpoint host (not needed for openai)
token: "" #(string): Authorization token (required for openai)

Help

Launch twai with the --help flag to see all available commands and options:

twai --help

You can use the --help flag with any command to view available options:

twai scrape --help

⚠️ Disclaimer

The automation of Twitter accounts is a violation of their Terms of Service and will result in your account(s) being terminated.

Read about Twitter Terms of Service and Community Guidelines.

twai was written as a proof of concept and the code has been released for educational purposes only. The authors are released of any liabilities which your usage may entail.

💖 Support

If you have found my code helpful, please give the repository a star ⭐

Additionally, if you would like to support my late-night coding efforts and the coffee that keeps me going, I would greatly appreciate a donation.

You can invite me for a coffee at ko-fi (0% fees):

Or at buymeacoffee:

Donate to my PayPal:

paypal.me/igolaizola

Or sponsor me on GitHub:

github.com/sponsors/igolaizola

Thanks for your support!

igolaizola / twai