s2t2 / text-embeddings-2024

research in progress

text-embeddings-2024

Text Embeddings for User Classification in Social Networks (2024).

Continuation of previous research.

Setup

Virtual Environment

Create and/or activate virtual environment:

conda create -n embeddings-2024 python=3.10
conda activate embeddings-2024

Install package dependencies:

pip install -r requirements.txt

Environment Variables

Create ".env" file with contents like the following:

# OPENAI:
OPENAI_API_KEY="sk-_________"

# GOOGLE CLOUD:
GOOGLE_APPLICATION_CREDENTIALS="/path/to/text-embeddings-2024/google-credentials.json"

Usage

BigQuery Service

python -m app.bq_service

OpenAI Service

python -m app.openai_service

MODEL_NAME="text-embedding-3-small" N_DIMENSIONS=1536 python -m app.openai_service

MODEL_NAME="text-embedding-3-large" N_DIMENSIONS=3072 python -m app.openai_service