Extract model costs into log and CSVs

[Simon Bauer]

• Extract model costs (per M request token + per M response token + per request + per response) and write into csv reports
• Check other API providers too (@bauersimon knows about that e.g. Mistral, Fireworks, Groq, ...)

[Rui Azevedo]

Groq Cloud

• source: https://console.groq.com/docs/models
• Only supports the following models (pricing: per 1M tokens, input/output):
  • LLaMA3 8b: $0.05/$0.08
  • LLaMA3 70b: $0.59/$0.79
  • Mixtral 8x7b: $0.24/$0.24
  • Gemma 7b: $0.07/$0.07
  • Whisper: $0.03/hour transcribed
  • Note: does not have a model which is in our top 10

Fireworks

• source: https://fireworks.ai/pricing
• pricing $/1M tokens (both input and output tokens)
• 0B - 16B: $0.20
• 16.1B - 80B: $0.90
• MoE 0B - 56B (e.g. Mixtral 8x7B): $0.50
• MoE 56.1B - 176B (e.g. DBRX, Mixtral 8x22B): $1.20
• Yi Large: $3.00
• Note: just has one model which is in our top 10: llama-3-70b-instruct

Mistral

• Only supports the following models (pricing: per 1M tokens, input/output):
  • open-mistral-7b: $0.25/$0.25
  • open-mixtral-8x7b: $0.7/$0.7
  • open-mixtral-8x22b: $2/$6
  • mistral-small-2402: $1/$3
  • codestral-2405: $1/$3
  • mistral-medium-2312: $2.7/$8.1
  • mistral-large-2402: $4/$12
  • mistral-embed: $0.1/$0.1
  • Mistral 7B: $0.75/$0.75
  • Mistral Small: $2.5/$7.5
  • Note: does not have a model which is in our top 10

Notes

• Looking at the Groq and Mistral JSON schema, there is no pricing information for models
  • All the API's require an API key even to list the models
• Fireworks also is missing pricing information for models