Giters

lm-sys / FastChat

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Please support top_logprobs via API

xufengduan opened this issue · comments

commented

I've noticed that the current implementation supports logprobs (thanks a lot!) but it appears to be different from how OpenAI handles them. I was wondering if it would be possible to enhance the functionality in the following ways:

  1. Support for top_logprobs: This would allow users to retrieve the log probabilities for the top tokens directly, which can be immensely useful for applications requiring more nuanced probability assessments.
  2. API Endpoint for Specific Token Probabilities: Providing an API method to query the probability of a specific token would be a great addition. This feature would enable more targeted data analysis and could enhance the usability of the API for various use cases.

Implementing these features could greatly increase the API's flexibility and utility for developers and researchers. I would appreciate any consideration you could give to these suggestions, and I am eager to assist with testing or feedback if needed.

Thank you for your time and all the work you do on this project.