Feature Request: Server-Side Session Management for State Maintenance

[Jack Zhou]

Description:

I am currently integrating with your API to develop a service that requires maintaining conversation histories to enhance interaction quality without requiring the client-side with state management. This approach would enable clients to interact with the endpoint in a stateless manner by simply sending a message and a session ID.

Suggested Feature:

I propose the addition of a session ID parameter in API calls, which would allow the server to manage and maintain session states. This would not only streamline client interactions but also enhance the scalability and flexibility of the service by delegating session management to the server side.

Implementation Suggestion:

Include a sessionId parameter in the API endpoint configurations that is sent with each request. The server can then use this ID to retrieve and store session-specific data, such as conversation histories and user preferences.

If implementing this feature is not currently feasible, I would greatly appreciate any guidance on how I might implement this functionality on my own.

I hope this feature will be considered for future updates as it could significantly enhance the API's functionality and appeal for developers requiring robust session management.

[Nathan Sarrazin]

Hi! Thanks for opening a feature request.

We currently use a cookie (by default named hf-chat) which contains a unique session id. So when you send requests with the cookie containing your sessionId you should be authenticated.

If I understood correctly, you would prefer to auth in the API without cookies. Maybe we could have a simple Authorization header that points to a token/session id?

Couple issues I see right away with using the session id for this:

• A user can have multiple sessions, and we don't have a way of managing sessions in the UI, so if the session is compromised you can't easily revoke it
• Sessions expire after 2 weeks without use
• We don't show the session Id to the user

I think the correct way to do this, would be to have a system that lets the user create API tokens from the UI that can be used to identify as the user in the API routes, when passed with a request header. Would that be a solution to your problem ?

[Jack Zhou]

Sorry for the confusion. After reading the code, I found it shall be convId, not sessionId. With convId, I'd take care of conversation history on server side (e.g., a custom endpoint that is openAI API compatible). This gives the developer a ton of flexibility. For example, I can then use OpenAI's thread offering: I can invoke a specific thread by convId and just append the newest message to the thread; In this case, I don't need the entire message history sent by the endpointOai.ts. This can also build a bridge between custom endpoint and the mongdb (if mongdb is used), which would allow endpoint to query it for info, write metadata to it, etc., making a second communication channel between client and endpoint.

I believe that with these capabilities, this project could greatly expand its potential applications and utility.