Giters

ShipBit / slickgpt

SlickGPT is a light-weight "use-your-own-API-key" web client for the OpenAI API written in Svelte. It offers GPT-4 integration, a userless share feature and other superpowers.

Home Page:https://slickgpt.vercel.app

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Seeing a lot of crashes mid thread

Arro opened this issue · comments

commented

More and more frequently, I send a prompt, it starts responding, then dies right as it's about to add the completed response to the output. So the last message in the list will be my question the LLM.

Once randomly a few days ago, I saw that missing response in a different thread, but only for a short moment. Then it disappeared.

I will update this issue next time it happens with any errors I see in the developer console. I have yet to do that.

commented

Also ran into this issue recently.

commented

Okay, so when you prompt something, the client calls an Vercel Edge function (middleware) and that will establish the connection to the OpenAI API and "proxy" the stream to the client. If that streams fails for some reason, SlickGPT deletes the latest (incomplete) completion from the history, so that you can re-submit your last prompt easily.
So basically, there are 3 points of failure:

  • OpenAI API is down or cuts the connection to Vercel (most likely, happens all the time, especially whenever they update stuff)
  • Vercel has problems/downtime and disconnects the client (rather unlikely but not impossible)
  • Your internet connection broke

I don't think we can do much on SlickGPT/client side except maybe output some better error messages.

commented

It may not delete the unfinished API response but stuck somewhere. From my usage experience, the disappeared response will show eventually, but may not in the identical session, maybe in another slickgpt chat session. Once the popped up message is so sudden that the entire session context jumped to the other session while I was typing. (hopes it is not some session-switching shortcuts)

commented

We are about to release a new major release that will change things:

  • We'll be offering fixed-price access to the OpenAI API using our Azure infrastructure. It's a lot faster than direct access (probably because Microsoft has special deals with OpenAI?). I haven't seen the problem there yet.
  • The Svelte endpoint layer will be completely removed. So even if you don't use that, there's one less point of failure because the client will be talking directly to OpenAI. Maybe that will help as well.

Stay tuned, it's coming soon.