Giters

janhq / jan

Jan is an open source alternative to ChatGPT that runs 100% offline on your computer. Multiple engine support (llama.cpp, TensorRT-LLM)

Home Page:https://jan.ai/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Is there a way to turn off auto-titles?

lactoseintolerant opened this issue · comments

commented

I search in the docs but nothing showed up. Is there a way to turn of the auto-titling feature? I don't want to "double call" my inputs tokens all the time. Or is there a way to set the title specifically to a separate, guaranteed cheap model?

commented

Thank you for your question! I understand that you would like to turn off the auto-titling feature or have a way to manually set the title for your threads.

Currently, our system only generates the title of a thread after a few initial messages have been sent. This is done to ensure that the title accurately reflects the content of the thread. However, I can assure you that the token used for generating the title is not "double called" and is only used once during the title generation process (if the title was generated, it will not regenerate the title again).

Additionally, we do offer the option to manually provide a title for your thread. This way, you can ensure that the title is exactly as you want it and avoid any potential issues with auto-generation.

I hope this helps clarify things for you. Let me know if you have any other questions!

commented

hi @cahyosubroto, we might need a document on this, please take a look and provide your idea on this, many thanks 🙏

commented

Hi, I just want to clarify what I mean by double calling. I tested now, and the API does indeed get called twice with the full input for my particular use case. I frequently start a new thread with a large input to Opus through Openrouter.

I tested it just now as follows with Sonnet instead: give it a roughly 1400 word text excerpt and an instruction. In jan.ai, this generates two Openrouter Sonnet calls with token sizes of 1905->160 and 1830->14. Presumably this is the response and a second call to write the short title.

When I use another tool like big-agi, I get a single Sonnet call for 1880 -> 214 (the outputs are different so they don't match exactly), and then a second GPT3.5 titling call for 94->2 (big-agi lets me set the titling model).

Now, if I manually set the title in jan.ai for a brand new thread and repeat, I get a single Sonnet call for 1905->116.

Therefore, for my particular use case of gigantic first inputs directly into Opus, auto-titling by jan.ai effectively doubles my expensive API call.

However, a solution exist as you mentioned and I verified. I can manually set the title first. So it's just slightly annoying but not the end of the world. I would still prefer to be able to disable or set the titling model, but understand there's no plan do do so.

Thanks, just clarifying and posting here for reference as the titling feature has no documentation.