Giters
codelion
/
optillm
Optimizing inference proxy for LLMs
Geek Repo:
Geek Repo
Github PK Tool:
Github PK Tool
Stargazers:
1311
Watchers:
20
Issues:
37
Forks:
117
codelion/optillm Issues
Is there any possibility we align some interest?
Closed
18 days ago
Comments count
1
Using llama-server issue with 'no_key' API key
Closed
18 days ago
Comments count
1
Scripts to reproduce benchmark results
Closed
18 days ago
Comments count
1
Implement cot decoding with llama.cpp
Updated
19 days ago
Comments count
4
Request for Reference Citations for CoT Prompting Methods
Closed
21 days ago
Comments count
1
(MOA) Fails with "List Index Out of Range" Error on OpenAI-Compatible Ollama API Endpoint
Updated
21 days ago
Comments count
6
Implement routing
Closed
23 days ago
Comments count
1
I can see cot_decode method has implemented, but we can't use it with the proxy.
Closed
a month ago
Comments count
13
When I tried the optillm with my own openai API compatible hosted model I get this error
Closed
a month ago
Comments count
6
Add a lighting template for running optillm
Updated
a month ago
Comments count
1
Integration with Gemini 1.5 models
Closed
a month ago
Comments count
2
token counting
Closed
a month ago
Comments count
2
[Question]: Which paper is mcts.py based on?
Closed
a month ago
Comments count
1
Can't install z3-solver, is it possible to support lean4?
Closed
a month ago
Comments count
20
Error processing request: litellm.AuthenticationError: AuthenticationError
Closed
a month ago
Comments count
2
Add support for logging with --log=debug
Closed
a month ago
Comments count
1
Add support for sympy in solver approach
Closed
a month ago
Comments count
1
Possible error in calculate_confidence() logic for cot_decoding.py
Closed
a month ago
Comments count
1
Add support to pass slug as extra_body argument instead of prefix of model name
Closed
a month ago
Response text missing when using third-party AI frontend with local endpoint
Closed
a month ago
Comments count
4
Streaming, Context, Port & Proxy vs Library
Closed
a month ago
Comments count
7
Clarification: proxy or library for cot_decoding??
Closed
a month ago
Comments count
2
Change api-key to optillm-api-key
Closed
a month ago
use with llama.cpp
Closed
a month ago
Comments count
8
Flask import fails
Closed
2 months ago
Comments count
5
Create a gradio based GUI to compare different approaches
Updated
2 months ago
Support AzureOpenAI client
Closed
2 months ago
Comments count
1
Gsm8k bad test
Closed
2 months ago
Comments count
1
Minimal working MCTS example
Closed
2 months ago
Comments count
3
Too many tokens
Closed
2 months ago
Comments count
2
initial_query both in system message and user message
Closed
2 months ago
Comments count
2