API accepting invalid model without producing error
andypbaker opened this issue · comments
in the call below, I've specified a model that does not exist- "a_model-that-does-not-exist"
The call still succeeds, but uses the model "Mistral Instruct" instead.
The chat client works fine with e.g. I have a model 'wizardlm-13b-v1.2.Q4_0 (3).gguf' in my gpt4all/bin directory, but when I try "model":"wizardlm-13b-v1.2.Q4_0 (3).gguf", in the body of the call, the response still shows "model : Mistral Instruct"
Any thoughts on whether I'm doing anything wrong here or is there a bug?
if it helps, here's a list of the installed models -
PS> $response = Invoke-RestMethod -Uri http://127.0.0.1:4891/v1/models -Method GET
Andyb, DEV-09
05/08/2024 20:07:53
C:\Users\andyb\documents
PS> $response.data
created : 0
id : Mistral Instruct
object : model
owned_by : humanity
parent :
permissions : {@{allow_create_engine=False; allow_fine_tuning=False; allow_logprobs=False; allow_sampling=False; allow_search_indices=False; allow_view=True; created=0;
group=; id=foobarbaz; is_blocking=False; object=model_permission; organization=*}}
root : Mistral Instruct
created : 0
id : Mistral OpenOrca
object : model
owned_by : humanity
parent :
permissions : {@{allow_create_engine=False; allow_fine_tuning=False; allow_logprobs=False; allow_sampling=False; allow_search_indices=False; allow_view=True; created=0;
group=; id=foobarbaz; is_blocking=False; object=model_permission; organization=*}}
root : Mistral OpenOrca
created : 0
id : GPT4All Falcon
object : model
owned_by : humanity
parent :
permissions : {@{allow_create_engine=False; allow_fine_tuning=False; allow_logprobs=False; allow_sampling=False; allow_search_indices=False; allow_view=True; created=0;
group=; id=foobarbaz; is_blocking=False; object=model_permission; organization=*}}
root : GPT4All Falcon
created : 0
id : Orca 2 (Medium)
object : model
owned_by : humanity
parent :
permissions : {@{allow_create_engine=False; allow_fine_tuning=False; allow_logprobs=False; allow_sampling=False; allow_search_indices=False; allow_view=True; created=0;
group=; id=foobarbaz; is_blocking=False; object=model_permission; organization=*}}
root : Orca 2 (Medium)
created : 0
id : Orca 2 (Full)
object : model
owned_by : humanity
parent :
permissions : {@{allow_create_engine=False; allow_fine_tuning=False; allow_logprobs=False; allow_sampling=False; allow_search_indices=False; allow_view=True; created=0;
group=; id=foobarbaz; is_blocking=False; object=model_permission; organization=*}}
root : Orca 2 (Full)
created : 0
id : Hermes
object : model
owned_by : humanity
parent :
permissions : {@{allow_create_engine=False; allow_fine_tuning=False; allow_logprobs=False; allow_sampling=False; allow_search_indices=False; allow_view=True; created=0;
group=; id=foobarbaz; is_blocking=False; object=model_permission; organization=*}}
root : Hermes
created : 0
id : Snoozy
object : model
owned_by : humanity
parent :
permissions : {@{allow_create_engine=False; allow_fine_tuning=False; allow_logprobs=False; allow_sampling=False; allow_search_indices=False; allow_view=True; created=0;
group=; id=foobarbaz; is_blocking=False; object=model_permission; organization=*}}
root : Snoozy
created : 0
id : Mini Orca (Small)
object : model
owned_by : humanity
parent :
permissions : {@{allow_create_engine=False; allow_fine_tuning=False; allow_logprobs=False; allow_sampling=False; allow_search_indices=False; allow_view=True; created=0;
group=; id=foobarbaz; is_blocking=False; object=model_permission; organization=*}}
root : Mini Orca (Small)
created : 0
id : all-MiniLM-L6-v2-f16.gguf
object : model
owned_by : humanity
parent :
permissions : {@{allow_create_engine=False; allow_fine_tuning=False; allow_logprobs=False; allow_sampling=False; allow_search_indices=False; allow_view=True; created=0;
group=; id=foobarbaz; is_blocking=False; object=model_permission; organization=*}}
root : all-MiniLM-L6-v2-f16.gguf
created : 0
id : mpt-7b-chat-newbpe-q4_0.gguf
object : model
owned_by : humanity
parent :
permissions : {@{allow_create_engine=False; allow_fine_tuning=False; allow_logprobs=False; allow_sampling=False; allow_search_indices=False; allow_view=True; created=0;
group=; id=foobarbaz; is_blocking=False; object=model_permission; organization=*}}
root : mpt-7b-chat-newbpe-q4_0.gguf
created : 0
id : replit-code-v1_5-3b-newbpe-q4_0.gguf
object : model
owned_by : humanity
parent :
permissions : {@{allow_create_engine=False; allow_fine_tuning=False; allow_logprobs=False; allow_sampling=False; allow_search_indices=False; allow_view=True; created=0;
group=; id=foobarbaz; is_blocking=False; object=model_permission; organization=*}}
root : replit-code-v1_5-3b-newbpe-q4_0.gguf
created : 0
id : rift-coder-v0-7b-q4_0.gguf
object : model
owned_by : humanity
parent :
permissions : {@{allow_create_engine=False; allow_fine_tuning=False; allow_logprobs=False; allow_sampling=False; allow_search_indices=False; allow_view=True; created=0;
group=; id=foobarbaz; is_blocking=False; object=model_permission; organization=*}}
root : rift-coder-v0-7b-q4_0.gguf
created : 0
id : starcoder-newbpe-q4_0.gguf
object : model
owned_by : humanity
parent :
permissions : {@{allow_create_engine=False; allow_fine_tuning=False; allow_logprobs=False; allow_sampling=False; allow_search_indices=False; allow_view=True; created=0;
group=; id=foobarbaz; is_blocking=False; object=model_permission; organization=*}}
root : starcoder-newbpe-q4_0.gguf
created : 0
id : wizardlm-13b-v1.2.Q4_0 (3).gguf
object : model
owned_by : humanity
parent :
permissions : {@{allow_create_engine=False; allow_fine_tuning=False; allow_logprobs=False; allow_sampling=False; allow_search_indices=False; allow_view=True; created=0;
group=; id=foobarbaz; is_blocking=False; object=model_permission; organization=*}}
root : wizardlm-13b-v1.2.Q4_0 (3).gguf
I now have this working selecting the correct model, but am still not clear why there was no error with a non-existent model.
Here's the working call -
PS> $body = '{
"stream":false,
"temperatur":0.6,
"max_tokens":1000,
"model":"wizardlm-13b-v1.2.Q4_0 (3).gguf",
"prompt":"
why did bugs bunny frequenty say whats cooking doc
"
}'$response = Invoke-RestMethod -Uri http://127.0.0.1:4891/v1/completions -Method POST -Body $body; $response
choices : {@{finish_reason=stop; index=0; logprobs=; references=System.Object[]; text=Bugs Bunny, a beloved cartoon character from Looney Tunes, often says "What's up,
Doc?" not "What's cooking, Doc?" This phrase originated in the 1949 film, "Hare Trimmed." In this movie, Bugs Bunny says to his companion Daffy Duck during a
chase scene, "What's all the hurry, doc? You're just trying to run down a fellow citizen!" Chester Sylaspeak, who was one of Warner Brothers' sound engineers
and writers at the time, thought that Bugs Bunny saying this line would be more comical instead of simply addressing his companion as "hun." Over time, it
became an iconic way for Bugs to greet Daffy or interact with other characters.
In terms of why he used this specific phrase in certain situations, there could be a few reasons:
1. Humor and character development: The writers used this line as part of Bugs' comedic personality, often using witty retorts to respond to Daffy or other
characters. By saying "What's up, Doc?" instead of simply shouting his greeting, it added an element of humor and kept the tone lighthearted within these scenes.
2. Contrast with Daffy: Bugs regularly interacted with both Daffy Dark (the grumpy duck) and Dusty Dog (his loyal dog assistant). By directly addressing "Doc"
to Daffy, it further emphasized their contrast in personalities - highlighting how Bugs was the easy-going or carefree character while Daffy portrayed
impatience or sarcasm.
3. Memorable phrasing: "What's up, Doc?" has a unique and rhythmic style to it that is easily recognizable even for those not familiar with Looney Tunes. It may
have contributed to Bugs' enduring popularity and served as a sort of trademark catchphrase.
4. Versatility: The phrase could be used in various contexts where humor or lightheartedness was needed, from friendly greetings between the two companions to
frustrated reactions when Daffy attempted some trickery. It also allowed for Bugs' clever and quick-witted responses to whatever situation he might find himself
in with his friends and adversaries.
In conclusion, "What's up, Doc?" became a defining aspect of Bugs Bunny's character, adding depth to the narrative storylines and humor throughout Looney Tunes
cartoons, contributing to its enduring appeal generation after generation.}}
created : 1715197352
id : foobarbaz
model : wizardlm-13b-v1.2.Q4_0 (3).gguf
object : text_completion
usage : @{completion_tokens=574; prompt_tokens=32; total_tokens=606}