Giters

sgl-project / sglang

SGLang is a structured generation language designed for large language models (LLMs). It makes your interaction with models faster and more controllable.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Please add Phi3 support

Curiosity007 opened this issue · comments

commented

Getting this error -

router init state: Traceback (most recent call last):
  File "/home/ubuntu/sglang/python/sglang/srt/managers/router/manager.py", line 73, in start_router_process
    model_client = ModelRpcClient(server_args, port_args)
  File "/home/ubuntu/sglang/python/sglang/srt/managers/router/model_rpc.py", line 657, in __init__
    self.model_server = ModelRpcService().exposed_ModelRpcServer(
  File "/home/ubuntu/sglang/python/sglang/srt/managers/router/model_rpc.py", line 70, in __init__
    self.model_runner = ModelRunner(
  File "/home/ubuntu/sglang/python/sglang/srt/managers/router/model_runner.py", line 294, in __init__
    self.load_model()
  File "/home/ubuntu/sglang/python/sglang/srt/managers/router/model_runner.py", line 303, in load_model
    model_class = get_model_cls_by_arch_name(architectures)
  File "/home/ubuntu/sglang/python/sglang/srt/managers/router/model_runner.py", line 58, in get_model_cls_by_arch_name
    raise ValueError(
ValueError: Unsupported architectures: Phi3ForCausalLM. Supported list: ['CohereForCausalLM', 'DbrxForCausalLM', 'GemmaForCausalLM', 'LlamaForCausalLM', 'LlavaLlamaForCausalLM', 'MistralForCausalLM', 'MixtralForCausalLM', 'QWenLMHeadModel', 'Qwen2ForCausalLM', 'StableLmForCausalLM', 'YiVLForCausalLM']
commented

hmm, I just wanted to try this as well... sadly not supported yet.

commented

@hnyls2002
what would be your thoughts? I'm using SGLang for bulk information extraction, so Phi3-mini would be very interesting to try since it's only 3.8B