sgl-project / sglang

SGLang is a structured generation language designed for large language models (LLMs). It makes your interaction with models faster and more controllable.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

ImportError: cannot import name 'function' from partially initialized module 'sglang'

lambda7xx opened this issue · comments

my code

from sglang import function, system, user, assistant, gen, set_default_backend, RuntimeEndpoint

@function
def multi_turn_question(s, question_1, question_2):
    s += system("You are a helpful assistant.")
    s += user(question_1)
    s += assistant(gen("answer_1", max_tokens=256))
    s += user(question_2)
    s += assistant(gen("answer_2", max_tokens=256))

set_default_backend(RuntimeEndpoint("http://localhost:30000"))

state = multi_turn_question.run(
    question_1="What is the capital of the United States?",
    question_2="List two local attractions.",
)

for m in state.messages():
    print(m["role"], ":", m["content"])

print(state["answer_1"])    

then error is

ImportError: cannot import name 'function' from partially initialized module 'sglang' (most likely due to a circular import) 

I also try the follow code


import torch , time 
import sglang as sgl
sys_prompt = ""


@sgl.function
def multi_turn_question(s, question_1, question_2,sys_prompt):
    s += sgl.system(sys_prompt)
    s += sgl.user(question_1)
    s += sgl.assistant(sgl.gen("answer_1", max_tokens=100))
    # s += user(question_2)
    # s += assistant(gen("answer_2", max_tokens=256))

runtime = sgl.Runtime(model_path="meta-llama/Llama-2-7b-chat-hf")

sgl.set_default_backend(runtime)

for i in range(2):
    state = multi_turn_question.run(
        question_1="What is the capital of the United States?",
        question_2="List two local attractions.",
        sys_prompt=sys_prompt
    )

the error log is

    @sgl.function
     ^^^^^^^^^^^^
AttributeError: partially initialized module 'sglang' has no attribute 'function' (most likely due to a circular import)