jxnl / instructor

structured outputs for llms

Home Page:https://python.useinstructor.com/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Error handling broken in `retry.py` under certain circumstances

lazyhope opened this issue · comments

  • This is actually a bug report.

Describe the bug
Under some network circumstances when

response = func(*args, **kwargs)

fails in the last attempt, the following lines will raise UnboundLocalError("cannot access local variable 'response' where it is not associated with a value")]:
except RetryError as e:
raise InstructorRetryException(
e,
last_completion=response,
n_attempts=attempt.retry_state.attempt_number,
messages=kwargs["messages"],
total_usage=total_usage,
) from e

To Reproduce
When user define a custom tenacity.Retrying like retry_if_not_exception_type(HTTPException) and api error happens:

import instructor
from openai import OpenAI
from pydantic import BaseModel
from tenacity import Retrying, retry_if_not_exception_type, stop_after_attempt

class UserInfo(BaseModel):
    name: str
    age: int

client = instructor.from_openai(OpenAI(api_key="incorrect_key"), mode=instructor.Mode.MD_JSON)
client.chat.completions.create(
    model="gpt-4o",
    max_retries=Retrying(retry=retry_if_not_exception_type(ValueError), stop=stop_after_attempt(2)),
    max_tokens=1024,
    messages=[
        {
            "role": "user",
            "content": "Jason is 25 years old",
        }
    ],
    response_model=UserInfo,
)

Expected behavior
The current implementation lacks enough error handling (e.g. for API connection errors), which should stop retrying immediately. And the InstructorRetryException inside each with attempt clause is redundant.