jxnl / instructor

structured outputs for llms

Home Page:https://python.useinstructor.com/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

AnthropicBedrock Tools is already out of beta

fer-git opened this issue · comments

  • This is actually a bug report.
  • I am not getting good LLM Results
  • I have tried asking for help in the community on discord or discussions and have not received a response.
  • I have tried searching the documentation and have not found an answer.

What Model are you using?

  • gpt-3.5-turbo
  • gpt-4-turbo
  • gpt-4
  • Other (please specify)

Describe the bug
I would like to use instructor with AnthropicBedrock client, but I get an error. I believe it is because the latest version of Anthropic tools is no longer in beta.

if mode == instructor.Mode.ANTHROPIC_TOOLS:
--->  create = client.beta.tools.messages.create  # type: ignore - unknown in stubs
else:
    create = client.messages.create

AttributeError: 'AnthropicBedrock' object has no attribute 'beta'

To Reproduce

anthropic==0.28
instructor==1.3.2
from anthropic import AnthropicBedrock
import instructor


client = AnthropicBedrock(aws_region="us-west-2")
ins_client = instructor.from_anthropic(client, mode=instructor.Mode.ANTHROPIC_TOOLS)

Expected behavior
It should not output error, instructor client should be successfully initialized

This also goes for the AnthropicVertex client. I quickly tested it by dropping the beta namespace. No errors. I'm still validating whether the response matches my expectations. Update: it works with AnthropicVertex. Going by the quote below, I'm going to assume Bedrock should work too.

Source:

https://docs.anthropic.com/en/docs/tool-use

Tool use is now generally available across the entire Claude 3 model family through the Anthropic Messages API, Amazon Bedrock, and Google Vertex AI