confident-ai / deepeval

The LLM Evaluation Framework

Home Page:https://docs.confident-ai.com/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Sub-dependency conflict with Opentelemetry

philipchung opened this issue · comments

I'm trying to use poetry to install both deepeval and opentelemetry-api & opentelemetry-sdk in my virtual environment. I would like to use both. Currently this fails for the following reason:

Because no versions of deepeval match >0.21.34,<0.22.0 and deepeval (0.21.34) depends on importlib-metadata (>=7.0.2,<7.1.0), deepeval (>=0.21.34,<0.22.0) requires importlib-metadata (>=7.0.2,<7.1.0). And because opentelemetry-api (1.24.0) depends on importlib-metadata (>=6.0,<=7.0), deepeval (>=0.21.34,<0.22.0) is incompatible with opentelemetry-api (1.24.0). And because opentelemetry-sdk (1.24.0) depends on opentelemetry-api (1.24.0) and no versions of opentelemetry-sdk match >1.24.0,<2.0.0, deepeval (>=0.21.34,<0.22.0) is incompatible with opentelemetry-sdk (>=1.24.0,<2.0.0). So, because my-project depends on both opentelemetry-sdk (^1.24.0) and deepeval (^0.21.34), version solving failed.

Is it possible to relax deepeval's dependencies on importlib-metadata to >=6.0 <7.1.0 so that dependency resolving is possible?

@philipchung Hey let me have a look. WHat ar eyou adding opentelemetry for?

@philipchung I see. We're also adding tracing to Confident AI for llamaindex specifically, out next week. In the meantime feel free to make a PR on relaxing the dependencies in setup.py and poetry 😊

merged gonna cut a release in the next 12 hours to include the change, thansk!

Thank you! I have sensitive data that prohibits me from uploads to a web service, which is why I am using local tracing tools like Arize Phoenix, etc.

@philipchung I see, well in that case you'll like our support for any offline LLMs :) Come join our discord btw: https://discord.com/invite/a3K9c8GRGt for any other questions. Also new release is out, thanks!

I still have a conflict in deepeval release 0.21.35 because setup.py was changed to "importlib-metadata~=6.0.2" instead of "importlib-metadata>=6.0.2". This limits the compatible importlib-metadata versions to >=6.0.2,<6.1.0. Can we change it to just >=6.0.2?

Some of my other dependencies (e.g. LiteLLM) require higher version of importlib-metadata (e.g. >=6.8.0)

@philipchung Forgot why i corrected it.. but the new release should be good! v0.21.36. Please let me know if it works, I'd like to catch after dependency bugs before the week starts!

@penguine-ip Thank you! I am now able to install the deepeval package in my environment without any dependency conflicts.