truera / trulens

Evaluation and Tracking for LLM Experiments

https://www.trulens.org/

truera/trulens Issues

[BUG] Using AzureOpenAI provider does not show token usage/price
Updated 3 days ago1
Deploy Trulens with FastAPI
Updated 3 days ago3
[BUG] Inconsistent results between records variable values and dashboard/leaderboard values
Updated 3 days ago5
[BUG] v0.20.3 of trulens-eval AzureOpenAI object has no attribute 'generate_score'
Closed 5 days ago2
Outdated documentation for Groundedness
Closed 6 days ago2
[BUG] Azure openai shows 404 error
Closed 8 days ago1
getting error that openai is not installed.
Closed 9 days ago15
[FEAT] ARES integration possible?
Closed 9 days ago2
Getting RecursionError: maximum recursion depth exceeded while calling a Python object while creating tru_recorder.
Updated 10 days ago7
[BUG] Problem with custom retriever. It's triggering "Found more than one `BaseRetriever` in app".
Updated 10 days ago4
[BUG] Bedrock provider always gives: "Groundedness per statement in source: 0%"
Updated 10 days ago4
[BUG] TruChain - no values returned for Groundedness feedback function for Bedrock
Updated 12 days ago3
Getting issues in using Trulens with Langchain Agents like ReAct/Structured Chat
Updated 14 days ago6
Feedback functions returning FAILED
Updated 19 days ago8
Groundedness evaluating sentences that don't exist in the LLM response
Closed 19 days ago5
[BUG] Could not locate app.retriever.get_relevant_documents.rets[:].page_content.collect() in app/record.
Updated 20 days ago4
[BUG] Groundedness and Context Relevance Error due to Debug Timeout
Closed 20 days ago4
Run 'tru.run_dashboard(port=8502)' Leaderboard shows error.
Updated 20 days ago9
Error: ValueError: Some selectors do not exist in the app or record.
Updated 24 days ago10
[BUG] Azure OpenAI error with evaluation using TruLlama/Llamaindex
Updated 25 days ago13
Error when using Streaming mode in trulens for openai chat completion.
Closed a month ago4
[BUG] TruLens throws ValidationError: Input should be an instance of llama_index.core.base_query_engine.BaseQueryEngine
Closed a month ago5
[BUG]get concurrent.futures._base.TimeoutError
Closed a month ago5
[FEAT] Can I use a full local LLM instead of openai?
Closed a month ago1
TruLens with Outside Logs example not working
Closed a month ago9
RuntimeError: Could not locate app.query.rets.source_nodes[:].note.text in app/record.
Closed a month ago10
[BUG] Dashboard ignores `database_url` when opening Apps or Evaluations pages without opening Leaderboards first
Closed a month ago4
[BUG] No return of feedback when using LiteLLM with Mistral
Closed a month ago10
[BUG] Trulens does not evaluate the cost per inference in the case of Bedrock.
Updated a month ago3
[BUG] Tru() function fails either due to missing alembic.ini file or KeyError: 'formatters'
Closed a month ago6
[FEAT] Feasibility to extend the record schema to accommodate custom fields
Updated a month ago
[FEAT] Change database configuration
Closed a month ago6
[BUG] TruLens Quickstart - OperationalError: (sqlite3.OperationalError) no such table: trulens_apps
Closed a month ago6
[BUG] import error after clean install
Closed 2 months ago5
[BUG] TruLlama quickstart example fails with Pydantic validation error for queue.Queue object.
Closed 2 months ago3
[BUG] Trulens is working properly with the Query Engine but not with the Chat Engine.
Closed 2 months ago6
[BUG] UnicodeDecodeError: 'gbk' codec can't decode
Closed 2 months ago3
Error Executing llama_index_quickstart.ipynb
Closed 2 months ago4
[BUG] Tru was already initialized. Cannot change database_url
Closed 2 months ago3
[FEAT] Adding support of base feedback functions to different languages
Closed 2 months ago2
[FEAT] Support for Python 3.12
Closed 2 months ago4
[BUG] A non-annotated attribute was detected: `TABLE_META = 'meta'`
Closed 2 months ago6
[BUG] Use Anthropic Claude 3 Opus as Feedback Provider
Closed 2 months ago14
[BUG] ValidationError: 1 validation error for DataSource component_type subclass of BaseComponent expected (type=type_error.subclass; expected_class=BaseComponent)
Closed 2 months ago6
Add Gemini Support in the provider list
Closed 3 months ago2
[BUG] pydantic_core._pydantic_core.ValidationError: 1 validation error for TruCustomApp
Closed 3 months ago2
[BUG] Some docs broken / 404?
Closed 3 months ago3
how can I use LLM from huggingface as the provider of the Trulens feedback?
Closed 3 months ago4
When Custom retriever and query engine with llamaindex,get ERROR
Closed 3 months ago2
[BUG] How do I evaluate my Llama-Index app for hallucination?
Closed 3 months ago1