faithfullness checks for ragas assumes a match
jjimenez opened this issue · comments
this line of code assumes that there is a match.
def count_verified_statements(verifications)
match = verifications.match(/Final verdict for each statement in order:\s*(.*)/)
verdicts = match.captures.first
verdicts
.split(".")
.count { |value| value.strip.to_boolean }
end
consider changing to
def count_verified_statements(verifications)
match = verifications.match(/Final verdict for each statement in order:\s*(.*)/)
verdicts = match&.captures&.first
verdicts
&.split(".")
&.count { |value| value.strip.to_boolean } || 0
end
@jjimenez Could you please help me understand when this happens? Does the LLM hallucinate and not return the correct output? Could you please share what that output is?