Skip to content

Commit

Permalink
Merge pull request #1222 from enrico-stauss/fix/contextual-precision-…
Browse files Browse the repository at this point in the history
…verdicts-endless-generation

Add doc count to the ContextualPrecision template (Fixes #1205)
  • Loading branch information
penguine-ip authored Dec 12, 2024
2 parents d71fcde + 7f0a81d commit dc95779
Showing 1 changed file with 5 additions and 2 deletions.
7 changes: 5 additions & 2 deletions deepeval/metrics/contextual_precision/template.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
class ContextualPrecisionTemplate:
@staticmethod
def generate_verdicts(input, expected_output, retrieval_context):
def generate_verdicts(input, expected_output, retrieval_context: str | list):
document_count_str = ""
if isinstance(retrieval_context, list):
document_count_str = f" ({len(retrieval_context)} document{'s' if len(retrieval_context) > 1 else ''})"
return f"""Given the input, expected output, and retrieval context, please generate a list of JSON objects to determine whether each node in the retrieval context was remotely useful in arriving at the expected output.
**
Expand Down Expand Up @@ -35,7 +38,7 @@ def generate_verdicts(input, expected_output, retrieval_context):
Expected output:
{expected_output}
Retrieval Context:
Retrieval Context{document_count_str}:
{retrieval_context}
JSON:
Expand Down

0 comments on commit dc95779

Please sign in to comment.