ragas

RAG 应用自动化评测框架。这是一个专门用于评测和优化 RAG 应用的开源框架，提供客观、可量化的评测结果，并支持自动生成测试数据集。

python import asyncio from ragas.metrics.collections import AspectCritic from ragas.llms import llm_factory# Setup your LLM llm = llm_factory("gpt-4o")# Create a metric metric = AspectCritic(name="summary_accuracy",definition="Verify if the summary is accurate and captures key information.",llm=llm)# Evaluate test_data = {"user_input": "summarise given text\nThe company reported an 8% rise in Q3 2024, driven by strong performance in the Asian market. Sales in this region have significantly contributed to the overall growth. Analysts attribute this success to strategic marketing and product localization. The positive trend in the Asian market is expected to continue into the next quarter.","response": "The company experienced an 8% increase in Q3 2024, largely due to effective marketing strategies and product adaptation, with expectations of continued growth in the coming quarter.",}score = await metric.ascore(user_input=test_data["user_input"],response=test_data["response"])print(f"Score: {score.value}")print(f"Reason: {score.reason}")