Use automatically generated suggestions based on evaluation results to improve your prompts.
Latitude’s Prompt Suggestions feature (known as the Refiner) analyzes your evaluation results to automatically recommend improvements for your prompts. It acts like an AI assistant helping you iterate faster and achieve better performance.
Pattern Analysis: Latitude analyzes these results, looking for correlations between prompt inputs, outputs, and evaluation scores. It identifies patterns where certain inputs lead to lower scores or specific failure modes.
Suggestion Generation: Based on these patterns, an AI model generates concrete suggestions for modifying your prompt. These suggestions might involve:
Prioritization: Suggestions are often prioritized based on the potential impact on evaluation scores.
Suggestions become more insightful as more evaluation data is collected. Aim
for at least 20-30 evaluated logs for meaningful analysis, though more data is
generally better.