Custom Evaluations

​

Overview

Latitude gives you complete freedom to create your own custom, LLM-as-judge evaluations from scratch. Custom evaluations can be categorized by their output types.

• Numeric evaluations
  • Use this type of evaluation when you want to score outputs within a specific range, for example, a score between 0 and 10.
• Text evaluations
  • Use this type of evaluation when you want to score outputs based on a specific set of categories.
• Boolean evaluations
  • Use this type of evaluation when you want to score outputs based on a yes or no answer.