Documentation Index
Fetch the complete documentation index at: https://docs.latitude.so/llms.txt
Use this file to discover all available pages before exploring further.
Annotations
Annotations are verdicts on individual traces. A human reviewer, a Latitude flagger, or an external system can attach a thumbs-up or thumbs-down verdict with feedback to a conversation, message, or text range. Finalized annotations become scores. They feed analytics, issue discovery, and evaluation alignment the same way regardless of where they came from.How Annotations Get Created
Annotations can come from:- Inline review from any trace detail view. See Inline Annotations.
- Flaggers when a trace matches a known failure category such as jailbreaking, frustration, or tool call errors. See Flaggers.
- Your own systems through the Annotations API.
How to Annotate
Every annotation has a verdict, feedback, and an optional issue link.- Choose a scope: conversation, message, or text range.
- Give a verdict: thumbs up for good behavior, thumbs down when something went wrong.
- Write feedback: explain the reason in natural language.
- Optionally link an issue: choose a known issue, or let Latitude match it during issue discovery.
Where to Annotate
Open any trace detail view—from Traces, Search, Issues, or Sessions—and use the annotation panel on the right. For batch review, start with a saved search, then work through the matching traces one at a time. If you want automatic coverage for known failure categories, use flaggers. If you are building your own feedback UI, submit annotations through the Annotations API.Why Annotate
Annotations are the foundation of Latitude’s reliability loop. They help you:- Calibrate evaluations by comparing automated scores with human judgment. See Alignment.
- Validate issues by confirming whether discovered failure patterns are real problems.
- Capture qualitative feedback that explains why something was good or bad.
How Annotations Connect to Other Features
| Feature | Relationship |
|---|---|
| Scores | Each finalized annotation becomes a score for analytics and dashboards. |
| Issues | Failed annotations can cluster into trackable issues. |
| Evaluations | Annotations provide ground truth for measuring evaluation accuracy. |
| Search | Search and saved searches help you find trace cohorts to review. |
| Flaggers | Flaggers create automatic annotations for common failure categories. |
Next Steps
- Inline Annotations: Annotate directly from trace views
- Flaggers: Automatic annotators for common failure categories
- Search: Build cohorts to annotate
- Evaluation Alignment: See how annotations calibrate evaluations