Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.latitude.so/llms.txt

Use this file to discover all available pages before exploring further.

​
Flaggers

Flaggers are Latitude’s built-in automatic annotators. They check completed traces for common failure categories and add annotations when they find a match. Use flaggers when you want project-wide coverage for known failure patterns without manually reviewing every trace. Flagger annotations feed issue discovery, evaluation alignment, score analytics, and trace-level review.

​
Available flaggers

Each project starts with flaggers for common reliability and safety categories:
Attempts to bypass safety constraints, system instructions, tool boundaries, or the assistant’s intended role.
Sexual or otherwise not-safe-for-work content that should be reviewed.
Cases where the assistant refuses, deflects, or over-restricts a request it should be able to handle.
Clear user dissatisfaction, repeated correction, annoyance, or loss of trust.
Cases where the assistant loses relevant earlier context from the same conversation.
Cases where the assistant avoids doing the requested work, gives a shallow answer, or pushes work back to the user without a good reason.
Agent behavior that cycles through tools or repeated actions without making progress.
Failed or errored tool invocations visible in the trace.
Structured-output responses that do not conform to the declared schema.
Empty, whitespace-only, or otherwise degenerate assistant responses when a substantive answer was expected.

​
How flaggers appear in Latitude

When a flagger matches a trace, Latitude adds an annotation to that trace. You can review it from the trace detail view, see it in score analytics, and use it as signal for issues and evaluations. Flaggers are especially useful for patterns that are easy to miss in manual review, such as low-volume safety failures, recurring frustration, or behavior that appears only across long sessions.

​
Configure flaggers

Open Project Settings to manage flaggers for a project. For each flagger you can adjust:
  • Enabled: Turn the flagger on or off.
  • Sampling: Control how aggressively Latitude checks traces for that category.
Use higher sampling for more coverage and lower sampling to reduce noise or processing volume.
Flagger categories are defined by Latitude. You can configure whether each flagger runs and how aggressively it samples, but not the underlying category definition.

​
Flaggers, search, and manual annotations

SurfaceBest for
FlaggersAutomatic detection of known failure categories across your project.
SearchInvestigating custom cohorts or patterns not covered by a built-in flagger.
Inline annotationsAdding human feedback to specific traces.
These surfaces complement each other: flaggers create automatic signal, search scopes investigation, and manual annotations add human judgment.