> ## Documentation Index
> Fetch the complete documentation index at: https://docs.latitude.so/llms.txt
> Use this file to discover all available pages before exploring further.

# Flaggers

> Automatic annotators that surface common failure categories on every trace

# Flaggers

Flaggers are Latitude's built-in automatic annotators. They check completed traces for common failure categories and add annotations when they find a match.

Use flaggers when you want project-wide coverage for known failure patterns without manually reviewing every trace. Flagger annotations feed issue discovery, evaluation alignment, score analytics, and trace-level review.

## Available flaggers

Each project starts with flaggers for common reliability and safety categories:

<AccordionGroup>
  <Accordion title="Jailbreaking">
    Attempts to bypass safety constraints, system instructions, tool boundaries, or the assistant's intended role.
  </Accordion>

  <Accordion title="NSFW">
    Sexual or otherwise not-safe-for-work content that should be reviewed.
  </Accordion>

  <Accordion title="Refusal">
    Cases where the assistant refuses, deflects, or over-restricts a request it should be able to handle.
  </Accordion>

  <Accordion title="Frustration">
    Clear user dissatisfaction, repeated correction, annoyance, or loss of trust.
  </Accordion>

  <Accordion title="Forgetting">
    Cases where the assistant loses relevant earlier context from the same conversation.
  </Accordion>

  <Accordion title="Laziness">
    Cases where the assistant avoids doing the requested work, gives a shallow answer, or pushes work back to the user without a good reason.
  </Accordion>

  <Accordion title="Thrashing">
    Agent behavior that cycles through tools or repeated actions without making progress.
  </Accordion>

  <Accordion title="Tool Call Errors">
    Failed or errored tool invocations visible in the trace.
  </Accordion>

  <Accordion title="Output Schema Validation">
    Structured-output responses that do not conform to the declared schema.
  </Accordion>

  <Accordion title="Empty Response">
    Empty, whitespace-only, or otherwise degenerate assistant responses when a substantive answer was expected.
  </Accordion>
</AccordionGroup>

## How flaggers appear in Latitude

When a flagger matches a trace, Latitude adds an annotation to that trace. You can review it from the trace detail view, see it in score analytics, and use it as signal for issues and evaluations.

Flaggers are especially useful for patterns that are easy to miss in manual review, such as low-volume safety failures, recurring frustration, or behavior that appears only across long sessions.

## Configure flaggers

Open **Project Settings** to manage flaggers for a project. For each flagger you can adjust:

* **Enabled**: Turn the flagger on or off.
* **Sampling**: Control how aggressively Latitude checks traces for that category.

Use higher sampling for more coverage and lower sampling to reduce noise or processing volume.

<Note>
  Flagger categories are defined by Latitude. You can configure whether each flagger runs and how aggressively it samples, but not the underlying category definition.
</Note>

## Flaggers, search, and manual annotations

| Surface                                        | Best for                                                                    |
| ---------------------------------------------- | --------------------------------------------------------------------------- |
| **Flaggers**                                   | Automatic detection of known failure categories across your project.        |
| **[Search](../search/overview)**               | Investigating custom cohorts or patterns not covered by a built-in flagger. |
| **[Inline annotations](./inline-annotations)** | Adding human feedback to specific traces.                                   |

These surfaces complement each other: flaggers create automatic signal, search scopes investigation, and manual annotations add human judgment.

## Related

* [Annotations Overview](./overview): How annotations connect to scores, issues, and evaluations
* [Inline Annotations](./inline-annotations): Leave human feedback on traces
* [Search](../search/overview): Investigate patterns that flaggers do not cover
* [Issues](../issues/overview): How annotations become trackable failure patterns