Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.latitude.so/llms.txt

Use this file to discover all available pages before exploring further.

​
Issue discovery

Issues are recurring failure patterns in your agent’s production traffic. Latitude groups failed scores into issues, then gives each issue a name, examples, trends, and a lifecycle your team can act on. The loop is simple: find bad behavior, explain it, monitor it, fix it, and catch regressions.

​
How issue discovery works

Traces
  ↓
Annotations, flaggers, evaluations, and custom checks
  ↓
Scores
  ↓
Issue discovery
  ↓
Issues with examples, trends, status, and linked evaluations

​
1. Traces capture real behavior

Your telemetry sends real user and agent interactions into Latitude as traces. These traces are the raw material for issue discovery.

​
2. Signals produce scores

Latitude uses several signal sources to decide whether a trace represents good or bad behavior:
  • Annotations: Human feedback left on traces during review.
  • Flaggers: Built-in automatic annotators for categories such as frustration, refusal, jailbreaking, tool errors, and empty responses.
  • Evaluations: Automated monitors that track issue patterns across incoming traces.
  • Custom scores: Domain-specific verdicts you submit from your own systems.
All of these produce scores: Latitude’s common unit for verdicts, feedback, analytics, and issue discovery.

​
3. Failed scores become issue candidates

When a score fails, Latitude compares its feedback and trace context against existing issues.
  • If it matches an existing issue, the score becomes a new occurrence of that issue.
  • If it does not match, Latitude can create a new issue with a generated name, description, and example traces.
You do not need to predefine every failure category. Latitude learns issue clusters from the failures that appear in your traffic.

​
4. Issues become monitors

Important issues can generate evaluations. These monitors watch live traffic for the same failure pattern, measure it over time, and detect regressions after you fix it. As new annotations, flagger matches, and scores arrive, Latitude can realign the evaluation so it stays calibrated to the latest signal.

​
The Issues page

Issues page showing discovered failure patterns with status, trends, and occurrence data
The Issues page shows discovered failure patterns for your project. Summary cards highlight regressions, escalating issues, active issues, new issues, and total events. A histogram shows issue occurrences over time. The table lists each issue with:
  • Issue name: The generated name for the failure pattern
  • Status: The current lifecycle state
  • Trend: Recent occurrence activity
  • Total events: How many times the issue has been detected
  • Affected users: The percentage of users impacted
  • Evaluations: Linked monitoring evaluations and their status
Use the Active / Inactive tabs to switch between current issues and resolved or ignored ones.

​
Issue detail

Click any issue to open the detail drawer:
Issue detail drawer showing status, trend, linked evaluation, and trace logs
The drawer shows the issue description, lifecycle state, trend, linked evaluations, and recent traces where the issue was detected. Open example traces to understand what triggered the issue and what the agent did wrong.

​
Issue lifecycle

StateMeaning
NewNewly discovered
EscalatingOccurrences are increasing compared to the recent baseline
ResolvedMarked fixed or inactive
RegressedNew occurrences appeared after the issue was resolved
IgnoredHidden from the active workflow because it is not worth tracking
An issue can be in multiple states at once. For example, a newly discovered issue can also be escalating.

​
Common workflows

​
Investigate traces

Use the issue’s example traces to understand common user intents, missing context, tool failures, retrieval problems, prompt gaps, or model behavior that needs to change.

​
Generate an evaluation

Click Generate Evaluation to create an automated monitor for the issue. The evaluation runs on future traces and helps track whether the issue is still happening.

​
Resolve or ignore

  • Resolve when the underlying problem is fixed. Keep monitoring enabled if you want Latitude to catch regressions.
  • Ignore when the issue is noise or not worth tracking. Ignored issues leave the active workflow.
  • Issue Management: Triage, investigate, resolve, and ignore issues
  • Annotations: Leave human feedback on traces
  • Flaggers: Detect common failure categories automatically
  • Evaluations: Monitor known failure patterns
  • Scores: Understand Latitude’s common measurement unit