Skip to main content

​
No-Code Quick Start

This guide walks you through the Latitude web interface. You’ll learn how to navigate the product, review agent interactions, annotate conversations, and understand signals, all without writing code.

​
Prerequisites

  • A Latitude account (sign up at latitude.so)
  • A project with telemetry already connected (ask your development team to set this up using the Developer Quick Start)

​
Understanding the Dashboard

When you open a project, you’ll see the main navigation with these sections:
  • Search: Find traces by meaning, and bookmark useful searches for later
  • Traces: Every interaction your agent has had, shown as a timeline
  • Signals: Failure patterns discovered from your agent’s interactions
  • Datasets: Saved trace collections for offline analysis
  • Settings: Project configuration, including flaggers

​
Exploring Traces

The Traces page shows every interaction your agent has processed. Each row represents one complete interaction, from user request to agent response. Click on a trace to see:
  • The conversation: The full message exchange between user and agent
  • Spans: Individual steps the agent took (LLM calls, tool uses, etc.)
  • Metadata: Timing, token usage, cost, and any custom metadata
  • Scores: Any evaluation, annotation, or custom scores attached to this trace
Use the filter sidebar to narrow traces by status, cost, duration, model, provider, tags, or custom metadata. Search lives right on the Traces page. Type a plain-English query into the search bar, such as “failed payments”, “frustrated user”, or “long latency on signup”, and Latitude returns the most relevant traces, ranked by a blend of keyword and semantic match. Use "quotes" for exact phrases. Filters work alongside the query, so you can scope to a time range, a specific model, traces with errors, or any custom metadata your application sends. When a search is one you’ll come back to, click Save search and give it a name. Saved searches are then a click away from the Saved searches dropdown next to the search bar, each showing its name, query, and filter count. Reopening one restores its query and filters so you, or a teammate, can pick the cohort back up. See Saved Searches for details.

​
Automatic Detection with Flaggers

Some failure categories are common enough that Latitude detects them for you. Every project starts with a set of built-in flaggers running on every completed trace:
  • Jailbreaking: Attempts to bypass safety constraints
  • NSFW: Sexual or otherwise inappropriate content
  • Refusal: The agent refuses requests it should handle
  • Frustration: Clear user dissatisfaction
  • Forgetting: The agent loses conversation context
  • Laziness: The agent avoids doing the requested work
  • Thrashing: The agent cycles between tools without making progress
  • Tool Call Errors: Failed tool invocations
  • Output Schema Validation: Structured output didn’t conform to the declared schema
  • Empty Response: The assistant returned an empty or degenerate response
When a flagger matches, it writes an annotation directly on the trace. That annotation feeds into signal discovery, scores analytics, and evaluation alignment the same way a human annotation would. You can adjust which flaggers are enabled and how aggressively they sample under Project Settings. See Flaggers for the full list.

​
Reviewing Traces

To leave human feedback on a trace, open it from any list (Search, Traces, a signal’s logs) and use the annotation panel on the right:
  • Click anywhere in the conversation to create a message-level annotation, or use the button for a conversation-level one.
  • Mark it as positive (thumbs up) or negative (thumbs down).
  • Write feedback describing what you observed.
  • Optionally link it to an existing signal, or leave the assignment automatic.
A typical review session combines saved searches and inline annotations: open the saved search you’re responsible for, click the first trace, annotate, move on. The saved search’s Annotated count goes up as you work.

​
Understanding Signals

The Signals page shows failure patterns your agent is experiencing. Signals are discovered automatically when failed scores share similar feedback, and you can also create one yourself. Each signal has:
  • A name and description summarizing the failure pattern
  • A lifecycle state: New, Escalating, or Ongoing
  • Linked evaluations that monitor for this signal on live traffic
  • Occurrence trends showing how often the signal appears
You can:
  • Generate an evaluation from a signal to monitor it on live traffic
  • Assign a signal and set its priority to triage it
  • Mute a signal that isn’t worth acting on, which moves it to the Archived tab and stops its notifications

​
Understanding Evaluations

The Evaluations page shows automated monitors that score your agent’s interactions in real time. Evaluations are often generated from signals. When you choose Generate an evaluation on a signal, Latitude builds a monitor that watches for that pattern on live traffic. You can also define a signal’s evaluation yourself when you create the signal. Each evaluation shows:
  • Score trends over time
  • Alignment with human annotations (how well the automation agrees with human reviewers)
  • Trigger configuration (which traces it monitors and how often)

​
Understanding Scores

Scores are the fundamental unit of measurement in Latitude. Every score has:
  • A value between 0 and 1
  • A pass/fail verdict
  • Feedback text describing the verdict
  • A source: evaluation, annotation (human review), or custom
Scores appear throughout the product: on traces, in evaluation dashboards, and in signal details.

​
What’s Next

  • Search: Build cohorts of traces with hybrid search
  • Saved Searches: Bookmark useful searches and assign ownership
  • Flaggers: Built-in automatic annotators for common failures
  • Scores: Deep dive into how scores work
  • Annotations: Human review workflows
  • Signals: Learn about signal lifecycle and management
  • Evaluations: Understand automated monitoring