Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.latitude.so/llms.txt

Use this file to discover all available pages before exploring further.

​
Core Concepts

Latitude is organized around a small set of concepts that connect production telemetry to issue discovery. A project receives agent activity as spans, traces, and sessions; search helps you find relevant traces; scores record judgments about trace quality; evaluations monitor known behaviours; and issues group repeated failures into trackable problems. 
Project
├─ Sessions
│  └─ Traces
│     └─ Spans
├─ Search over traces
├─ Scores attached to traces
├─ Evaluations monitoring traces
└─ Issues grouped from failed scores

​
Projects

A project is the main workspace for one agent, application, or AI feature. Projects keep production data and reliability workflows scoped to the right product boundary. Project-scoped data includes:
  • spans, traces, and sessions
  • search and saved searches
  • scores from annotations, evaluations, flaggers, and custom checks
  • issues discovered from failed scores
  • evaluations and simulations
  • project settings such as monitoring behaviour
Projects page showing a list of projects with issues, datasets, and trace counts
Projects belong to organizations. Organization membership, API keys, and settings control who can access a project and how defaults cascade.

​
Spans

A span is the smallest unit of captured work. It represents one operation inside an agent run, such as an LLM call, tool invocation, retrieval step, HTTP request, or custom instrumented operation. Spans can capture:
  • input and output content when available
  • start time, end time, and duration
  • success or error status
  • tags and metadata
  • provider, model, token usage, and cost for LLM calls
Use spans when you need to understand exactly what happened inside a trace.

​
Traces

A trace is one complete interaction from start to finish, composed of one or more spans. For a chat agent, a trace commonly represents a single user turn and the agent work needed to respond. Traces are the main unit used across Latitude:
  • the trace detail view shows the full execution path
  • search returns matching traces
  • annotations, scores, and evaluations attach to traces
  • flaggers run on completed traces
  • issues group related failed trace signals
Latitude waits for a trace to stop receiving new spans before treating it as complete. Downstream workflows such as search indexing, flaggers, evaluations, scores, and issue discovery run on completed traces so they do not act on partial executions.

​
Sessions

A session groups related traces into a multi-turn conversation or workflow. Traces always exist; sessions exist when your application sends a stable session id with telemetry. Use sessions when multiple traces belong to the same user conversation, support ticket, workflow, or agent task. For example, each chat turn can be one trace while the whole conversation shares one session. Sessions help teams:
  • review full conversation context across turns
  • group traces by a stable conversation id
  • investigate user-level or workflow-level behaviour
  • understand whether a failure was isolated or part of a longer interaction
Search finds traces by meaning, exact text, metadata filters, or a combination of all three. It is the discovery layer for behaviours your team wants to inspect, review, and track. Use search to find traces such as:
  • users showing frustration
  • failed payments or onboarding problems
  • tool loops and tool call errors
  • hallucinated or incomplete answers
  • traces from a specific user, model, release, environment, tag, or metadata value
Saved searches let teams bookmark useful behavioural cohorts and review them repeatedly.

​
Scores

A score is Latitude’s common verdict model for trace quality. Scores can come from human annotations, built-in flaggers, automated evaluations, or custom checks submitted through the API. Scores answer questions such as:
  • Did this trace pass or fail a quality check?
  • What feedback explains the verdict?
  • Which signal source produced the verdict?
  • Should this failure contribute to issue discovery?
Failed, eligible scores are the primary input for issue discovery. Scores also power analytics and help teams compare automated judgments with human feedback.

​
Issues

An issue is a recurring failure pattern discovered from failed scores. Instead of leaving failures as isolated examples, Latitude groups similar failures into named issues with examples, trends, affected users, lifecycle states, and linked evaluations. Issues help teams:
  • triage new and escalating production problems
  • inspect representative traces for the failure pattern
  • decide whether a cluster is real, noise, resolved, or ignored
  • generate evaluations that monitor the same failure mode on future traces
  • catch regressions after a fix ships
Issue states include new, escalating, resolved, regressed, and ignored. An issue can have multiple states at once, such as new and escalating.

​
Evaluations

An evaluation is an automated monitor that checks traces for a specific behaviour or quality criterion. Evaluations run on completed traces and produce scores, so their results feed the same analytics and issue discovery workflows as annotations, flaggers, and custom checks. Evaluations help teams:
  • monitor known failure modes on live traffic
  • measure whether a fix reduced or eliminated a behaviour
  • catch regressions after an issue is resolved
  • turn important issues into ongoing monitors
  • compare automated judgments with human annotations through alignment metrics
Latitude can generate evaluations from issues, then use human feedback to keep those evaluations aligned with the team’s judgment.

​
How the concepts fit together

  1. A project receives telemetry from your agent.
  2. Agent operations arrive as spans.
  3. Related spans form completed traces.
  4. Related traces can be grouped into sessions.
  5. Search finds traces that match behaviours your team cares about.
  6. Annotations, flaggers, evaluations, and custom checks create scores on traces.
  7. Evaluations monitor known behaviours and can be generated from important issues.
  8. Failed scores cluster into issues that your team can triage, monitor, and fix.

​
Next steps

  • Start tracing: send your first spans and traces to Latitude.
  • Observability: inspect traces, sessions, and telemetry metadata.
  • Search: find traces by meaning, exact text, and filters.
  • Evaluations: monitor known behaviours and issue patterns.
  • Issue Discovery: understand how failed scores become issues.