Experiments

An Experiment is a side-by-side dashboard. You define a baseline slice of your data and one or more comparison slices, and Latitude computes every metric it already tracks for each slice, then shows how each comparison moved relative to the baseline — greener when a change is good, redder when it’s bad.

​

Variants

Each slice is a Variant: a saved population selector made of three things.

• Filters — the same session filters you use on the dashboard (model, status, user, tool, metadata, duration, and so on).
• Search query — an optional free-text or semantic query.
• Time range — a preset (last day, last week, last month…) or a fixed calendar window. Presets stay live, so a “last 7 days” variant always covers the trailing week.

​

Baseline

You can mark one Variant as the baseline. Every other Variant’s metrics are compared against it, allowing you to see easily how your new changes compare to previous versions of your agent.

​

What gets compared

For every Variant, an experiment computes the full set of analytics Latitude produces, grouped by type:

• Sessions — count, distinct users, total and average cost and tokens, error rate, cache hit rate, and duration percentiles.
• Users — distinct users and per-user rollups (sessions, traces, cost, duration, error rate).
• Tools — calls, distinct tools, usage rate, error rate, and duration percentiles, plus the top tools.
• Signals — distinct signals, occurrences, affected sessions/traces/users, and cost impact, plus the top signals.
• Behaviours — observations, distinct clusters, and detected moments, plus the top behaviours.

​

Population deviation

A Variant whose population differs from the baseline by more than 25% (in session or user count) is flagged, since very different population sizes make the comparisons less reliable.

​

Semantic queries

A semantic search query returns a ranked sample rather than an exact set, so any Variant whose query has a semantic component is marked approximate and its metrics are best-effort. Filter-only and literal/phrase queries are exact.

​

Creating an experiment

From a project, open Experiments and create one with a name and optional description. A new experiment starts with two variants (a baseline and one comparison) so it’s useful immediately; edit their filters, query, and time range, then read the comparison. “Import from search” seeds a Variant from a saved search’s filters and query.