Skip to main content

​
Monitors

A monitor watches your production traffic and opens an incident when a condition you care about is met. Each monitor has one alert, which watches a single signal or saved search with its own condition and severity. When the alert fires, Latitude records an incident and sends you a notification through your existing notification channels. Monitors live on the Monitors page of each project. Open it from the project sidebar.
The Monitors page listing system and user monitors with their status, last incident, and condition

The Monitors page: system and user monitors side by side, each with its status, last incident, and a plain-English summary of its condition.

​
System monitors vs. your monitors

There are two kinds of monitors:
  • System monitors are set up automatically for every project. They watch your signals and are marked with a System badge. You can’t create, rename, or delete them, and you can’t change what they watch, but you can mute them.
  • Your monitors are the ones you create. Today they watch saved searches. You have full control: edit the alert, rename, mute, and delete.
A system monitor row showing its name next to the System badge

System monitors have a System badge next to their name.

The Monitors list shows both, with columns for: Click any row to open its details panel, where you can see the alert, the full incident history, and the mute control.

​
The three system monitors

Every project comes with three monitors that watch the signal lifecycle. Each one notifies you about a different moment:
  • Signal discovered: fires each time a brand-new signal is detected in your traffic.
  • Signal regressed: fires when a signal that had gone quiet starts being detected again, so a problem you thought was behind you doesn’t slip back unnoticed.
  • Signal escalating: fires when an ongoing signal is being detected more than expected for this time of day and week. This is the one monitor with a tunable knob: its sensitivity (a value from 1 to 6) controls how large a deviation from the normal pattern is needed before it fires. Open the monitor and edit the alert to change it.
System monitors can’t be deleted: they’re part of how Latitude keeps you informed about signals. If you don’t want to be notified by one, mute it (see Mute below).
The three system monitors replace the old per-project signal-notification checkboxes. Tuning who gets notified, and on which channel, still happens in your notification preferences. See How notifications work.
You create your own monitors on top of saved searches. A saved search defines which traces you care about; the monitor decides when that pattern is worth an alert. There are two ways into the create flow:
  1. From the Monitors page: click the Monitor button in the top-right and pick a saved search inside the form.
  2. From a saved search: open the Saved searches dropdown next to the search bar on the Traces page. Hover a saved search and click its bell icon for Create monitor (or View monitor if one already watches it), which opens the same form with that search pre-selected.
Searches containing a semantic part (plain unquoted words) can’t be monitored. Semantic search ranks the closest traces by meaning instead of applying an exact rule, so a monitor has no match rule to count new traces against. Use quoted "literal" or backtick `phrase` terms — those match exactly, and the monitor counts every match.
The Saved searches dropdown with a bell icon to create a monitor on a saved-search row

The Saved searches dropdown: hover a saved search and click the bell icon to create a monitor that watches it.

The create form asks for:
  • Name: a human-readable label, e.g. “Tool error spikes”.
  • Description: optional; what the monitor is for.
  • An alert: a card where you pick the saved search, the alert kind, the condition, and a severity.
The New monitor form with a saved-search alert and its plain-English preview sentence

Create a monitor form: a name, an optional description, and the alert card. The preview sentence at the bottom of the card restates the configuration in plain English.

As you fill in the alert, a one-line preview sentence at the bottom of the card restates the configuration in plain English, so you can confirm it does what you intend before saving. For example:
The alert card has a severity selector (Low, Medium, High, or Urgent) that sets the tone of the incident badge and the notification. The same scale is used for signal priority, and your notification settings can filter alerts by a minimum severity.

​
How monitors measure time

Every alert kind evaluates a sliding window of recent activity, and a session enters that window by its latest activity rather than by when it started. A session that has been running for 40 minutes is therefore checked while it progresses and again when it finishes, so long agent sessions alert instead of ageing out of the window before they end. Two things follow from that:
  • Each session alerts at most once per monitor. A session that stays active across several checks doesn’t re-alert, and a monitor can’t be flooded by one long session.
  • An incident points back at the start of the session. The incident’s “started” time is when the offending session began, which can be well before the alert arrived.
Dashboards, the Traces and Sessions pages, and the analytics API all list sessions by their start time instead. Same filters, different time axis: a session that just triggered a monitor can appear further back in those views, and a fixed date range saved inside a search is ignored while monitoring it (the sliding activity window is the monitor’s only time axis).

​
The three saved-search alert kinds

A saved-search alert can watch its search in one of three ways, shown as tabs on the alert card: Match, Threshold, and Escalating.

​
Match: alert when a matching session appears

Use Match when even one matching session is worth knowing about, for example a saved search for status = 5xx in production.
A Match alert card with the saved-search selector and severity options

A Match alert card: pick the saved search to watch and a severity. It fires the first time each matching session is detected.

Matches are alerted once per session, and the monitor is checked at most every 5 minutes:
  • A burst of 500 matching sessions arriving at once produces one alert, not 500.
  • A single long session alerts once, at the first check that sees it, however long it keeps going.
  • While new matching sessions keep appearing, you keep getting alerts, at most one every 5 minutes, so an ongoing problem stays visible instead of going quiet after the first alert.
  • After a quiet period with no matches, the next new session opens a fresh alert again.
Because a session alerts only once, a Match monitor on a busy search reads as a stream of distinct problems rather than the same one repeating.

​
Threshold: alert at a milestone or a spike

Use Threshold when one match doesn’t matter, but a lot of them do. The threshold row reads as a sentence you complete: Alert when traces are detected … [amount] [comparison] [baseline]. There are three flavors.

​
Absolute count

A simple count: alert once you’ve seen a set number of matching traces.
A Threshold alert card configured to alert when traces are detected 100 times

A Threshold alert card in absolute-count mode: the threshold row reads 'Alert when traces are detected 100 times'.

​
Multiplier: relative to a baseline

Spike detection: alert when the current volume rises to some multiple above a baseline you choose. Pick the comparison times more than and then choose the baseline:
  • The average of the last N hours/days: the normal-traffic case. Latitude divides the baseline window into 5-minute slices and compares the sessions active in your current slice against the average slice.
  • The previous period (yesterday, or the previous week): for traffic that follows daily or weekly patterns, where “normal” depends on the day or time.
Worked example. Your saved search averages 20 matches per 5-minute slice over the last 7 days. With a multiplier of 3, the alert fires when the current 5-minute window crosses 60 matches (3 × 20). If your baseline rises to 30/slice next week, the trigger automatically rises to 90, and you don’t re-tune it.

​
Expected: a baseline that learns your patterns

The smartest option: alert when you’re seeing more traffic than expected for this time of day and week. Unlike average or previous-period, you don’t pick a comparison window at all: Latitude learns what’s normal for each time of day and day of the week on its own.
Choose expected when your traffic swings a lot depending on the time of day or day of week (busy weekday mornings, quiet weekends). A single average blurs those swings together; expected accounts for them.
How “expected” works. From your trace history, Latitude builds a profile of normal volume for each slot in the week (every hour-of-day × day-of-week pair) together with a tolerance band for how much that volume naturally varies. On each check it compares the current window’s count against the expectation for that slot and fires when the count breaks above the band. Sensitivity sets how wide the band is: a higher value tightens it, so smaller deviations trip the alert. It’s the same seasonal detector that powers the Signal escalating system monitor.

​
Escalating: alert only when a spike sticks around

Use Escalating when you want to ignore short blips that fix themselves, and only hear about a problem that sticks around. An escalating alert combines any of the threshold modes above with a window:
An Escalating alert combining a more-than-expected threshold with a sustained window

An Escalating alert in expected mode: 'detected 3 times more than expected', plus a Window that requires the condition to be sustained for at least 5 minutes.

The window field does two things:
  • It’s the measurement window: the condition has to stay true for the whole window, not just for a moment. A single one-minute spike won’t trip it.
  • It’s the cool-down: once an incident opens, it stays open until things have been calm for that same window, so it doesn’t rapidly open and close.
An escalating alert can wrap any threshold mode (including a plain absolute count, as below) with that window:
An Escalating alert combining an absolute threshold with a sustained-for-at-least window

The same Escalating kind wrapping a plain absolute count: 'detected 100 times, sustained for at least 5 minutes'.

That’s why escalating alerts are the right choice for the kind of thing you’d want to be paged about: short, noisy spikes get filtered out automatically.

​
Mute, delete, and edit

​
Mute

Muting a monitor stops it from notifying you, but it keeps recording incidents. Open the monitor and use the Mute / Unmute button, or use the actions menu on the list row. Muting is the right tool for a monitor that’s correct but currently noisy: you keep the incident history for later, you just stop the notifications. A muted monitor shows the Muted status in the list; a live one shows Live.

​
Delete

You can delete your own monitors from the list’s actions menu. A few rules:
  • System monitors can’t be deleted: mute them instead.
  • A monitor keeps its alert for life — the alert can’t be removed on its own; it’s edited in place or deleted along with the monitor.
  • Deleting a monitor stops it firing and hides it from the list. Incidents it already recorded remain queryable.

​
Edit

What’s editable depends on the monitor type: System monitors are locked: you can’t change their name, alert, or severity. The single tunable value is the Signal escalating sensitivity.

​
How notifications work

When a monitor fires, the incident goes through Latitude’s normal notification system, the same one used everywhere else. Monitors themselves don’t carry notification settings:
  • Who gets notified and on which channel is controlled by your per-user and per-project notification preferences, not by the monitor.
  • A monitor’s only effect on delivery is its mute state: a muted monitor still records incidents but stops the notifications from going out.
Incidents can be delivered by email and to Slack. Email is toggled per person in notification preferences. Slack is connected once per organization at Settings → Organization → Integrations: after you connect a workspace, you route the Monitors group to the Slack channels you choose, with the Signals group routed separately for signal notifications. See Notifications. To change channel routing or per-user preferences, use your notification settings rather than the monitor.

​
Next steps