​
Overview

Latitude allows you to automatically trace all your LLM calls with our OpenTelemetry integration. This enables you to analyze and debug all your LLM generations which just a couple lines of code, as well as getting into more granular investigations if needed.

​
How it works

Once you’ve enabled the OpenTelemetry integration in your app, Latitude will automatically display all your LLM calls and their corresponding spans in the Traces section.

Clicking on a trace will display a side panel with the details of the trace.

The trace also contains a transcript of the full prompt run and subsequent messages.

Finally, the Trace might also include a timeline split into different spans, which can be used to analyze the performance of each span individually.

You can filter traces by various attributes such as the trace ID, start time, and more. This will allow you to quickly find the trace you’re interested in.

​
Create a prompt from a trace

If you have a trace that you want to use as a prompt, you can easily create a new prompt from it. This will allow you to easily share the trace with your team and use it as a reference for future iterations.

To create a prompt from a trace, you can use the create prompt from trace button in the trace details side panel.

This process will run your trace through our LLM agent which will generate the best prompt based on the trace’s contents.

​
Adding spans to a trace

Spans are a great way to split complex workfows into its individual steps and analyze them individually. Spans can be added to a trace by wrapping the relevant code section in a telemetry.span method:

import { Latitude } from '@latitude-data/sdk'

const latitude = new Latitude('YOUR_API_KEY')

const completion = latitude.telemetry.span(
  {
    name: 'handleChat',
  },
  () => {
    openai.chat.completions.create({
      model: 'gpt-4o-mini',
      max_tokens: 1000,
      // ...
    })
  },
)

You can add several attributes to spans such as serializable metadata, a distinct ID, and more.

​
Adding attributes to a span

Spans accept a variety of attributes that can be used to enrich the trace and provide more insights into the performance of your LLM calls. Those attributes are:

  • name: The name of the span. This is a required field.
  • metadata: A map of key-value pairs that can be used to store additional information about the span.
  • prompt: A map of key-value pairs that can be used to link the span to a prompt in the prompt manager.
  • distinctId: A unique identifier for the span. Useful to trace a span to a tenant in your app.

These attributes can be added to a span by passing them as an object to the telemetry.span method:

import { Latitude } from '@latitude-data/sdk'

const latitude = new Latitude('YOUR_API_KEY')

const completion = latitude.telemetry.span(
  {
    name: 'handleChat',
    metadata: {
      userId: '123',
    },
    distinctId: '123',
    // ...
  },
  () => {
    openai.chat.completions.create({
      model: 'gpt-4o-mini',
      max_tokens: 1000,
      // ...
    })
  },
)

​
Assigning spans to prompts

If your are using Latitude’s prompt manager to iterate a prompt, it is advisable to link the LLM generations in your code to the relevant prompts in the prompt manager. This will provide several benefits such as the ability to track the performance of your prompt over time, the ability to set up running evaluations, and more.

You can easily do so by wrapping a generation in a span with a pointer to the prompt:

import { Latitude } from '@latitude-data/sdk'

const latitude = new Latitude('YOUR_API_KEY')

const completion = latitude.telemetry.span(
  {
    name: 'handleChat',
    prompt: {
      uuid: 'YOUR_PROMPT_UUID',
      versionUuid: 'YOUR_PROMPT_VERSION_UUID','bb // Optiona
      parameters: { user_message: 'Hello' }, /'bb/ Optional
    },
  },
  () => {
    openai.chat.completions.create({
      model: 'gpt-4o-mini',
      max_tokens: 1000,
      // ...
    })
  },
)
Running EvaluationsLogs