Skip to main content

​
Scores API

Use custom scores for measurements produced by your application or evaluation pipeline. Use annotations for human feedback such as thumbs up and thumbs down. All endpoints use the project slug in the URL. The API key or OAuth token supplies the organization scope.

​
Custom scores

Create a custom score with:
Custom score creation is append-only. Reusing sourceId creates another score; it does not replace an earlier score or act as an idempotency key.

​
Human feedback

Create a human annotation with:
The response contains a Latitude-generated id. Store that identifier with the message or vote in your application, then use it to read, update, or delete the annotation:

​
Create a thumbs-up annotation

​
Change it to thumbs down

PATCH retains the annotation ID. You may send any combination of value, passed, and feedback; omitted fields keep their current values. To undo the vote, send DELETE using the same annotation ID. To restore all annotations for a trace, use the paginated trace endpoint:
The trace list includes each annotation ID, so a client can restore its message-to-annotation mapping after a reload. The annotation response also includes traceId, spanId, and sessionId; Latitude derives the session and default span from the target trace.

​
Message and text anchors

Annotations accept an optional anchor object. Use messageIndex to attach feedback to a message, then optionally add partIndex, startOffset, and endOffset for a text selection.

​
Trace ingestion timing

The target trace must be queryable before Latitude can attach a score or annotation. A request sent immediately after exporting telemetry can temporarily return 404 Trace not found. Retry that response with bounded exponential backoff and jitter. A trace-not-found response does not create an annotation, so retrying after that response is safe. Once POST succeeds, retain the returned annotation ID for later reads and mutations. Filter-based trace selection does not bypass ingestion. It must match exactly one existing trace and is useful when you have a unique trace attribute but not its trace ID.

​
How feedback feeds the system

Published custom scores and annotations participate in analytics. Failed scores and annotations can enter signal discovery, and annotations can be compared with evaluation scores for alignment analysis.