Changelog
All notable changes to Latitude.
Latitude Telemetry
We’ve released the 1.0 version of Latitude’s Typescript SDK, which ships with support for OpenLLMetry. You can now trace all your LLM generations with a couple lines of code and start using Latitude’s powerful prompt manager by creating prompts from your traces.
Here’s the full list of supported providers/sdks at launch:
And here’s the documentation to get started with Telemetry.
Version history
We’ve added a new history section in projects. It displays the full history of prior committed versions of your project.
From a prompt, you can easily inspect its history by clicking on the history button at the top of the editor.
File / Image prompt parameters
We’ve introduced support for “file” content-type in PromptL, unlocking OpenAI input_audio audio files, Anthropic document PDF files and Google fileData audio, video and document files.
And since Latitude uses PromptL, you can now upload files and images to prompts as prompt parameters.
Public prompts also support this feature, btw.
Other stuff
- General improvements to stability and performance
Welcome! A bit of a lighter update as we gear for a last big suprise before christmas, next week. Here’s this week’s highlights:
Create prompts from documents
Following up from the previous feature, we now allow you to create prompts by simply uploading a document. This is a great way to quickly create prompts from a document without having to write the prompt yourself.
Log Filters
We’ve added a new feature that allows you to filter your logs by a set of commonly requested criteria.
Keep an eye out for more filters to come!
Other stuff
- We’ve increased our keep alive timeout setting to 10 minutes in order to match it with OpenAI’s default setting. This helps when you are trying to generate a big JSON response. That being said, we always recommend using streaming responses when you expect a large response, as it effectively means you have limitless timeout.
- Updated some old documentation
- Released a new homepage!
- General improvements to stability and performance
Public prompts
We have released a new feature that allows you to share your prompts with the world. Here’s an example of a public prompt we’ve created.
To share your prompts, simply click the new share button in the prompt editor.
Export logs
You can now select logs from the logs section and automatically create a dataset from them or download them as a CSV file.
This allows you to easily create golden datasets of parameters for your prompts and use them to test new prompt iterations at scale.
Collapsible parameters
We have updated the prompt preview in our prompt editor. Now parameter values are automatically collapsed and can be expanded with a simple click.
Overview page
We have added a new project overview page that gives you a quick overview of the project’s overall cost, prompts, and evaluations.
Compile Latitude prompts locally
You can now compile Latitude prompts straight from your code using our SDK. This allows you to still use Latitude’s prompt editor, for iterating and evaluating your prompts at scale, while maintaining your existing provider integrations to run the prompts from your code. docs.
Other improvements
- Added tool calls to LLM-as-judge evaluations’ context
- Fixed datasets containing JSON not working correctly in some scenarios
- General improvements in stability and performance
PromptL activated in Latitude
We have implemented the new version of our template syntax – PromptL – in Latitude. As a reminder, PromptL is a new template syntax that has native support for html/xml tags, contextless chain steps, as well as a slew of other improvements that make it writing prompts in Latitude the best way to write prompts. docs
Refine prompt directly from evaluation logs
One of the most powerful features of Latitude is the ability to improve your prompts based on results from evaluations – we call it Refiner. We have now made this process easier by directly allowing users to choose evaluation results from the evaluations page and trigger the refiner from there.
Evaluation results in Logs
You can now see evaluation results in the logs section of your prompts. For each log that has an evaluation result associated, the result will show up in the details section of that log.
Other improvements
- You can now edit a version title and description before publishing it
- You can now rename projects
- Several improvements in stability and performance
Human / Code evaluations
We have released a new type of evaluations: manual / code evaluations. This new evaluation type allows users to evaluate their LLM outputs with human feedback or code-based evaluations, and push the results to Latitude using our SDKs/API.
You can also submit results directly from Latitude’s UI.
New prompt template syntax
We have open sourced the new version of our prompt templating syntax and we’ve even given it a new name: PromptL. This new syntax introduces some highly requested features such as support for html/xml tags without needing to escape them, chain steps with custom contexts, and more.
The new syntax will be enabled to all new prompts in Latitude by default starting Monday 25th November. Since the new syntax is not compatible with the old one, existing prompts will not get automatically upgraded to the new syntax and users will be in charge of updating them.
New parameters section for prompts
We have revamped our parameters section in prompts and introduced some highly requested features. Users can now choose between inputing parameters manually, from datasets, or from existing prompt logs. Moreover, any choice they make in any of these sections gets automatically stored in session so that you don’t lose track of the latest inputs you chose if you ever navigate to another section and later come back.
Prompt analytics
We have added some key metrics to the logs section of your prompts. You can now see at a glance the number of prompt runs, average latency and cost, and more.
Default provider and models
We have added a new section in the settings page where you can set default providers and models for your prompts. This allows you to quickly change the default settings for your prompts without having to go through the prompt creation flow every time.
More improvements
- You can now get and create prompts from the SDK/API docs
- You can now eject from simple LLM-as-judge evaluations into more complete advanced evaluations that give you complete control over the evaluation prompt
- Updated UI code snippets on how to push logs and evaluations to Latitude
- Several improvements in infrastructure stability and performance
- Several improvements and fixes to UI/UX
New evaluations playground
We have completely revamped our evaluations to make it super simple to create new evaluations from scratch. From now on you’ll only need to worry about typing the goal of your evaluation—as well as any additional instructions that might be useful.
Latitude Cookbook
We’ve started work on Latitude’s Cookbook showcasing common use cases with Latitude’s SDK. Here you can find the first examples.
Anthropic cache
We have added support for Anthropic’s prompt caching beta feature.
Rust SDK
Our community member @Dominik Spitzli has implemented a Rust port of Latitude’s SDK!
Latitude Typescript SDK v1 released
We’ve released the first major version of Latitude’s SDK, v1.0.0, currently in beta. It adds support for evaluations, pushing logs, JSON API, and more.
Other improvements
- Dramatically improved performance of the prompt editor on large prompts
- Improved error reporting in the prompt editor
- Long-lived modals no longer close on click-outside or hitting ESC key
- Prompt input parameters are now stored in memory so that you can navigate to other sections and come back without losing the latest inputs you used in a specific prompt
Upload external logs
Users have long asked us to evaluate their prompts without having to run them via Latitude’s Gateway. Well, we now support this use case. You can now upload external logs to Latitude for evaluation so that, even if you run your prompts outside of Latitude, you can keep tracking their performance. We support uploading logs to Latitude both from the UI and our SDK/HTTP API.
Trigger evaluations from SDK
In cases where AI agents have long running conversations with users users only want to evaluate the agent’s performance at particular points in time (i.e when the conversation has finished). You can now trigger evaluations from our SDK / HTTP API, giving you the tools to trigger evaluations at the precise moment you require it.
JSON API
We’ve released the v2 version of our Gateway API, which supports non-streaming responses for the run
and chat
endpoints. We’ve also released the v1 major version of our SDK, which introduces support for the new HTTP API version, as well as the features above described.
Other improvements
- Improved performance of prompt editor in large prompts
- Added code examples on how to use the SDK to the OSS repository
- Improved and fixed documentation in several places
- Several performance and stability improvements