Cache

Latitude implements a caching system for prompt responses to optimize performance and reduce costs. This guide explains how caching works and when it’s applied.

​

How Caching Works

When you execute a prompt, Latitude automatically caches the response if certain conditions are met. The cache key is generated based on:

• The workspace ID
• The prompt configuration
• The conversation context

This means that identical prompts with the same parameters in the same workspace will return cached results.

​

Cache Conditions

Caching is only applied when:

• The temperature is set to 0 or not specified
• The prompt execution is successful

This is because non-zero temperatures introduce randomness in the responses, making caching less useful as each execution is intended to be unique.

​

Benefits

Caching provides several advantages:

• Reduced Costs: Cached responses don’t consume additional API tokens
• Faster Response Times: Cached results are returned immediately
• Consistency: Identical prompts always return the same response

​

Cache Duration

Currently, cached responses are stored indefinitely. However, you can force a fresh execution by:

• Modifying any part of the prompt configuration
• Changing the conversation context
• Using a non-zero temperature