Pradeep54
Databricks Employee
Databricks Employee

@DinoSaluzzi If you can restructure your system& user prompt in a similar manner to the examples provided, prompt caching should start working as expected.

messages = [
    {
        "role": "system",
        "content": [
            {
                "type": "text",
                "text": "You are a helpful Apache Spark expert. Always provide concise, technical answers.",
                "cache_control": {"type": "ephemeral"}
            }
        ]
    },
    {
        "role": "user",
        "content": "What are the top 3 benefits of using Apache Spark?"
    }
]

 

I can confirm that both cache_read_input_tokens and cache_creation_input_tokens are updating correctly, which indicates that caching is being applied.
 Please note that prompt caching does not activate for smaller prompts , it is typically triggered only when the prompt size crosses a certain threshold.