Hi there, I'm wondering what determines the Serverless Compute spark version? Is it based on the current DBR LTS? And is there a way to modify the spark version for serverless compute?
For example, when I check the spark version for our serverless compute, I see that it matches the current DBR 17.3 LTS.
Hi @ChristianRRL ,
The answer is simple 🙂 Serverless compute always run on the latest runtime version. You cannot choose it like in standard compute.
Connect to serverless compute | Databricks on AWS
In Serverless compute you can only choose different environment (i.e each environment version includes a specific Python version and a set of Python packages with defined versions.)
Serverless environment versions - Azure Databricks | Microsoft Learn
