Serverless Compute Spark Version Flexibility?

[ChristianRRL]

Hi there, I'm wondering what determines the Serverless Compute spark version? Is it based on the current DBR LTS? And is there a way to modify the spark version for serverless compute?

For example, when I check the spark version for our serverless compute, I see that it matches the current DBR 17.3 LTS.

[szymon_dybczak · Accepted solution]

Hi @ChristianRRL ,

The answer is simple 🙂 Serverless compute always run on the latest runtime version. You cannot choose it like in standard compute.

Connect to serverless compute | Databricks on AWS

In Serverless compute you can only choose different environment (i.e each environment version includes a specific Python version and a set of Python packages with defined versions.)

Serverless environment versions - Azure Databricks | Microsoft Learn

[Databricks77]

Serverless compute always run on the latest runtime version. You cannot choose it like in standard compute.