topic Re: How to select performance mode for Databricks Delta Live Tables in Data Engineering

How to select performance mode for Databricks Delta Live Tables

QuanSun — Mon, 28 Apr 2025 13:40:54 GMT

Hi everyone,

For triggered pipelines, you can select the serverless compute performance mode using the Performance optimized setting in the pipeline scheduler. When this setting is disabled, the pipeline uses standard performance mode.

we can select 2 performance modes for pipelines (i.e. delta live tables).

But I tried everything, I was not able to find the location to select/disable/enable performance modes. Could you please tell me how to do so? Or share a screenshot?

Btw, in my Azure databricks workspace, I can use serverless compute for both pipelines and jobs. I don't worry about jobs, but only pipelines (i.e. delta live tables) serverless.

Thanks in advance.

Re: How to select performance mode for Databricks Delta Live Tables

lingareddy_Alva — Mon, 28 Apr 2025 16:34:09 GMT

@QuanSun

In Delta Live Tables (DLT):
You can now choose Performance Mode when scheduling a Triggered pipeline.
Two options:
Performance optimized (uses more aggressive scaling, higher resource burst)
Standard performance (normal serverless behavior)
BUT 👉 Performance mode setting only appears when you edit the pipeline schedule, not at cluster settings.

How To Select / Change Performance Mode for a DLT Pipeline
Step-by-Step (Azure Databricks):
1. Go to Workflows → Delta Live Tables Pipelines.
2. Select your existing pipeline or Create a New Pipeline.
3. In the Pipeline Settings Page, scroll to “Trigger settings” or find “Schedule Trigger”.
4. There you will find "Performance optimized" checkbox (✅😞
If you check it → Performance optimized mode (better scaling, faster startup, higher cost)
If you uncheck it → Standard mode (normal scaling)
5. Save the pipeline settings and Deploy.

Continuous Pipelines do not have Performance mode option yet.
Triggered Pipelines must use serverless for this checkbox to appear.
Sometimes in older UI versions (early 2024), the checkbox might not show. Make sure your workspace is updated.
If You Don’t See the Option
Make sure your workspace is updated to DBR 13.3 LTS or later.

Re: How to select performance mode for Databricks Delta Live Tables

QuanSun — Mon, 28 Apr 2025 20:20:53 GMT

Hi @lingareddy_Alva,

Thank you so much for your reply.

I checked my existing pipelines and also tried to create a new databricks workspace (see below). But I cannot find "Trigger settings" or "Schedule Trigger" in pipeline setting. Did I miss anything?

"If You Don’t See the Option
Make sure your workspace is updated to DBR 13.3 LTS or later." Sorry, just confused here: I am using serverless dlt, I cannot configure DBR. Could you please elaborate this?

Thanks.

Re: How to select performance mode for Databricks Delta Live Tables

BF7 — Thu, 02 Oct 2025 16:29:50 GMT

I would like an answer to his question also, I need to see how to turn this off, but any check box relating to performance optimization in my serverless pipeline does not show up.

Re: How to select performance mode for Databricks Delta Live Tables

saurabh18cs — Fri, 03 Oct 2025 13:54:51 GMT

Hi @BF7 @QuanSun first you select serverless and save those setting then you will see following option under job details on top right

Performance-optimized mode
Fast, performant compute startup and execution speed.

Standard mode
Disabling performance optimization gives you startup times similar to Classic infrastructure and reduces your cost.

Currently only applies to certain Serverless workloads.

Re: How to select performance mode for Databricks Delta Live Tables

BF7 — Thu, 23 Oct 2025 19:17:26 GMT

I have learned that this parameter is not governed in the pipeline configuration itself, but in the job task that runs the pipeline. This is confusing to me and I don't like it.

Re: How to select performance mode for Databricks Delta Live Tables

mimimon — Mon, 10 Nov 2025 04:13:34 GMT

May I know if this was automatically on through all DLT tables? How do we monitor timestamp of turning this on and off and the id who did it? Or is automatically configured?