How to enable Delta live tables serverless in Databricks?

[nggianno]

I am trying to enable the Serverless mode in the Delta Live Tables, based on what the official Databricks channel YouTube video "Delta Live Tables A to Z: Best practices for Modern Data Pipelines".And I cannot find it in my UI. Could you help me with that? How is it possible to enable?

[Kayla]

Are you on GCP by chance? If so, I don't think serverless options are available yet.

[nggianno]

No, I am in AWS. 

[Kaniz]

Hi @nggianno, Enabling the Serverless mode in Delta Live Tables involves configuring pipeline settings. Let’s walk through the steps:

1. Choose a Product Edition:

   • Delta Live Tables offers different product editions:
     • Core: For streaming ingest workloads without advanced features like change data capture (CDC) or expectations.
     • Pro: Supports Core features and allows updating tables based on changes in source data.
     • Advanced: Supports Core, Pro features, and enforces data quality constraints with expectations.
   • You can select the product edition when creating or editing a pipeline¹.

2. Choose a Pipeline Mode:

   • You can update your pipeline continuously or with manual triggers based on the pipeline mode.
   • To create a pipeline:
     • Click Workflows in the sidebar.
     • Go to the Delta Live Tables tab.
     • Click Create Pipeline.
     • Provide a name for the pipeline.
     • Select Triggered for the pipeline mode.

3. Serverless Mode:

   • To enable serverless pipelines, follow these steps:
     • Click Delta Live Tables in the sidebar.
     • Click Create Pipeline.
     • Give the pipeline a name.
     • Optionally, select the Serverless checkbox to use fully managed compute for this pipeline.
   • Note that you can still pass configuration parameters to a serverless pipeline, but any parameter se...⁴.

Remember to consult the official documentation for detailed information on configuring pipeline sett...¹.

[nggianno]

The problem is that the "Serverless" checkbox does not appear in my UI Pipeline Settings. So, I do not know how to enable serverless given your instructions. Can you tell me why the button is not displayed or how to display it or how to enable DLT serverless with another way?