Cannot develop Delta Live Tables using Runtime 14 or 15.

[oakhill]

When trying to develop a Delta Live Table-pipeline with my very generic clusters (runtime 14.3 or 15.4 LTS), I get th following error:

The Delta Live Tables (DLT) module is not supported on this cluster. You should either create a new pipeline or use an existing pipeline to run DLT code. 

My completely identical cluster using runtime 13.3 can import DLTs. But on Runtime 13.3 I can't improt helper modules when using Git-folders

Databricks Runtime version upgrade required
Please upgrade DBR version to 14.3 or above to execute commands on notebooks inside Git folders.

I cannot find anything in the documentation to provide me with assistance on how to actually solve these problems. Any help is appreciated.

[Brahmareddy]

Hi @oakhill,

How are you doing today?

Give a try with below suggestion and let me know if it works.

To fix the DLT error, use a Managed Pipeline instead of your generic cluster, as DLT isn't supported on Runtime 14.3 or 15.4. If you need Git folder support, use Runtime 14.3+ on a separate cluster. For DLT, stick with Runtime 13.3, but handle helper modules outside Git folders. Reach out to Databricks support for more guidance if needed.

Regards,

Brahma

[oakhill]

Hi @Brahmareddy, thank you for your response. Unfortunately, attaching the notebook to a managed pipeline will cause every notebook cell to trigger a validation-update. Does not work as a developer experience at all.

Hoping someone at Databricks can chime in.