topic DLT with Unity Catalog: Multipart table name in Data Engineering

DLT with Unity Catalog: Multipart table name

Pierre1 — Thu, 01 Aug 2024 14:04:17 GMT

Hello,

I can't seem to find up to date info on how to handle catalog.schema.table in DLT live table and Unity catalog.

My statement is the following and is failing with the error: Multipart table name is not supported. Any workaround possible?

Thanks a lot Pierre

create or replace streaming live table `prcatalog`.`dlt-bronze`.raw_traffic_dl2 TBLPROPERTIES( 'quality' = 'bronze') as select * from cloud_files( 'abfss://pr-dlt-landing@fsdlapidpoc.dfs.core.windows.net/raw_traffic', 'csv' );

Re: DLT with Unity Catalog: Multipart table name

szymon_dybczak — Fri, 02 Aug 2024 11:46:58 GMT

Hi @Pierre1 ,

Actually, you don't provide this infromation in the code. You specify this information when you create dlt pipeline.

If you do not select a catalog and target schema for a pipeline, tables are not published to Unity Catalog and can only be accessed by queries in the same pipeline.

For more information take a look on below documentation entry:

Use Unity Catalog with your Delta Live Tables pipelines - Azure Databricks | Microsoft Learn