Unable to load data in DLT tables from Federated data sources

[Databricks-acn]

I tried to run this query and failing to load the data .What do I need to do load from federated data sources using DLT if this is not correct

CREATE OR REPLACE LIVE TABLE bulkuploadhistory
COMMENT 'Table generated for bulkuploadhistory.'
TBLPROPERTIES ('quality' = 'bronze')
AS
SELECT * FROM `master_data_dev`.dbo.bulkuploadhistory

But I get the error below 

org.apache.spark.sql.AnalysisException: Unable to process statement for Table 'bulkuploadhistory'.
org.apache.spark.SparkException: [INTERNAL_ERROR] The Spark SQL phase analysis failed with an internal error. You hit a bug in Spark or the Spark plugins you use. Please, report this bug to the corresponding communities or vendors, and provide the full stack trace.
java.lang.AssertionError: assertion failed: Hive table should not be stored in Unity Catalog

 #DLT

#UC

[Kaniz]

Hi @Databricks-acn, One possible solution could be to set the Spark configuration spark.sql.hive.convertMetastoreOrc to ....

To load data from federated data sources using DLT, you can follow these steps:

• You can load data from any data source supported by Apache Spark™ on Databricks using Delta Live Tables.
• You can define datasets (tables and views) in Delta Live Tables against any query that returns a Spark DataFrame, including streaming DataFrames and Pandas for Spark DataFrames.
• For data ingestion tasks, Databricks recommends using streaming tables for most use cases. Streaming tables are good for ingesting data from cloud object storage using Auto Loader or from message buses like Kafka.
• Not all data sources have SQL support. You can mix SQL and Python notebooks in a Delta Live Tables pipeline to use SQL for all operations beyond ingestion.

[Databricks-acn]

When will be the Unity catalogue will be generally available for production workloads in DLT. I can currently see its in preview while trying to create the pipeline

[Kaniz]

Hi @Databricks-acn, I don’t have a specific date for when the Unity Catalog will be generally available for production workloads in Delta Live Tables (DLT). As you mentioned, it is currently in public preview. 

[Databricks-acn]

Considering the databricks release cycles , is there any tenative dates that I can be aware of so that I can plan for my production releases

[Kaniz]

Thank you for posting your question in our community! We are happy to assist you.

To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?

This will also help other community members who may have similar questions in the future. Thank you for your participation and let us know if you need any further assistance!