Delta Live Tables with Common Data Model as source

[BobBubble2000]

Hi,

I'm investigating whether it's possible to use Common Data Model CDM (in particular the Dynamics 365 exported csv and cdm files) as a Delta Live Tables data source? Can someone point me in the right direction?

Thanks!

[JunYang]

Hi @BobBubble2000 
The MSFT Azure document briefly mentioned that Spark Common Data Model connector natively supports Azure Synapse Analytics. On the other hand, Azure Databricks as a data consumer to read data from the Common Data Model folders in Data Lake Storage Gen2, this should be achievable as well. You can firstly look for a way to install Spark CDM connector on Azure Databricks, and then you can read CMD data into a dataframe. If you can achieve this step, you can also import dlt module as well as the Spark CMD connector in your DLT notebook and turn the CMD data into a live table in your DLT pipeline. 

Reference:

https://github.com/Azure/spark-cdm-connector/blob/spark3.1/documentation/overview.md

https://learn.microsoft.com/en-us/common-data-model/data-lake

[developer_bi]

hello, I am  currently looking the same task. Have you been successful in installing the connector in databricks?