- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-21-2023 08:30 PM
Hi @BobBubble2000
The MSFT Azure document briefly mentioned that Spark Common Data Model connector natively supports Azure Synapse Analytics. On the other hand, Azure Databricks as a data consumer to read data from the Common Data Model folders in Data Lake Storage Gen2, this should be achievable as well. You can firstly look for a way to install Spark CDM connector on Azure Databricks, and then you can read CMD data into a dataframe. If you can achieve this step, you can also import dlt module as well as the Spark CMD connector in your DLT notebook and turn the CMD data into a live table in your DLT pipeline.
Reference:
https://github.com/Azure/spark-cdm-connector/blob/spark3.1/documentation/overview.md
https://learn.microsoft.com/en-us/common-data-model/data-lake