Thank you @SergioSchena It helps, I will reach out to the partner team.We ran into a lot of issue while building the similar tool for a customer some time back. The issue we ran into was with columns changing as per the evolution of the data. Year wi...
Yes, thats the limitation. I checked through the documentation and its not mentioned anywhere strictly.here is the details on how to connect : Connect to Azure Data Lake Storage and Blob Storage - Azure Databricks | Microsoft Learn
The CSV side of the solutions has a lot of issues if there are changes in the source system. The CSV files will not have the headers. You have to create them from the JSON files. ( Like CDM format here : https://github.com/microsoft/CDM/blob/master/d...
