Transport Data from Sharepoint Excel file to Databrics Table

[Hasiok1337]

Hello 🙂

Is there a way in a Databricks notebook to pull data from an Excel file stored on SharePoint and upload it into my table in Databricks?
I have a situation where I maintain a few tables on SharePoint and a few tables with the same data in Databricks. My goal is to upload data from SharePoint into the Databricks tables.
I was considering something like this: if a file on SharePoint changes → trigger Power Automate → execute a Databricks notebook. I dont have access to blob or graph also.
Is this approach feasible?

[szymon_dybczak]

Hi @Hasiok1337 ,

Your approach is valid in my opinion. But you can also check SharePoint connector. Is in beta currently, but should work. It gives you out of the box way to extract files from SharePoint into databricks 

[Hasiok1337]

Hmm, that's interesting. Could you explain in more detail how it can be done using the SharePoint connector? Or maybe help me with my first idea? You know, I'm asking because for these two approaches I have no idea how it could be done in Databricks/notebooks...

[szymon_dybczak]

Hi @Hasiok1337 ,

Sure, but I think the best way is to follow the guide at databricks docs. It's really well described. As a first step you need to configure authentication to sharepoint using OAuth U2M (which is recommended). Detailed guide how to do this you can find here;

Configure OAuth U2M for Microsoft SharePoint ingestion | Databricks Documentation

Once you will have your authentication part done, then you need to create microsoft sharepoint ingestion pipeline. Again, this is really well described at docs, so just try to follow along below tutorial (they even provide example notebook):

Create a Microsoft SharePoint ingestion pipeline | Databricks Documentation