SP_6721
Honored Contributor II

Hi @eenaagrawal ,

There isn't a specific built-in integration in Databricks to directly interact with Sharepoint. However, you can accomplish this by leveraging libraries like Office365-REST-Python-Client, which enable interaction with Sharepoint's REST API.
Here are the steps you can follow:

1. Set Up a Service Principal in Azure (if using SharePoint Online)

  • Register an app in Azure Active Directory to obtain a Client ID and Client Secret.
  • Assign the appropriate permissions to the app for accessing your SharePoint resources.

2. Install Required Libraries

  • In your Databricks notebook, install the necessary Python libraries: 
  • Code: %pip install Office365-REST-Python-Client

3. Generate Access Token

  • You will need an access token to interact with SharePoint. Authenticate using the registered app's IDs (from Azure).
  •  Example:
       from office365.runtime.auth.authentication_context import AuthenticationContext
       from office365.sharepoint.client_context import ClientContext
       site_url = " "   client_id = " "   client_secret = " "   context_auth = AuthenticationContext(site_url)   if context_auth.acquire_token_for_app(client_id, client_secret):
           ctx = ClientContext(site_url, context_auth)
           print("Authentication successful")
       else:
           print(context_auth.get_last_error())

4. Read the File in Databricks

  • Read or prepare the file you want to upload to Sharepoint within Databricks. For instance:
       file_path = "/dbfs/path/to/your/file.csv"with open(file_path, "rb") as file:
           file_content = file.read()

5. Upload File to SharePoint

  • Specify the SharePoint document library or folder and upload the file:
       target_folder_url = "/sites/yoursite/Shared Documents/folder_name"   file_name = "file.csv"
       target_folder = ctx.web.get_folder_by_server_relative_url(target_folder_url)   file_upload = target_folder.upload_file(file_name, file_content)
       ctx.execute_query()
       print(f"File {file_name} uploaded successfully.")

6. Verify the Uploaded File

  • After uploading, you can query your SharePoint site to confirm the file exists in the intended location