emma_s
Databricks Employee
Databricks Employee

HI,

You could possibly achieve something near to this using the lakeflow connect Sharepoint connector. It's currently in beta so it would need to be enabled in your workspace. Although it isn't triggered on file updates, because it only ingests incrementally, new or changed files, then you could schedule it to run hourly. If nothing had changed, then it would use minimal consumption. The docs for this are here: https://docs.databricks.com/aws/en/ingestion/lakeflow-connect/sharepoint

Thanks,

Emma