Scaling Delta Sharing across Non-Databricks Recipients: Handling Credential Rotation and Lifecycle

Data Engineering

Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.

Hi everyone,

Hi everyone, I’m architecting a data-sharing program using Delta Sharing for external partners who do not have a Databricks workspace.

How are others automating the recipient credential lifecycle? Do you have an automated process for rotating these credentials, or is there a standard pattern for revoking access without manual intervention?
For high-scale sharing (100+ recipients), are you maintaining these via Terraform/DABs, or is there a "request-based" workflow you’ve built?
Have you encountered significant performance differences between partners using Spark vs. those using native Python/Pandas connectors?

Data Architect | 13 Years Domain Expertise | Databricks SA Champion Cohort