Connect Purview to AWS Databricks Unity Catalog?

[ShaSap]

Hello!  Newbie here, so apologies if this is a super basic question.  Trying to figure out if we can connect Purview to an AWS instance of Databricks (vice Azure instance), but I have only seen articles on connecting Azure Databricks to Purview.  I reached out to a Purview support/questions email address a week+ ago, but haven't heard back from them so thought I'd ask here.

Background: We're currently using UC for data governance, but connecting to Purview would allow us to: 1) Have the data lineage extend out to Power BI, and 2) be able to store more lengthy data descriptions (sometimes our data has nuances that require lengthier descriptions than what makes sense to put in the column comments of Databricks' Catalog).

Thanks for your time!

[-werners-]

Interesting question.  And frankly I have no idea if it would extend to AWS. My first guess is: no.
It might be possible with some kind of custom connector, but chances are this will not deliver the same experience.
So you might have to wait on the reply from MS on this.  I am curious for one.

[ShaSap]

Btw, we never heard back from MS on my emailed question, but I was able to speak to a customer support individual by calling their support line (after several different attempts... man they really designed that thing to deflect human interaction).  The feedback/instructions she gave us were not instructions that actually connected UC to Purview, rather they were more generic instructions on connecting an AWS db to Purview.  We ended up deciding that there likely was no built-in way to do this and will simply work with what UC has.

[malinadiego123]

---

@ShaSap wrote:

Hello!  Newbie here, so apologies if this is a super basic question.  Trying to figure out if we can connect Purview to an AWS instance of Databricks (vice Azure instance), but I have only seen articles on connecting Azure Databricks to Purview.  I reached out to a Purview support/questions email address a week+ ago, but haven't heard back from them so thought I'd ask here.

Background: We're currently using UC for data governance, but connecting to Purview would allow us to: 1) Have the data lineage extend out to Power BI, and 2) be able to store more lengthy data descriptions (sometimes our data has nuances that require lengthier descriptions than what makes sense to put in the column comments of Databricks' Catalog).

Thanks for your time!

---

¡Hola! ¡Bienvenido a la comunidad! Tu pregunta es muy válida. Actualmente, Azure Purview admite principalmente Azure Databricks, y no hay soporte oficial para conectarlo directamente a Databricks en AWS. Sin embargo, podrías considerar el uso de APIs o soluciones de integración personalizadas para transferir metadatos desde tu Databricks en AWS a Purview, aunque esto podría requerir un poco de configuración adicional.

Para tus necesidades de linaje de datos y descripciones más extensas, podrías explorar otras herramientas de gobernanza que tengan un soporte más amplio para múltiples plataformas. También sería bueno estar atento a las actualizaciones de Azure, ya que frecuentemente amplían sus servicios.

Si aún no has recibido respuesta del equipo de soporte de Purview, podría ser útil hacer un seguimiento o revisar su documentación y foros comunitarios para cualquier novedad. ¡Buena suerte y no dudes en preguntar si tienes más preguntas!

[malinadiego123]

¡Hola! Gracias por tu respuesta y por la información. Es útil saber que Azure Purview tiene una integración principal con Azure Databricks en consejos android, aunque la falta de soporte directo para Databricks en AWS es un punto a considerar. La idea de usar APIs o soluciones de integración personalizadas es interesante, aunque entiendo que puede requerir un esfuerzo adicional.

Explorar otras herramientas de gobernanza con mejor soporte para múltiples plataformas también parece una buena estrategia, especialmente para linaje de datos. Definitivamente estaré atento a las actualizaciones de Azure, ya que siempre están mejorando sus servicios.

Si no recibí respuesta del equipo de soporte, seguiré el consejo de hacer un seguimiento y revisar la documentación y los foros. Aprecio tu ayuda y seguiré preguntando si tengo más dudas. ¡Gracias de nuevo!

---

@ShaSap wrote:

Hello!  Newbie here, so apologies if this is a super basic question.  Trying to figure out if we can connect Purview to an AWS instance of Databricks (vice Azure instance), but I have only seen articles on connecting Azure Databricks to Purview.  I reached out to a Purview support/questions email address a week+ ago, but haven't heard back from them so thought I'd ask here.

Background: We're currently using UC for data governance, but connecting to Purview would allow us to: 1) Have the data lineage extend out to Power BI, and 2) be able to store more lengthy data descriptions (sometimes our data has nuances that require lengthier descriptions than what makes sense to put in the column comments of Databricks' Catalog).

Thanks for your time!

---