Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
@geetha Venkatesanโ can you explain what sort of certificate you're trying to use? spark-submit works over the Jobs API, which does not require submitting any cert.
ECS is hosted on our private cloud and it uses internal certificate owned by us. So from databricks when we need to connect to ECS, we need to import our internal certificate in databricks, so that connection happens between databricks and ECS .
As per this document, we need to use DButils and we can use this only on notebook. But ours is spark scala application jar that runs in spark submit mode. So we want to know how we can import the certificate and make it use at the startup of the cluster.
I have the same problem... when i worked with the hive_metastore in past, i was able tu use file system and also use API certs.
Now i'm using the unity catalog and i can't upload a certificate, can somebody help me?
Connect with Databricks Users in Your Area
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโt want to miss the chance to attend and share knowledge.
If there isnโt a group near you, start one and help create a community that brings people together.
