How do we add a certificate file in Databricks for sparksubmit type of job?

[geetha_venkates]

How do we add a certificate file in Databricks for sparksubmit type of job? 

[Anonymous]

Hello, @geetha Venkatesan​! Welcome and thank you for asking. Let's give the community a bit longer to answer. We'll circle back if we need to.

[BilalAslamDbrx]

@geetha Venkatesan​ can you explain what sort of certificate you're trying to use? spark-submit works over the Jobs API, which does not require submitting any cert.

[geetha_venkates]

ECS is hosted on our private cloud and it uses internal certificate owned by us. So from databricks when we need to connect to ECS, we need to import our internal certificate in databricks, so that connection happens between databricks and ECS .

[franco_patano · Accepted solution]

Here are directions on how to add a custom CA Cert. Does this help?

https://kb.databricks.com/python/import-custom-ca-cert.html

[geetha_venkates]

As per this document, we need to use DButils and we can use this only on notebook. But ours is spark scala application jar that runs in spark submit mode. So we want to know how we can import the certificate and make it use at the startup of the cluster.

[DBLearner74]

Since DBFS based files getting deprecated, is there a newer method to this?

[nicozambelli]

I have the same problem... when i worked with the hive_metastore in past, i was able tu use file system and also use API certs.

Now i'm using the unity catalog and i can't upload a certificate, can somebody help me?