topic Re: Databricks cluster Encryption keystore_password in Data Engineering

Databricks cluster Encryption keystore_password

kjoth — Thu, 25 Nov 2021 10:55:23 GMT

How to set up this value? Is this any value we can provide or the default value we have to p

#!/bin/bash
 
keystore_file="/dbfs/<keystore_directory>/jetty_ssl_driver_keystore.jks"
keystore_password="gb1gQqZ9ZIHS"
sasl_secret=$(sha256sum $keystore_file | cut -d' ' -f1)

Re: Databricks cluster Encryption keystore_password

-werners- — Thu, 25 Nov 2021 11:47:23 GMT

I do not exactly understand the question.

Do you want to store secrets like this?

https://docs.databricks.com/security/secrets/index.html

Re: Databricks cluster Encryption keystore_password

Prabakar — Thu, 25 Nov 2021 12:02:28 GMT

Hi @karthick J do not change the password from the script. You need to just modify the <keystore_directory> and create the init script.

In the first step of the notebook, you will copy the Keystore file to a directory in DBFS.

As it is Databricks keystore file you cant change the password.

Re: Databricks cluster Encryption keystore_password

Hubert-Dudek — Thu, 25 Nov 2021 12:05:57 GMT

I think as @Werner Stinckens said you need to provide more details what exactly you want to accomplish and where this script is put.

For all secrets the best way is to use databricks secret or Azure Key Valut (or AWS KMS) so you don't store your passwords in code. If you want to encrypt dbfs is better to make own mount and use ready Azure/S3 encryption options using own keys there.

Re: Databricks cluster Encryption keystore_password

Prabakar — Thu, 25 Nov 2021 12:07:09 GMT

Hi @karthick J please refer to this notebook.

https://docs.microsoft.com/en-us/azure/databricks/_static/notebooks/cluster-encryption-init-script.html

Further, if you will be using %pip magic command the below post will be helpful.

https://community.databricks.com/s/question/0D53f00001VjwmcCAB/pipconda-doesnt-work-with-encrypted-clusters-starting-dbr-9x

Re: Databricks cluster Encryption keystore_password

Prabakar — Thu, 25 Nov 2021 12:11:37 GMT

This is not for secrets but for Encrypting Traffic Between Worker Nodes. 😉

Re: Databricks cluster Encryption keystore_password

kjoth — Thu, 25 Nov 2021 13:23:45 GMT

Yes, I should have provided more detail, as the question is not clear. I will make sure to ask with clear details forward. Like @Prabakar Ammeappin said, the query was on databricks cluster encrypting Traffic bewteen workers nodes.

Re: Databricks cluster Encryption keystore_password

kjoth — Thu, 25 Nov 2021 13:24:12 GMT

Thanks @Prabakar Ammeappin