topic Re: External KMS integration with Databricks like AWS KMS, Azure Key Vault. in Data Engineering

External KMS integration with Databricks like AWS KMS, Azure Key Vault.

AnubhavG — Fri, 18 Nov 2022 10:18:47 GMT

I would like to know how we can integrate Databricks with External KMS providers, like currently it is doing with AWS KMS and Azure Key Valut?

Can we import keys from any other KMS?

Re: External KMS integration with Databricks like AWS KMS, Azure Key Vault.

Vivian_Wilfred — Fri, 18 Nov 2022 15:51:36 GMT

Hi @Anubhav Gupta , you might have to import your external KMS key to AWS KMS or AKV and then integrate it with Databricks.

Re: External KMS integration with Databricks like AWS KMS, Azure Key Vault.

AnubhavG — Fri, 18 Nov 2022 15:58:06 GMT

Thanks @Vivian Wilfred for the reply.

But I actually wanted to know, How can external KMS providers integrate with Databricks directly like AWS and AKV.

Re: External KMS integration with Databricks like AWS KMS, Azure Key Vault.

Vivian_Wilfred — Fri, 18 Nov 2022 16:26:13 GMT

@Anubhav Gupta Databricks is hosted on the cloud provider which means that all resources used by databricks in the backend are in the cloud. For instance, if you create a cluster, the VMs are launched in AWS as EC2 instances. So the integration of KMS has to be done on the cloud where the resources are present.

Does this clarify your question?

Re: External KMS integration with Databricks like AWS KMS, Azure Key Vault.

AnubhavG — Fri, 18 Nov 2022 16:56:01 GMT

@Vivian Wilfred Alright I got you.

So, you mean to say we cannot run Databricks on its own or without having any cloud provider support such as AWS and AKV?

Re: External KMS integration with Databricks like AWS KMS, Azure Key Vault.

Vivian_Wilfred — Fri, 18 Nov 2022 17:10:00 GMT

Yes, that's correct.

Please select the comment as 'best answer' to resolve the query. Thanks!