encryption

[Phani1]

Hi Databricks, Could you please guide me on the below scenario?

Here is the use case we are trying to solve for

1. Currently environment is using “Voltage” as an encryption tool for encrypting the data in S3 in conjunction with business business-provided data encryption & masking catalog.
2. We are looking to replace the above tool, with inherent Databricks capabilities if possible and yet apply 256-bit AES encryption standards.

• Scenario-

1. Is it possible to apply data encryption at S3 bucket where Schema & Catalog is maintained by using Customer_managed Key - need to solve for this.
2. Once encryption is done, we need to then read the data into Databricks DF using custom UDFs, where standard functions like- “aes_encrypt/ aes_decrypt” function could be used along with the data masking functionalities of Unity Catalog.

[DeborahRoe]

Thanks for the clear guidance! Integrating AWS KMS for CMK in Databricks and using custom UDFs for encryption/decryption align perfectly with our goals. Appreciate your assistance!

[AliaCollier]

To replace "Voltage" with Databricks encryption, follow these steps: set up a Customer Managed Key in AWS, configure the S3 bucket, read data in Databricks, and implement custom UDFs for AES encryption/decryption.