encryption

Phani1
Databricks MVP

Hi Databricks, Could you please guide me on the below scenario?

Here is the use case we are trying to solve for

  1. Currently environment is using “Voltage” as an encryption tool for encrypting the data in S3 in conjunction with business business-provided data encryption & masking catalog.
  2. We are looking to replace the above tool, with inherent Databricks capabilities if possible and yet apply 256-bit AES encryption standards.
  • Scenario-
    1. Is it possible to apply data encryption at S3 bucket where Schema & Catalog is maintained by using Customer_managed Key - need to solve for this.
    2. Once encryption is done, we need to then read the data into Databricks DF using custom UDFs, where standard functions like- “aes_encrypt/ aes_decrypt” function could be used along with the data masking functionalities of Unity Catalog.