topic Re: How can I configure S3 Client-Side Encryption (CSE-KMS ) for my data pipeline in Data Engineering

How can I configure S3 Client-Side Encryption (CSE-KMS ) for my data pipeline

sajith_appukutt — Wed, 09 Jun 2021 07:36:37 GMT

Re: How can I configure S3 Client-Side Encryption (CSE-KMS ) for my data pipeline

sajith_appukutt — Fri, 18 Jun 2021 00:37:28 GMT

You could create a custom key provider that implements the EncryptionMaterialsProvider interface and configure the databricks mount to use CSE

dbutils.fs.mount(
  "s3a://cse-bucket",
  "/mnt/cse-data", 
  extraConfigs = Map(
    "fs.s3.cse.enabled" -> "true",
    "fs.s3.cse.encryptionMaterialsProvider" -> "com.mynamespace. MyEncryptionMaterialsProviders",
    "kms.key.id" -> "xxx"
  )
)

Re: How can I configure S3 Client-Side Encryption (CSE-KMS ) for my data pipeline

AdrianRojas — Thu, 13 Apr 2023 23:35:26 GMT

a bit old, but I just faced the same issue, specifying a custom EncryptionMaterialsProvider (as described in the previous post) did the trick for me but I did had to also specify my kms endpoint, just because my region:

"fs.s3.cse.kms.endpoint" -> "kms.<region>.amazonaws.com"