- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-09-2021 12:36 AM
Accepted Solutions
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-17-2021 05:37 PM
You could create a custom key provider that implements the EncryptionMaterialsProvider interface and configure the databricks mount to use CSE
dbutils.fs.mount(
"s3a://cse-bucket",
"/mnt/cse-data",
extraConfigs = Map(
"fs.s3.cse.enabled" -> "true",
"fs.s3.cse.encryptionMaterialsProvider" -> "com.mynamespace. MyEncryptionMaterialsProviders",
"kms.key.id" -> "xxx"
)
)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-17-2021 05:37 PM
You could create a custom key provider that implements the EncryptionMaterialsProvider interface and configure the databricks mount to use CSE
dbutils.fs.mount(
"s3a://cse-bucket",
"/mnt/cse-data",
extraConfigs = Map(
"fs.s3.cse.enabled" -> "true",
"fs.s3.cse.encryptionMaterialsProvider" -> "com.mynamespace. MyEncryptionMaterialsProviders",
"kms.key.id" -> "xxx"
)
)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-13-2023 04:35 PM
a bit old, but I just faced the same issue, specifying a custom EncryptionMaterialsProvider (as described in the previous post) did the trick for me but I did had to also specify my kms endpoint, just because my region:
"fs.s3.cse.kms.endpoint" -> "kms.<region>.amazonaws.com"

