Mount Azure Blob Storage with Cluster access control

LukaszJ
Contributor III

Hello.

I want to mount and share for the one group the container from Azure Blob Storage (It could be simple blob storage or Azure Data Lake Storage gen 2). But I am not able to do it because I am using Cluster with Table Access Control.

This is my code and the error:

storage_name = "***"
container_name = "***"
conf_key = "***"
mount_point = "/mnt/***"
 
dbutils.fs.mount(
    source = f"wasbs://{container_name}@{storage_name}.blob.core.windows.net",
    mount_point = mount_point,
    extra_configs = {f"fs.azure.account.key.{storage_name}.blob.core.windows.net": conf_key}
)
py4j.security.Py4JSecurityException: Constructor public com.databricks.backend.deamon.dbutils.FSUtilsParallel(org.apache.spark.SparkContext) is not whitelisted

How can I mount a container? When I am using standard cluster everything is okay however, then people (on ACL cluster) cannot see the container.

Best regards,

Łukasz