Mount Azure Blob Storage with Cluster access control

LukaszJ
Contributor III

‎08-17-2022 10:26 AM

Hello.

I want to mount and share for the one group the container from Azure Blob Storage (It could be simple blob storage or Azure Data Lake Storage gen 2). But I am not able to do it because I am using Cluster with Table Access Control.

This is my code and the error:

storage_name = "***"
container_name = "***"
conf_key = "***"
mount_point = "/mnt/***"
 
dbutils.fs.mount(
    source = f"wasbs://{container_name}@{storage_name}.blob.core.windows.net",
    mount_point = mount_point,
    extra_configs = {f"fs.azure.account.key.{storage_name}.blob.core.windows.net": conf_key}
)
py4j.security.Py4JSecurityException: Constructor public com.databricks.backend.deamon.dbutils.FSUtilsParallel(org.apache.spark.SparkContext) is not whitelisted

How can I mount a container? When I am using standard cluster everything is okay however, then people (on ACL cluster) cannot see the container.

Best regards,

Łukasz

Vivian_Wilfred
Databricks Employee
Databricks Employee

‎08-17-2022 01:32 PM

Hi @Łukasz Jaremek​ , dbutils.fs.* commands wouldn’t work on Table ACL clusters since users do not have direct storage access. This is expected behavior.

0 Kudos

LukaszJ
Contributor III
In response to Vivian_Wilfred

‎08-17-2022 10:51 PM

Dear @Vivian Wilfred​ ,

Thanks you for the answer.

I see. So what can I do when people want to save DataFrames to the Azure Blob Storage container as csv?

Best regards,

Łukasz

0 Kudos

Vidula
Databricks Partner

‎09-11-2022 11:21 PM

Hi @Łukasz Jaremek​ 

Does @Vivian Wilfred​ response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?

We'd love to hear from you.

Thanks!

0 Kudos

LukaszJ
Contributor III

‎11-17-2022 01:05 AM

I have a good solution to the problem:

I am using Python library.

There are some documentation.

Topic to be closed.

Best regards,

Łukasz

View solution in original post

0 Kudos