cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Mount Azure Blob Storage with Cluster access control

LukaszJ
Contributor III

Hello.

I want to mount and share for the one group the container from Azure Blob Storage (It could be simple blob storage or Azure Data Lake Storage gen 2). But I am not able to do it because I am using Cluster with Table Access Control.

This is my code and the error:

storage_name = "***"
container_name = "***"
conf_key = "***"
mount_point = "/mnt/***"
 
dbutils.fs.mount(
    source = f"wasbs://{container_name}@{storage_name}.blob.core.windows.net",
    mount_point = mount_point,
    extra_configs = {f"fs.azure.account.key.{storage_name}.blob.core.windows.net": conf_key}
)
py4j.security.Py4JSecurityException: Constructor public com.databricks.backend.deamon.dbutils.FSUtilsParallel(org.apache.spark.SparkContext) is not whitelisted

How can I mount a container? When I am using standard cluster everything is okay however, then people (on ACL cluster) cannot see the container.

Best regards,

Łukasz

1 ACCEPTED SOLUTION

Accepted Solutions

LukaszJ
Contributor III

I have a good solution to the problem:

I am using Python library.

There are some documentation.

Topic to be closed.

Best regards,

Łukasz

View solution in original post

5 REPLIES 5

Vivian_Wilfred
Honored Contributor
Honored Contributor

Hi @Łukasz Jaremek​ , dbutils.fs.* commands wouldn’t work on Table ACL clusters since users do not have direct storage access. This is expected behavior.

Dear @Vivian Wilfred​ ,

Thanks you for the answer.

I see. So what can I do when people want to save DataFrames to the Azure Blob Storage container as csv?

Best regards,

Łukasz

Vidula
Honored Contributor

Hi @Łukasz Jaremek​ 

Does @Vivian Wilfred​ response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?

We'd love to hear from you.

Thanks!

LukaszJ
Contributor III

I have a good solution to the problem:

I am using Python library.

There are some documentation.

Topic to be closed.

Best regards,

Łukasz