08-17-2022 10:26 AM
Hello.
I want to mount and share for the one group the container from Azure Blob Storage (It could be simple blob storage or Azure Data Lake Storage gen 2). But I am not able to do it because I am using Cluster with Table Access Control.
This is my code and the error:
storage_name = "***"
container_name = "***"
conf_key = "***"
mount_point = "/mnt/***"
dbutils.fs.mount(
source = f"wasbs://{container_name}@{storage_name}.blob.core.windows.net",
mount_point = mount_point,
extra_configs = {f"fs.azure.account.key.{storage_name}.blob.core.windows.net": conf_key}
)
py4j.security.Py4JSecurityException: Constructor public com.databricks.backend.deamon.dbutils.FSUtilsParallel(org.apache.spark.SparkContext) is not whitelisted
How can I mount a container? When I am using standard cluster everything is okay however, then people (on ACL cluster) cannot see the container.
Best regards,
Łukasz
11-17-2022 01:05 AM
I have a good solution to the problem:
I am using Python library.
There are some documentation.
Topic to be closed.
Best regards,
Łukasz
08-17-2022 01:32 PM
Hi @Łukasz Jaremek , dbutils.fs.* commands wouldn’t work on Table ACL clusters since users do not have direct storage access. This is expected behavior.
08-17-2022 10:51 PM
Dear @Vivian Wilfred ,
Thanks you for the answer.
I see. So what can I do when people want to save DataFrames to the Azure Blob Storage container as csv?
Best regards,
Łukasz
08-17-2022 01:35 PM
09-11-2022 11:21 PM
Hi @Łukasz Jaremek
Does @Vivian Wilfred response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?
We'd love to hear from you.
Thanks!
11-17-2022 01:05 AM
I have a good solution to the problem:
I am using Python library.
There are some documentation.
Topic to be closed.
Best regards,
Łukasz
Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections.
Click here to register and join today!
Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.