Constructor public com.databricks.backend.daemon.dbutils.FSUtilsParallel is not whitelisted when mounting a s3 bucket

[Anonymous]

Hello all, I'm experiencing this issueConstructor public com.databricks.backend.daemon.dbutils.FSUtilsParallel is not whitelisted when I'm trying to mount a s3 bucket.

%python
dbutils.fs.mount("s3a://dd-databricks-staging-storage/data/staging/datalake/storage", "/mnt/storage")

The cluster is a standard cluster

Can someone explain me what is going on?

[Anonymous]

10.4 LTS the things evolved a bit.

After investigation, most of these problems are because of the instance profile in use.

I'll update the post with the solution as soon as we get it.

[data_boy_2022]

Did you find a solution? I am running into a similar problem.

[Anonymous]

I'm not on infra team, but it seems that

 spark.databricks.pyspark.enablePy4JSecurity false

did the job

[leonids2005]

WE have this problem running cluster with 11.2 and shared access mode.

 spark.databricks.pyspark.enablePy4JSecurity false - this does not help because it says

 spark.databricks.pyspark.enablePy4JSecurity is not allowed when choosing access mode

here is the error we are getting

[17:30] Leonid Sokolovskiy

py4j.security.Py4JSecurityException: Method public com.databricks.backend.daemon.dbutils.DBUtilsCore$Result com.databricks.backend.daemon.dbutils.DBUtilsCore.mount(java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.util.Map) is not whitelisted on class class com.databricks.backend.daemon.dbutils.DBUtilsCore

---------------------------------------------------------------------------

Please help!