topic Re: Constructor public com.databricks.backend.daemon.dbutils.FSUtilsParallel is not whitelisted when mounting a s3 bucket in Data Engineering

Constructor public com.databricks.backend.daemon.dbutils.FSUtilsParallel is not whitelisted when mounting a s3 bucket

Anonymous — Tue, 12 Apr 2022 17:27:19 GMT

Hello all, I'm experiencing this issueConstructor public com.databricks.backend.daemon.dbutils.FSUtilsParallel is not whitelisted when I'm trying to mount a s3 bucket.

%python
dbutils.fs.mount("s3a://dd-databricks-staging-storage/data/staging/datalake/storage", "/mnt/storage")

The cluster is a standard cluster

Can someone explain me what is going on?

Re: Constructor public com.databricks.backend.daemon.dbutils.FSUtilsParallel is not whitelisted when mounting a s3 bucket

Anonymous — Thu, 14 Apr 2022 07:31:43 GMT

10.4 LTS the things evolved a bit.

After investigation, most of these problems are because of the instance profile in use.

I'll update the post with the solution as soon as we get it.

Re: Constructor public com.databricks.backend.daemon.dbutils.FSUtilsParallel is not whitelisted when mounting a s3 bucket

data_boy_2022 — Sat, 10 Sep 2022 05:04:45 GMT

Did you find a solution? I am running into a similar problem.

Re: Constructor public com.databricks.backend.daemon.dbutils.FSUtilsParallel is not whitelisted when mounting a s3 bucket

Anonymous — Sat, 10 Sep 2022 12:18:33 GMT

I'm not on infra team, but it seems that

spark.databricks.pyspark.enablePy4JSecurity false

did the job

Re: Constructor public com.databricks.backend.daemon.dbutils.FSUtilsParallel is not whitelisted when mounting a s3 bucket

leonids2005 — Mon, 03 Oct 2022 17:00:20 GMT

WE have this problem running cluster with 11.2 and shared access mode.

spark.databricks.pyspark.enablePy4JSecurity false - this does not help because it says

spark.databricks.pyspark.enablePy4JSecurity is not allowed when choosing access mode

here is the error we are getting

[17:30] Leonid Sokolovskiy

py4j.security.Py4JSecurityException: Method public com.databricks.backend.daemon.dbutils.DBUtilsCore$Result com.databricks.backend.daemon.dbutils.DBUtilsCore.mount(java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.util.Map) is not whitelisted on class class com.databricks.backend.daemon.dbutils.DBUtilsCore

---------------------------------------------------------------------------

Please help!