cancel
Showing results for 
Search instead for 
Did you mean: 
Data Governance
Join discussions on data governance practices, compliance, and security within the Databricks Community. Exchange strategies and insights to ensure data integrity and regulatory compliance.
cancel
Showing results for 
Search instead for 
Did you mean: 

"spark.databricks.pyspark.enablePy4JSecurity is not allowed when choosing an access mode"

Christine
Contributor II

Due to some problems, I need to set Py4JSecurity to false, however I get the following message:

"spark.databricks.pyspark.enablePy4JSecurity is not allowed when choosing an access mode"

My cluster is in Shared access mode and a runtime version 12.1 (includes Apache Spark 3.3.1, Scala 2.12) with Unity catalog, Worker type Standard_DS4_v2, and driver type Standard_DS3_v2.

I need it to be a shared cluster that uses unity catalog, so is there any way to get around this?

Thanks!

4 REPLIES 4

Debayan
Esteemed Contributor III
Esteemed Contributor III

Hi, Could you please test it on a standard standalone cluster/single user cluster and confirm? The error is occurring because enabling the security may break user isolation.

Please let us know if this helps. 

Also please tag @Debayan​ with your next response which will notify me, Thank you!

Hi @Debayan Mukherjee​,

It works in a single user cluster, but I was wondering if it is possible to use it on a shared cluster?

Debayan
Esteemed Contributor III
Esteemed Contributor III

Hi @Christine Pedersen​ , It is not possible to use it in a shared cluster as of now. Thank you!

@Debayan Mukherjee​ Thank you for the clarification! I will continue with single-user cluster.

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!