cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Governance
Join discussions on data governance practices, compliance, and security within the Databricks Community. Exchange strategies and insights to ensure data integrity and regulatory compliance.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

"spark.databricks.pyspark.enablePy4JSecurity is not allowed when choosing an access mode"

Christine
Contributor II

Due to some problems, I need to set Py4JSecurity to false, however I get the following message:

"spark.databricks.pyspark.enablePy4JSecurity is not allowed when choosing an access mode"

My cluster is in Shared access mode and a runtime version 12.1 (includes Apache Spark 3.3.1, Scala 2.12) with Unity catalog, Worker type Standard_DS4_v2, and driver type Standard_DS3_v2.

I need it to be a shared cluster that uses unity catalog, so is there any way to get around this?

Thanks!

4 REPLIES 4

Debayan
Esteemed Contributor III

Hi, Could you please test it on a standard standalone cluster/single user cluster and confirm? The error is occurring because enabling the security may break user isolation.

Please let us know if this helps. 

Also please tag @Debayanโ€‹ with your next response which will notify me, Thank you!

Hi @Debayan Mukherjeeโ€‹,

It works in a single user cluster, but I was wondering if it is possible to use it on a shared cluster?

Debayan
Esteemed Contributor III

Hi @Christine Pedersenโ€‹ , It is not possible to use it in a shared cluster as of now. Thank you!

@Debayan Mukherjeeโ€‹ Thank you for the clarification! I will continue with single-user cluster.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group