โ02-16-2022 09:59 PM
The error is as below. The error is intermittent. eg. - The same code throws the below issue for run 3 but doesn't throws issue for run 4. Then again throws issue for run 5.
An error occurred while calling o1509.getCause. Trace:
py4j.security.Py4JSecurityException: Method public synchronized java.lang.Throwable org.datanucleus.exceptions.NucleusException.getCause() is not whitelisted on class class org.datanucleus.exceptions.NucleusDataStoreException
at py4j.security.WhitelistingPy4JSecurityManager.checkCall(WhitelistingPy4JSecurityManager.java:473)
at py4j.Gateway.invoke(Gateway.java:294)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:251)
at java.lang.Thread.run(Thread.java:748)
ErrorMessage written to ADLS Successfully..
org.apache.spark.sql.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to alter table.
โ03-14-2022 07:58 AM
Hello @Darshana Ganeshโ
Are you setting "spark.databricks.pyspark.enablePy4JSecurity": "true" config on Table ACL enabled clusters?
โ02-16-2022 10:07 PM
On executing the below code in azure databricks notebook:
print (spark.version)
for 8.3 (includes Apache Spark 3.1.1, Scala 2.12) , I am getting the output as 3.1.0
and that for 9.1 LTS (includes Apache Spark 3.1.2, Scala 2.12), I am getting 3.1.2
โ03-09-2022 07:06 AM
Hi @Darshana Ganeshโ , Can you please share the code stack here?
โ02-17-2022 08:42 AM
Hello, @Darshana Ganeshโ - My name is Piper, and I'm a moderator for Databricks. Thank you for your question and the extra information. We'll give the community some time to answer before we circle back if we need to.
โ03-14-2022 07:58 AM
Hello @Darshana Ganeshโ
Are you setting "spark.databricks.pyspark.enablePy4JSecurity": "true" config on Table ACL enabled clusters?
โ04-27-2022 08:46 AM
Hey @Darshana Ganeshโ
Just wanted to check in if you were able to resolve your issue or do you need more help? We'd love to hear from you.
Thanks!
Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections.
Click here to register and join today!
Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.