2 weeks ago
Hi there,
Im trying to run DE 2.1 - Querying Files Directly on my workspace with a default cluster configuration for found below,
but I cannot seem to run this file (or any other labs) as it gives me this error message
2 weeks ago
Hi @Brammer88,
spark.databricks.pyspark.enablePy4JSecurity
to false
. However, this option is not recommended due to security implications.spark.jvm.class.allowlist
configuration property in your Spark configuration.org.apache.spark.sql.internal.CatalogImpl.clearCache
to the allowlist.2 weeks ago
Hi Kaniz,
Im not sure why, but it seems like its not working after all and I was cheering to soon.
I have added a line
spark.jvm.class.allowlist org.apache.spark.sql.internal.CatalogImpl.clearCache
to my cluster in the spark config (under advanced options), but i still get the same response;
a week ago
Hi @Kaniz,
Do you have any other solution/idea as to what might cause the issue? Or do you need any more details for investigation? Thanks again for your help! I am completely stuck as I am unable to do any work with the labs in this way.
Best,
Bram
2 weeks ago
works, thanks for the quick response!
a week ago
Hi @Kaniz,
Sorry, I was too fast with this reply, im not sure why, but it seems like its not working after all and I was cheering to soon.
I have added a line
spark.jvm.class.allowlist org.apache.spark.sql.internal.CatalogImpl.clearCache
to my cluster in the spark config (under advanced options), but i still get the same response;
Do you have any other solution/idea as to what might cause the issue? Or do you need any more details for investigation? Thanks again for your help! I am completely stuck as I am unable to do any work with the labs in this way.
Best,
Bram
6 hours ago
Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections.
Click here to register and join today!
Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.