04-17-2024 01:04 AM
Hi there,
Im trying to run DE 2.1 - Querying Files Directly on my workspace with a default cluster configuration for found below,
but I cannot seem to run this file (or any other labs) as it gives me this error message
04-17-2024 03:25 AM
Hi @Brammer88,
spark.databricks.pyspark.enablePy4JSecurity
to false
. However, this option is not recommended due to security implications.spark.jvm.class.allowlist
configuration property in your Spark configuration.org.apache.spark.sql.internal.CatalogImpl.clearCache
to the allowlist.04-18-2024 03:36 AM
Hi Kaniz,
Im not sure why, but it seems like its not working after all and I was cheering to soon.
I have added a line
spark.jvm.class.allowlist org.apache.spark.sql.internal.CatalogImpl.clearCache
to my cluster in the spark config (under advanced options), but i still get the same response;
04-22-2024 12:25 AM
Hi @Kaniz_Fatma,
Do you have any other solution/idea as to what might cause the issue? Or do you need any more details for investigation? Thanks again for your help! I am completely stuck as I am unable to do any work with the labs in this way.
Best,
Bram
04-17-2024 11:41 PM
works, thanks for the quick response!
04-22-2024 01:43 AM
Hi @Kaniz,
Sorry, I was too fast with this reply, im not sure why, but it seems like its not working after all and I was cheering to soon.
I have added a line
spark.jvm.class.allowlist org.apache.spark.sql.internal.CatalogImpl.clearCache
to my cluster in the spark config (under advanced options), but i still get the same response;
Do you have any other solution/idea as to what might cause the issue? Or do you need any more details for investigation? Thanks again for your help! I am completely stuck as I am unable to do any work with the labs in this way.
Best,
Bram
04-30-2024 01:34 PM
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group