cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Governance
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
Christine
by • Contributor
  • 31748 Views
  • 12 replies
  • 15 kudos

Resolved! Cannot use RDD and cannot set "spark.databricks.pyspark.enablePy4JSecurity false" for cluster

I have been using "rdd.flatMap(lambda x:x)" for a while to create lists from columns however after I have changed the cluster to a Shared acess mode (to use unity catalog) I get the following error: py4j.security.Py4JSecurityException: Method public ...

Data Governance
Reply
  • 31748 Views
  • 12 replies
  • 15 kudos
View Replies
Latest Reply
KandyKad
New Contributor II
  • 15 kudos

Faced this issue multiple times.Solution:1. Don't use Shared Cluster or cluster without Unity Catalog enabled for running 'rdd' queries on Databricks.2. Instead create a Personal Cluster (Single User) with basic configuration and with Unity Catalog e...

  • 15 kudos
11 More Replies
data_boy_2022
by • New Contributor III
  • 2074 Views
  • 2 replies
  • 3 kudos

Resolved! Unity Catalog + Streaming error: method public X is not whitelisted on class DataStreamReader

I want to import data using the autoloader from a S3 bucket into a table which is managed inside a Unity Catalog.Right now, I run the code on an interactive cluster inside a notebook. In the future the code should run in a job cluster. The error I ge...

Screenshot 2022-09-09 at 6.15.00 PM
Data Governance
Reply
  • 2074 Views
  • 2 replies
  • 3 kudos
View Replies
Latest Reply
data_boy_2022
New Contributor III
  • 3 kudos

Works with single user mode. Thank you!

  • 3 kudos
1 More Replies
Labels