cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
DataRabbit
by • New Contributor II
  • 13601 Views
  • 3 replies
  • 0 kudos

Resolved! py4j.security.Py4JSecurityException: Constructor public org.apache.spark.ml.feature.VectorAssembler(java.lang.String) is not whitelisted.

Hello, i have a problem.When I try to run the MLlib Assembler (from pyspark.ml.feature import VectorAssembler) I get this error and I don't know what to do anymore. Please help.

Data Engineering
Reply
  • 13601 Views
  • 3 replies
  • 0 kudos
View Replies
Latest Reply
DBXC
Contributor
  • 0 kudos

How could we whitelist this error below with DBR 13.3 and above? Py4JError: An error occurred while calling None.org.apache.spark.ml.recommendation.ALS. Trace: py4j.security.Py4JSecurityException: Constructor public org.apache.spark.ml.recommendation...

  • 0 kudos
2 More Replies
Mikki007
by • New Contributor II
  • 5934 Views
  • 2 replies
  • 1 kudos

Resolved! Constructor public org.apache.spark.SparkConf(boolean) is not whitelisted.

My code:from great_expectations.datasource import SparkDFDatasourcefrom pyspark.sql import SparkSessionsession_name = 'mk_spark_session'spark = SparkSession.builder.appName(session_name).getOrCreate()datasource = SparkDFDatasource(spark)query = "SELE...

Data Engineering
Reply
  • 5934 Views
  • 2 replies
  • 1 kudos
View Replies
Latest Reply
jose_gonzalez
Databricks Employee
  • 1 kudos

Are you using a high concurrency cluster? if you do, please try to run this code on standard cluster

  • 1 kudos
1 More Replies
Anonymous
by • Not applicable
  • 6232 Views
  • 4 replies
  • 1 kudos

Constructor public com.databricks.backend.daemon.dbutils.FSUtilsParallel is not whitelisted when mounting a s3 bucket

Hello all, I'm experiencing this issueConstructor public com.databricks.backend.daemon.dbutils.FSUtilsParallel is not whitelisted when I'm trying to mount a s3 bucket. %python dbutils.fs.mount("s3a://dd-databricks-staging-storage/data/staging/datalak...

Data Engineering
Reply
  • 6232 Views
  • 4 replies
  • 1 kudos
View Replies
Latest Reply
leonids2005
New Contributor II
  • 1 kudos

WE have this problem running cluster with 11.2 and shared access mode. spark.databricks.pyspark.enablePy4JSecurity false - this does not help because it says spark.databricks.pyspark.enablePy4JSecurity is not allowed when choosing access modehere is ...

  • 1 kudos
3 More Replies
nemesis
by • New Contributor
  • 1377 Views
  • 2 replies
  • 0 kudos

Same Public ip of two machines in workspace.

Is there any way to change public ip of driver node? when I try to launch two different instance (single node), I see those two machine have the same public ip.I'm writing this command into notebook of each instance. I got the same ip.from requests i...

Data Engineering
Reply
  • 1377 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
Vidula
Honored Contributor
  • 0 kudos

Hi @Hakan Tekbas​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thank...

  • 0 kudos
1 More Replies
Labels