cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
cancel
Showing results for 
Search instead for 
Did you mean: 

Security exception while using Feature Store. How can I get this whitelisted?

VM
Contributor

I was following the Databricks Academy "New Capabilities Overview: Feature Store" module. However when I try to run the code in the example notebook I get a security exception as explained below.

When I try to run the example notebook "01-Populate a feature store" I get an error while creating a feature table, I get a

py4j.security.Py4JSecurityException: Method public java.lang.String org.apache.spark.api.java.JavaSparkContext.getLocalProperty(java.lang.String) is not whitelisted on class class org.apache.spark.api.java.JavaSparkContext

This error is on the first line of the code

fs.create_feature_table(
 
   name=f"fxs_eu_ntwkdaor_efsx_001.tmp_{username}_trip_pickup_features",
 
   keys=["zip", "ts"],
 
   features_df=pickup_features,
 
   partition_columns="yyyy_mm",
 
   description="Taxi Fares. Pickup Features",
 
)
 
fs.create_feature_table(
 
   name=f"fxs_eu_ntwkdaor_efsx_001.tmp_{username}_trip_dropoff_features",
 
   keys=["zip", "ts"],
 
   features_df=dropoff_features,
 
   partition_columns="yyyy_mm",
 
   description="Taxi Fares. Dropoff Features",
 
)

:

How can I get this whitelisted?

1 ACCEPTED SOLUTION

Accepted Solutions

VM
Contributor

The issue was due to the use of High Concurrency cluster. It was resolved when a standard cluster ML cluster was used. See screenshot.

View solution in original post

8 REPLIES 8

Kaniz
Community Manager
Community Manager

Hi @Vikram Mahawal​, Are you using the high Concurrency cluster mode?

VM
Contributor

Hi @Kaniz Fatma​ 

Yes. It is a high concurrency cluster. Screenshot attached.

Kaniz
Community Manager
Community Manager

Hi @Vikram Mahawal​, Thank you for the update. Please attach a standard cluster and check.

VM
Contributor

The issue was due to the use of High Concurrency cluster. It was resolved when a standard cluster ML cluster was used. See screenshot.

Kaniz
Community Manager
Community Manager

Hi @Vikram Mahawal​, Thank you for your update. Now that your issue has been resolved, would you mind selecting the best answer for the community? Thanks again.

998925
New Contributor II

@Kaniz Fatma​ - this is happening again with the following configuration.

Screen Shot 2022-12-05 at 6.51.59 PM 

What configuration shall we use now that Cluster mode Standard is no longer a configuration option?

998925
New Contributor II

@Kaniz Fatma​ any thoughts on the above? thanks so much

OMG
New Contributor II

Hi @Daniel Barrundia​ - please select "No isolation shared" Access mode, it should resolve this problem.image

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.