05-27-2022 03:39 AM
I was following the Databricks Academy "New Capabilities Overview: Feature Store" module. However when I try to run the code in the example notebook I get a security exception as explained below.
When I try to run the example notebook "01-Populate a feature store" I get an error while creating a feature table, I get a
py4j.security.Py4JSecurityException: Method public java.lang.String org.apache.spark.api.java.JavaSparkContext.getLocalProperty(java.lang.String) is not whitelisted on class class org.apache.spark.api.java.JavaSparkContext
This error is on the first line of the code
fs.create_feature_table(
name=f"fxs_eu_ntwkdaor_efsx_001.tmp_{username}_trip_pickup_features",
keys=["zip", "ts"],
features_df=pickup_features,
partition_columns="yyyy_mm",
description="Taxi Fares. Pickup Features",
)
fs.create_feature_table(
name=f"fxs_eu_ntwkdaor_efsx_001.tmp_{username}_trip_dropoff_features",
keys=["zip", "ts"],
features_df=dropoff_features,
partition_columns="yyyy_mm",
description="Taxi Fares. Dropoff Features",
)
:
How can I get this whitelisted?
06-10-2022 05:32 AM
The issue was due to the use of High Concurrency cluster. It was resolved when a standard cluster ML cluster was used. See screenshot.
06-01-2022 05:01 AM
Hi @Vikram Mahawal, Are you using the high Concurrency cluster mode?
06-01-2022 05:14 AM
Hi @Kaniz Fatma
Yes. It is a high concurrency cluster. Screenshot attached.
06-10-2022 05:37 AM
Hi @Vikram Mahawal, Thank you for the update. Please attach a standard cluster and check.
06-10-2022 05:32 AM
The issue was due to the use of High Concurrency cluster. It was resolved when a standard cluster ML cluster was used. See screenshot.
06-10-2022 05:35 AM
Hi @Vikram Mahawal, Thank you for your update. Now that your issue has been resolved, would you mind selecting the best answer for the community? Thanks again.
12-05-2022 04:52 PM
12-09-2022 09:23 AM
@Kaniz Fatma any thoughts on the above? thanks so much
12-29-2022 05:15 AM
Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections.
Click here to register and join today!
Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.