Constructor public org.apache.spark.ml.feature.Bucketizer(java.lang.String) is not whitelisted.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-30-2024 03:39 AM
Hi,
I am getting the error Constructor public org.apache.spark.ml.feature.Bucketizer(java.lang.String) is not whitelisted. when using a serverless compute cluster. I have seen in some other articles that this is due to high concurrency - does anyone know a way around this while still using a serverless compute cluster?
Thanks
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-02-2024 09:20 AM
The error you're encountering, "Constructor public org.apache.spark.ml.feature.Bucketizer(java.lang.String) is not whitelisted", typically arises when using a shared mode cluster. This is because Spark ML is not supported in shared clusters due to security reasons. Some Spark ML calls are restricted and whitelisting all classes is not feasible for security reasons.
The solution is to use a Single User cluster, but as of now the Serverless cluster is designed in Shared Access mode and cannot be modified
