Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-23-2023 03:11 PM
@jakubk wrote:
Do I really need a dedicated cluster per user to be able to use unity catalog & load data?? That can't be right
I certainly Hope that ain't the case. I can call spark.catalog.TableExists without issue from a Personal Compute cluster, but when I try to call it from a Shared Compute cluster with Access mode = Shared, I get this error:
"py4j.security.Py4JSecurityException: Method public boolean org.apache.spark.sql.internal.CatalogImpl.tableExists(java.lang.String) is not whitelisted ..."
How do I check if a table exists from a Shared Cluster if I'm not allowed to use spark.catalog.tableExists?