@jakubk wrote:
Do I really need a dedicated cluster per user to be able to use unity catalog & load data?? That can't be right

I certainly Hope that ain't the case. I can call spark.catalog.TableExists without issue from a Personal Compute cluster, but when I try to call it from a Shared Compute cluster with Access mode = Shared, I get this error:
"py4j.security.Py4JSecurityException: Method public boolean org.apache.spark.sql.internal.CatalogImpl.tableExists(java.lang.String) is not whitelisted ..."

How do I check if a table exists from a Shared Cluster if I'm not allowed to use spark.catalog.tableExists?