Sorry, I vaugely remember we used to create persistent views on dataframe earlier.

Currently, spark dataframe doesn't allow you to create pesistent view on dataframe, rather you have to create table to use it in SQL warehouse.

# Assuming there is an existing table named 'existing_table'
df = spark.table("system.information_schema.catalog_privileges")

df.createOrReplaceTempView("temp_view") # Temp View at sesion levle, once session is closed, the view is dropped
df.createOrReplaceGlobalTempView("gbl_temp_view") # Global Temp View accessible across all sessions, even if the current session is closed, the view is still accessible.

spark.table("global_temp.gbl_temp_view").display()

And you cannot create persistant view on top of dataframe or temp-view, which throws error.

Instead, you can create view on tables (even joins) directly which can be accessed in SQL warehouse, but you have to recreate the spark logic into SQL for that you can leverage Databricks Assitance to convert it with less effort.

create
or replace view hive_metastore.labuser8777982_1736395171_wa99_da_adewd_lab.sys_catalog_privileges_new as
select *
from system.information_schema.catalog_privileges -- this is actual table in system schema
where privilege_type = 'EXECUTE'

Regards,

Hari Prasad.