- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-09-2021 11:48 AM
A temp view is a pointer.
The information for a temp view is stored in the spark catalog
You can drop a temp view with
spark.catalog.dropTempView("view_name")
You could also drop a temp view in a sql cell with
DROP TABLE "temp_view_name"
Here is some code to demonstrate
df = spark.sql("select 1 id") # creates a dataframe
df.createOrReplaceTempView("temp_test") # registers it as a temp view
spark.catalog.listTables() # Shows all tables including temp views
spark.catalog.dropTempView("temp_test") # drops the temp view
Note that the dataframe still exists, display(df) to verify. The tempview is merely a pointer to the dataframe.
If you are using sql the sample code would look something like this.
CREATE TEMPORARY VIEW temp_test as select 1; --Creates the view
Drop table temp_test; --drops the view