https://community.databricks.com/t5/get-started-discussions/unpersist-doesn-t-clear/m-p/54446#M1911 <P>Thank you so much for taking time and explaining the concepts</P> Fri, 01 Dec 2023 16:44:31 GMT anandreddy23 2023-12-01T16:44:31Z https://community.databricks.com/t5/get-started-discussions/unpersist-doesn-t-clear/m-p/50570#M1711 <P>from pyspark.sql import SparkSession<BR />from pyspark import SparkContext, SparkConf<BR />from pyspark.storagelevel import StorageLevel<BR />spark = SparkSession.builder.appName('TEST').config('spark.ui.port','4098').enableHiveSupport().getOrCreate()</P><P>df4 = spark.sql(' \<BR />select * from hive_schema.table_name limit 1')<BR />print("query completed " )<BR />df4.unpersist()<BR />df4.count()</P><P>df4.show()</P><P>I have execute above code to&nbsp; clear the dataframe release the memory. However, df4.show() still works and shows the data. Could you please help me with right method to free memory occupied by a spark DF please ?</P> Tue, 07 Nov 2023 15:37:41 GMT https://community.databricks.com/t5/get-started-discussions/unpersist-doesn-t-clear/m-p/50570#M1711 anandreddy23 2023-11-07T15:37:41Z https://community.databricks.com/t5/get-started-discussions/unpersist-doesn-t-clear/m-p/54446#M1911 <P>Thank you so much for taking time and explaining the concepts</P> Fri, 01 Dec 2023 16:44:31 GMT https://community.databricks.com/t5/get-started-discussions/unpersist-doesn-t-clear/m-p/54446#M1911 anandreddy23 2023-12-01T16:44:31Z