https://community.databricks.com/t5/data-engineering/spark-is-reading-data-from-source-even-i-am-persisting-the-data/m-p/23381#M16126 <P>hI all</P><P></P><P>I am reading data and I am caching the data and then I am performing Action Count to get the data in memory, but still, in dag I found out that every time it reads data from SOURCE.</P> Thu, 17 Jun 2021 08:02:03 GMT User16826994223 2021-06-17T08:02:03Z https://community.databricks.com/t5/data-engineering/spark-is-reading-data-from-source-even-i-am-persisting-the-data/m-p/23381#M16126 <P>hI all</P><P></P><P>I am reading data and I am caching the data and then I am performing Action Count to get the data in memory, but still, in dag I found out that every time it reads data from SOURCE.</P> Thu, 17 Jun 2021 08:02:03 GMT https://community.databricks.com/t5/data-engineering/spark-is-reading-data-from-source-even-i-am-persisting-the-data/m-p/23381#M16126 User16826994223 2021-06-17T08:02:03Z https://community.databricks.com/t5/data-engineering/spark-is-reading-data-from-source-even-i-am-persisting-the-data/m-p/23382#M16127 <P>It looks like the the spark memory is not sufficient to cache all the data so it read always from source</P> Sat, 26 Jun 2021 17:52:35 GMT https://community.databricks.com/t5/data-engineering/spark-is-reading-data-from-source-even-i-am-persisting-the-data/m-p/23382#M16127 User16826994223 2021-06-26T17:52:35Z