https://community.databricks.com/t5/data-engineering/sharikrishna26-medium-com/m-p/14497#M8969 <P><B>Spark Dataframe Metadata</B></P><P></P><P>Spark Dataframe is structurally the same as the table. However, it does not store any schema information in the metadata store. Instead, we have a runtime metadata catalog to store the Dataframe schema information. It is similar to the metadata store, but Spark will create it at the runtime to store schema information in the catalog</P><P></P><P>Why, reasons for storing schema information in the catalog?</P><P>Please find below*******************link</P><P></P><P><A href="https://sharikrishna26.medium.com/spark-dataframe-metadata-a2544c16081d" target="test_blank">https://sharikrishna26.medium.com/spark-dataframe-metadata-a2544c16081d</A></P> Mon, 26 Dec 2022 16:07:02 GMT SIRIGIRI 2022-12-26T16:07:02Z https://community.databricks.com/t5/data-engineering/sharikrishna26-medium-com/m-p/14497#M8969 <P><B>Spark Dataframe Metadata</B></P><P></P><P>Spark Dataframe is structurally the same as the table. However, it does not store any schema information in the metadata store. Instead, we have a runtime metadata catalog to store the Dataframe schema information. It is similar to the metadata store, but Spark will create it at the runtime to store schema information in the catalog</P><P></P><P>Why, reasons for storing schema information in the catalog?</P><P>Please find below*******************link</P><P></P><P><A href="https://sharikrishna26.medium.com/spark-dataframe-metadata-a2544c16081d" target="test_blank">https://sharikrishna26.medium.com/spark-dataframe-metadata-a2544c16081d</A></P> Mon, 26 Dec 2022 16:07:02 GMT https://community.databricks.com/t5/data-engineering/sharikrishna26-medium-com/m-p/14497#M8969 SIRIGIRI 2022-12-26T16:07:02Z https://community.databricks.com/t5/data-engineering/sharikrishna26-medium-com/m-p/14498#M8970 <P>Great ​</P> Mon, 26 Dec 2022 17:01:06 GMT https://community.databricks.com/t5/data-engineering/sharikrishna26-medium-com/m-p/14498#M8970 Rishabh-Pandey 2022-12-26T17:01:06Z https://community.databricks.com/t5/data-engineering/sharikrishna26-medium-com/m-p/14499#M8971 <P>this is awesome thanks </P> Tue, 27 Dec 2022 01:00:51 GMT https://community.databricks.com/t5/data-engineering/sharikrishna26-medium-com/m-p/14499#M8971 Aviral-Bhardwaj 2022-12-27T01:00:51Z