topic docs.databricks.com in Machine Learning https://community.databricks.com/t5/machine-learning/docs-databricks-com/m-p/17807#M971 <P>Advantage of using Photon Engine</P><P>The following summarizes the advantages of Photon:</P><UL><LI>Supports SQL and equivalent DataFrame operations against Delta and Parquet tables.</LI><LI>Expected to accelerate queries that process a significant amount of data (100GB+) and include aggregations and joins.</LI><LI>Data is accessed repeatedly and likely in the Delta Lake cache.</LI><LI>More robust scan performance on tables with many columns and many small files.</LI><LI>Faster Delta and Parquet writing using&nbsp;UPDATE,&nbsp;DELETE,&nbsp;MERGE&nbsp;INTO,&nbsp;INSERT and&nbsp;CREATE&nbsp;TABLE&nbsp;AS&nbsp;SELECT especially for wide tables (hundreds to thousands of columns).</LI><LI>Photon replaces sort-merge joins with hash-joins.</LI></UL><P>More details are on - <A href="https://docs.databricks.com/runtime/photon.html" target="test_blank">https://docs.databricks.com/runtime/photon.html</A></P><P></P> Mon, 28 Jun 2021 13:23:09 GMT User16826994223 2021-06-28T13:23:09Z docs.databricks.com https://community.databricks.com/t5/machine-learning/docs-databricks-com/m-p/17807#M971 <P>Advantage of using Photon Engine</P><P>The following summarizes the advantages of Photon:</P><UL><LI>Supports SQL and equivalent DataFrame operations against Delta and Parquet tables.</LI><LI>Expected to accelerate queries that process a significant amount of data (100GB+) and include aggregations and joins.</LI><LI>Data is accessed repeatedly and likely in the Delta Lake cache.</LI><LI>More robust scan performance on tables with many columns and many small files.</LI><LI>Faster Delta and Parquet writing using&nbsp;UPDATE,&nbsp;DELETE,&nbsp;MERGE&nbsp;INTO,&nbsp;INSERT and&nbsp;CREATE&nbsp;TABLE&nbsp;AS&nbsp;SELECT especially for wide tables (hundreds to thousands of columns).</LI><LI>Photon replaces sort-merge joins with hash-joins.</LI></UL><P>More details are on - <A href="https://docs.databricks.com/runtime/photon.html" target="test_blank">https://docs.databricks.com/runtime/photon.html</A></P><P></P> Mon, 28 Jun 2021 13:23:09 GMT https://community.databricks.com/t5/machine-learning/docs-databricks-com/m-p/17807#M971 User16826994223 2021-06-28T13:23:09Z