Photon Benchmark

[Enrique1987]

I'm conducting my own comparative study between a cluster with Photon enabled and a cluster without Photon to see what improvements occur. According to Databricks, there should be up to 12x better performance, but I'm only finding about a 20% improvement. Does anyone know more about which operations Photon might be interesting for or in what scenarios it excels? Or does anyone know more about the benchmark Databricks conducted in this case?
BR, Enrique

[szymon_dybczak]

Hi @Enrique1987 ,

You can find more information about photon in below whitepaper:

https://people.eecs.berkeley.edu/~matei/papers/2022/sigmod_photon.pdf