ā02-03-2022 12:44 AM
I enabled Photon 9.1 LTS DBR in cluster that was already using Docker Image of the latest version, when I ran a SQL QUery using my cluster, I could not see any Photon engine working in my executor that should be actually running in Photon Engine.
When I disable docker image from my cluster, I could see Photon engine running. Whether I have to enable some feature to support Photon in my docker image or does docker does not Photon feature?
ā03-29-2022 09:37 AM
Hello @Praganessh Sā , Photon is currently in Public Preview. The only way to use it is to explicitly run Databricks-provide Runtime images which contain it. Please see: https://docs.databricks.com/runtime/photon.html#databricks-clusters
and
https://docs.databricks.com/clusters/configure.html#photon-images
ā02-03-2022 08:55 AM
Hello, @Praganessh Sā! Welcome to the community. š
My name is Piper and I'm a moderator for the community. Let's give it a while to see what your peers have to say. If we need to, we'll swing back around.
Thank you for your patience.
ā02-04-2022 12:10 AM
Interesting...
The fact that it does not use docker makes me think the photon functionality is not (yet) copied to the docker images.
Maybe someone from Databricks can confirm.
ā03-15-2022 11:39 PM
Hi @Praganessh Sā and @Werner Stinckensā , We're looking for the confirmation for you. Please bear with us. Apologies for the delay.
ā03-29-2022 04:19 AM
Hi @Praganessh Sā , Would you please raise a support ticket with Databricks?
We are not aware of any Photon+DCS limitations, but depending on what you have in the docker image, it's certainly possible Photon could somehow break.
We would need to debug this specific case.
ā03-29-2022 09:37 AM
Hello @Praganessh Sā , Photon is currently in Public Preview. The only way to use it is to explicitly run Databricks-provide Runtime images which contain it. Please see: https://docs.databricks.com/runtime/photon.html#databricks-clusters
and
https://docs.databricks.com/clusters/configure.html#photon-images
Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections.
Click here to register and join today!
Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.