cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Cluster doesn't support Photon with Docker Image enabled

Pragan
New Contributor

I enabled Photon 9.1 LTS DBR in cluster that was already using Docker Image of the latest version, when I ran a SQL QUery using my cluster, I could not see any Photon engine working in my executor that should be actually running in Photon Engine.

When I disable docker image from my cluster, I could see Photon engine running. Whether I have to enable some feature to support Photon in my docker image or does docker does not Photon feature?

1 ACCEPTED SOLUTION

Accepted Solutions

Anonymous
Not applicable

Hello @Praganessh S​ , Photon is currently in Public Preview. The only way to use it is to explicitly run Databricks-provide Runtime images which contain it. Please see: https://docs.databricks.com/runtime/photon.html#databricks-clusters

and

https://docs.databricks.com/clusters/configure.html#photon-images

View solution in original post

5 REPLIES 5

Anonymous
Not applicable

Hello, @Praganessh S​! Welcome to the community. 🙂

My name is Piper and I'm a moderator for the community. Let's give it a while to see what your peers have to say. If we need to, we'll swing back around.

Thank you for your patience.

-werners-
Esteemed Contributor III

Interesting...

The fact that it does not use docker makes me think the photon functionality is not (yet) copied to the docker images.

Maybe someone from Databricks can confirm.

Kaniz
Community Manager
Community Manager

Hi @Praganessh S​ and @Werner Stinckens​ , We're looking for the confirmation for you. Please bear with us. Apologies for the delay.

Kaniz
Community Manager
Community Manager

Hi @Praganessh S​ , Would you please raise a support ticket with Databricks?

We are not aware of any Photon+DCS limitations, but depending on what you have in the docker image, it's certainly possible Photon could somehow break.

We would need to debug this specific case.

Anonymous
Not applicable

Hello @Praganessh S​ , Photon is currently in Public Preview. The only way to use it is to explicitly run Databricks-provide Runtime images which contain it. Please see: https://docs.databricks.com/runtime/photon.html#databricks-clusters

and

https://docs.databricks.com/clusters/configure.html#photon-images

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.