Photon Acceleration not getting enabled for ML runtime in Azure

[Sajid1]

I tried to enable the photon acceleration in ML runtime 9.1 LTS ML (Scala 2.12,Spark 3.1.2) but getting error "selected runtime version does not support photon".

I tried for other versions of ML runtime with single and multinode , access mode being single.

• Node : Standard_E4ds_v4

Kindly let me know where am I going wrong

[karthik_p]

@Sajid Thavalengal Rahiman​ Yes please, we have also tried that and found it is not supported for ML Runtimes. but other way to run your data science workloads is use standard DBR above 9.1 LTS and enable photon and run your workloads and keep on tuning your worker type and driver type

Photon runtime | Databricks on AWS

[Kaniz]

Hi @Sajid Thavalengal Rahiman​ , We haven’t heard from you since the last response from @karthik p​, and I was checking back to see if you have a resolution yet.

If you have any solution, please share it with the community as it can be helpful to others. Otherwise, we will respond with more details and try to help.

Also, Please don't forget to click on the "Select As Best" button whenever the information provided helps resolve your question.

[Kaniz]

Hi @Sajid Thavalengal Rahiman​, Photon is available for clusters running Databricks Runtime 9.1 LTS and above.

[Sajid1 · Accepted solution]

But it is not available to ML runtime. Photon acceleration is provided for only Standard runtime above 9.1 LTS

[Anonymous]

Hi @Sajid Thavalengal Rahiman​ 

Does @Kaniz Fatma​ response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?

We'd love to hear from you.

Thanks!