topic Photon Acceleration not getting enabled for ML runtime in Azure in Data Engineering

Photon Acceleration not getting enabled for ML runtime in Azure

Sajid1 — Mon, 10 Oct 2022 12:34:52 GMT

I tried to enable the photon acceleration in ML runtime 9.1 LTS ML (Scala 2.12,Spark 3.1.2) but getting error "selected runtime version does not support photon".

I tried for other versions of ML runtime with single and multinode , access mode being single.

Node : Standard_E4ds_v4

Kindly let me know where am I going wrong

Re: Photon Acceleration not getting enabled for ML runtime in Azure

karthik_p — Thu, 13 Oct 2022 14:30:43 GMT

@Sajid Thavalengal Rahiman Yes please, we have also tried that and found it is not supported for ML Runtimes. but other way to run your data science workloads is use standard DBR above 9.1 LTS and enable photon and run your workloads and keep on tuning your worker type and driver type

Photon runtime | Databricks on AWS

Re: Photon Acceleration not getting enabled for ML runtime in Azure

Anonymous — Thu, 17 Nov 2022 06:39:35 GMT

Hi @Sajid Thavalengal Rahiman

Does @Kaniz Fatma response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?

We'd love to hear from you.

Thanks!

Re: Photon Acceleration not getting enabled for ML runtime in Azure

Sajid1 — Thu, 17 Nov 2022 06:43:50 GMT

But it is not available to ML runtime. Photon acceleration is provided for only Standard runtime above 9.1 LTS