Hi!
So I used this MLFlow experiment I found from the databricks website: https://docs.databricks.com/_static/notebooks/machine-learning-with-unity-catalog.html
And I created this cluster using a custom Docker image I created myself: 
Usually when I create a cluster to run an MLFlow experiment, I use the ML version such as 10.4 LTS ML (includes Apache Spark 3.2.1, Scala 2.12). However, when I tried to select this for the Docker image, this error shows up:
So I thought, I'll just use the regular 10.4 LTS one. But when I used this and ran my MLFlow experiment, nothing shows up in the experiment runs.
Why is this happening? What can I do to make this work?
Please help! Thank you very much!
Have you tried the steps mentioned in the below URL:
https://docs.databricks.com/clusters/custom-containers.html#step-3-launch-your-cluster
Hi @Debayan Mukherjee!
Yes, I've been following this URL and have gone through step 3 for launch your cluster for the UI. This is what I have and it is pretty similar to the screenshot you have: 
The cluster was successfully created with the Docker image, but I cannot use the cluster for running MLflow experiments. I'm not sure what I did wrong.
Thank you!
