Currently I am using Azure Machine Learning Studio for my work, and would like to compare performance of Azure and Databricks automl algorithms. Is it possible to write a notebook in Azure to start the automl algorithm in Databricks? My data is found in the Azure workspace.
So far I managed to connect to my Databricks cluster and start a SparkSession via databricks-connect (version 13 as my cluster uses the 13.0 ML runtime), but it doesn't help in this situation.
Does anyone have experience with this kind of situation?
