Input training dataset field empty in Configure AutoML experiment

[sideshowBob1337]

Trying to start an ML experiment on data in an extant metastore within a catalogue (SQL querys run fine on the database).  I can start an ML cluster, then attempt to start an AutoML expirement but I get stuck selecting training data - there are no databases or tables listed?

Do I have to create a dataset, if so where?  Can I link the existing metastore within the catalogue, if so how?

[feiyun0112]

you can read SQL data to DataFrame and run AutoML in the notebook

Train ML models with Databricks AutoML Python API | Databricks on AWS

[sideshowBob1337]

Thanks for the suggestion, it turns out the problem is with the setup of the compute cluster i.e. the cluster I have been using is set up correctly and can access the metastore but isnt ML runtime and when I set up my own ML runtime cluster I cannot configure it to see the metastore.  I think this means the metastore is external to databricks  and I have asked the admin to either set up an ML cluster with the right authentication or allow me to create one.  Long story short I cant do as you suggest because attaching the notebook to an ML cluster also breaks the connection to the SQL query. 

[Kaniz_Fatma]

Hey there! Thanks a bunch for being part of our awesome community! 🎉 

We love having you around and appreciate all your questions. Take a moment to check out the responses – you'll find some great info. Your input is valuable, so pick the best solution for you. And remember, if you ever need more help , we're here for you! 

Keep being awesome! 😊🚀