a month ago - last edited a month ago
Hello All,
I just tried running the following notebook (https://docs.databricks.com/aws/en/notebooks/source/machine-learning/optuna-mlflow.html) on the Databricks Free Edition platform , through Microsoft Account Authentication. It takes 15 minutes to run 8 trials then outputs a grpc_status:13 error before it breaks. Could this be because the distributed MlflowSparkStudy class cannot work on the Free Edition, or is this an internal bug.
Thank you in advance !
a month ago - last edited a month ago
@Edwin1
I can appreciate this isn't a solution with proper reasoning but changing the following in your notebook, on the free edition, should allow you to use the notebook:
I reduced the number of trials and it worked 🤔.
Best guess it's around the compute limitations? Not a definitive conclusion though.
Here's AI's take on the situation:
Hopefully that's a step in the right direction for you @Edwin1 ☺️.
All the best,
BS
a month ago
For what it's worth, I just realized the free edition is a single node cluster, so this is mute point anyway.
Thank you!
a month ago - last edited a month ago
@Edwin1
I can appreciate this isn't a solution with proper reasoning but changing the following in your notebook, on the free edition, should allow you to use the notebook:
I reduced the number of trials and it worked 🤔.
Best guess it's around the compute limitations? Not a definitive conclusion though.
Here's AI's take on the situation:
Hopefully that's a step in the right direction for you @Edwin1 ☺️.
All the best,
BS
a month ago
Great. Thank you. That worked. I still need more compute and networking resources to make it justifiable, but this confirms that it works !!!
4 weeks ago
No worries ☺️. If you end up building out a cool project with the Free Edition, I'm sure the community would love to hear about it. I've not got around to anything ML-related in Databricks yet, very eager though 🫡. There's a dedicated space for it https://community.databricks.com/t5/community-articles/bd-p/Knowledge-Sharing-Hub
Best of luck with the project.
All the best,
BS
Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!
Sign Up Now