- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-03-2023 05:12 PM
Hi Everyone,
I've encountered an issue while trying to make use of the hive-metastore capability in Databricks to create a new database and table for our latest use case. The specific command I used was "create database if not exists newDB". However, I've noticed that the cluster operation has been running unusually long (over 10 minutes), which is not typical based on my experience.
In an attempt to troubleshoot, I came across a suggestion to restart the hive-metastore service using the command "%sh sudo service hive-metastore restart". This, however, resulted in an error: "Unit hive-metastore.service not found".
I've tried restarting the existing clusters and even created a new Standard_DS3_V2 cluster with 13.3 LTS runtime, but the issue persists. I'm wondering if I'm missing something in my approach or if there's a potential configuration issue. Could anyone advise on further steps I could take to diagnose this problem? Are there specific logs or settings I should be checking?
Any insights or suggestions would be greatly appreciated.
Thank you!
Accepted Solutions
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-04-2023 06:40 PM
Quick update, I resolved the issue. the problem was I created Azure Databricks inside Vnet without opening up default ports for Databricks to connect with MySql. Once I created the route table with the appropriate ports opened I can connect to Metastore. Thanks for the support Kaniz.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-04-2023 06:08 AM
Thanks for the debugging steps It helped me understand that Azure Databricks cannot reach the MYSQL server. I see this error now.
"Failed to initialize pool: Could not connect to address=(host=consolidated-northeuropec2-prod-metastore-2.mysql.database.azure.com)(port=3306)(type=master) : Socket fail to connect to host:consolidated-northeuropec2-prod-metastore-2.mysql.database.azure.com, port:3306. connect timed out"
From the research, it sounded like the problem with the firewall.
I enabled the inbound connection for port 3306. (I am not quite sure I did correctly or not). Please share your thoughts on this error.
Thanks
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-04-2023 06:40 PM
Quick update, I resolved the issue. the problem was I created Azure Databricks inside Vnet without opening up default ports for Databricks to connect with MySql. Once I created the route table with the appropriate ports opened I can connect to Metastore. Thanks for the support Kaniz.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-14-2024 01:39 AM
@Surya0 : i am facing same issue. stack trace is
Could not connect to address=(host=consolidated-northeuropec2-prod-metastore-2.mysql.database.azure.com)(port=3306)(type=master) : Socket fail to connect to host:consolidated-northeuropec2-prod-metastore-2.mysql.database.azure.com, port:3306. consolidated-northeuropec2-prod-metastore-2.mysql.database.azure.com
My ADB is also running in our VNET.
this is my inbound
and following is my outbound
I did open

