Node daemon ping timeout in 780000 ms for instance i-05ff43bc1d12412b5 @ 10.172.216.96. Please check network connectivity between the data plane and the control plane.
This error typically occurs when there’s a problem with communication between the data plane (where your cluster nodes reside) and the control plane (where the cluster management services operate).
Let’s troubleshoot this issue:
-
Network Connectivity:
- First, ensure that there’s proper network connectivity between the data plane and the control plane. Check if there are any network issues, firewalls, or security groups blocking communication.
- Verify that the security groups associated with your instances allow the necessary traffic (such as SSH, HTTP, and HTTPS) between the nodes and the control plane.
-
EC2 Instance Logs:
- Review the EC2 instance logs in AWS for more details about the error. Look for any specific messages related to connectivity issues or timeouts.
- Check if there are any issues related to the instance itself (such as resource constraints or misconfigurations).
-
Subnet Route Table and Security Groups:
- Examine the subnet route table configuration. Ensure that the route table allows traffic between the data plane and the control plane.
- Verify that the security groups associated with your instances are correctly configured. Make sure no changes have been inadvertently made.
-
Cluster Initialization:
- Sometimes, clusters fail to initialize due to transient issues. Retry creating the cluster after some time.
- If the problem persists, consider checking the status of Databricks services. You can find the status information on the Databricks status page (under AWS).
I hope this helps resolve the issue! If you have any other questions or need additional guidance, feel free to ask. 🚀
