topic Re: SQL warehouse failing to start ( Please check network connectivity from the data plane to the control plane ) in Data Engineering

SQL warehouse failing to start ( Please check network connectivity from the data plane to the control plane )

Cano — Thu, 12 Jan 2023 21:04:50 GMT

Hi,

My SQL warehouse is failing to start with the following error message:

Details for the latest failure: Error: [id: InstanceId(i-01b84b6705ff09104), status: INSTANCE_INITIALIZING, workerEnvId:WorkerEnvId(workerenv-3023557811934763-c8cef827-a038-4552-a74d-829ee273d683), lastStatusChangeTime: 1673489057531, groupIdOpt Some(0),requestIdOpt Some(0112-020302-ltb2xfyq-2b759b89-28f3-42ff-b),version 2] with threshold 700 seconds timed out after 704732 milliseconds. Please check network connectivity from the data plane to the control plane. Code: BOOTSTRAP_TIMEOUT Cluster-id (internal): 0112-020302-ltb2xfyq Additional details: {"instance_id":"i-01b84b6705ff09104"}

I followed the steps here https://docs.databricks.com/administration-guide/cloud-configurations/aws/customer-managed-vpc.html#required-data-plane-addresses

All the requirements for security group, subnet, NACLs, route table were met.

I even tried to create a new workspace but that also failed.

I have also attached the log file.

Your urgent help is hugely appreciated.

Re: SQL warehouse failing to start ( Please check network connectivity from the data plane to the control plane )

Cano — Fri, 13 Jan 2023 15:50:49 GMT

Hi @Debayan Mukherjee

Everything is managed through the security group and it allows all traffic. And there is also a NAT gateway associated to the subnet to provide internet connectivity.

I see the Databricks Instances spin up in my AWS account, is there Is anyway I can connect to these instances to a run a few tests?

I have attached a screenshot of both inbound and outbound rules for your viewing.

Please let me know if you have any other suggestions.

Re: SQL warehouse failing to start ( Please check network connectivity from the data plane to the control plane )

Cano — Fri, 13 Jan 2023 16:16:27 GMT

Please note that I'm not using a customer managed VPC, but a VPC that was created by Databricks when I created the workspace.

Re: SQL warehouse failing to start ( Please check network connectivity from the data plane to the control plane )

Debayan — Thu, 12 Jan 2023 21:44:50 GMT

Hi, There is a line in the attached logs as below:

[Bootstrap Event] Can reach ohio.cloud.databricks.com: [FAILED]

[Bootstrap Event] DNS output for databricks-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com:

Server: 10.187.0.2

Address: 10.187.0.2#53

This means somehow the webapp endpoint cannot reach as per the document: https://docs.databricks.com/administration-guide/cloud-configurations/aws/customer-managed-vpc.html?&_ga=2.220166233.159433179.1673268087-304889111.1666382193#required-data-plane-addresses

Could you please make sure if it has been added and let us know if it helps?

Re: SQL warehouse failing to start ( Please check network connectivity from the data plane to the control plane )

Debayan — Mon, 16 Jan 2023 04:56:56 GMT

Hi @charles okoh , Thanks for all the details. I understand it is a default VPC. In that case, could you please raise a support case with Databricks?