Currently I trying to Create a Compute Cluster on a Workspaces with Privatelink and Custom VPC.

I'm using Terraform: https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/aws-private-link-wo...

After the deployment is completed, I try to Create a Compute cluster but I'm getting following error:

Failed to add 3 workers to the compute. Will attempt retry: true. Reason: Driver unresponsive

Spark driver became unresponsive on startup. This issue can be caused by invalid Spark configurations or malfunctioning init scripts. Please refer to the Spark driver logs to troubleshoot this issue, and contact Databricks if the problem persists.

Internal error message: Spark failed to start: Driver unresponsive. Possible reasons: library conflicts, incorrect metastore configuration, and init script misconfiguration.

I tried everything: creating the S3 Gateway Endpoint, STS Interface Endpoint, Kinesis-Streams Interface Endpoint, 

also in the Security Group I have the corresponding ports on Inbound and Outbound rules:

Security Group - Network Workspace - Inbound Rules

Security Group - Network Workspace - Outbound Rules

Any help will be appretiated. Thanks!