Databricks VPC - EKS VPC security groups - Databricks Community

Data Engineering

Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.

Hi,

We have a databricks deployment in our AWS account in a dedicated VPC which we created a VPC peering to our EKS VPC, in the EKS main security group we added a rule that opens all TCP ports from the Databricks VPC and now it's working.

Once I try to limit that rule databricks has no connectivity to our EKS, what are the needed ports that I need to open because if it's possible I don't want to open all the ports.

I also saw this document: https://docs.databricks.com/administration-guide/cloud-configurations/aws/vpc-peering.html

and I see in step 8 that I should add the port in the unmanaged SG, but in terms of EKS I don't know which port should I add exactly