cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Jobs are failed with AWS_INSUFFICIENT_FREE_ADDRESSES_IN_SUBNET_FAILURE

Manimkm08
New Contributor III

We have assigned 3 dedicated subnets (one per AZ ) to the Databricks workspace each with /24 CIDR but noticed that all the jobs are running into a single subnet which causes 

AWS_INSUFFICIENT_FREE_ADDRESSES_IN_SUBNET_FAILURE.

Is there a way to segregate the jobs across all the available subnets instead of using the same one all the time ?

Error Message:

Unexpected state for cluster (1217-000012-7mijsxio): AWS_INSUFFICIENT_FREE_ADDRESSES_IN_SUBNET_FAILURE(CLIENT_ERROR): aws_api_error_code:InvalidParameterValue,aws_error_message:Not enough free addresses in subnet subnet-0bd28fa8a819c24f1

3 REPLIES 3

karthik_p
Esteemed Contributor

@Mani Sriniโ€‹ you should not get this error, if below recommendations are followed

  1. have you created 2 private subnets with different AZ

please re-validate Customer-managed VPC | Databricks on AWS subnets section

Debayan
Esteemed Contributor III
Esteemed Contributor III

Hi, this is a failure when AWS subnet has been exhausted and there is no IP available for new services.

https://aws.amazon.com/premiumsupport/knowledge-center/subnet-insufficient-ips/

Manimkm08
New Contributor III

@karthik pโ€‹ Have configured one subnet per AZ(total 3). Have followed the same steps as mentioned in the document. Is there a way to check whether the Databricks uses all the subnets or not?

@Debayan Mukherjeeโ€‹ am not getting how to use LB in this setup. Can you throw some light /share some document on how to use it on custom managed VPC.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.