Jobs are failed with AWS_INSUFFICIENT_FREE_ADDRESSES_IN_SUBNET_FAILURE

[Manimkm08]

We have assigned 3 dedicated subnets (one per AZ ) to the Databricks workspace each with /24 CIDR but noticed that all the jobs are running into a single subnet which causes 

AWS_INSUFFICIENT_FREE_ADDRESSES_IN_SUBNET_FAILURE.

Is there a way to segregate the jobs across all the available subnets instead of using the same one all the time ?

Error Message:

Unexpected state for cluster (1217-000012-7mijsxio): AWS_INSUFFICIENT_FREE_ADDRESSES_IN_SUBNET_FAILURE(CLIENT_ERROR): aws_api_error_code:InvalidParameterValue,aws_error_message:Not enough free addresses in subnet subnet-0bd28fa8a819c24f1

[karthik_p]

@Mani Srini​ you should not get this error, if below recommendations are followed

1. have you created 2 private subnets with different AZ

please re-validate Customer-managed VPC | Databricks on AWS subnets section

[Debayan]

Hi, this is a failure when AWS subnet has been exhausted and there is no IP available for new services.

https://aws.amazon.com/premiumsupport/knowledge-center/subnet-insufficient-ips/

[Manimkm08]

@karthik p​ Have configured one subnet per AZ(total 3). Have followed the same steps as mentioned in the document. Is there a way to check whether the Databricks uses all the subnets or not?

@Debayan Mukherjee​ am not getting how to use LB in this setup. Can you throw some light /share some document on how to use it on custom managed VPC.