https://community.databricks.com/t5/data-engineering/jobs-are-failed-with-aws-insufficient-free-addresses-in-subnet/m-p/16066#M10281 <P>@karthik p​&nbsp;Have configured one subnet per AZ(total 3). Have followed the same steps as mentioned in the document. Is there a way to check whether the Databricks uses all the subnets or not?</P><P>@Debayan Mukherjee​&nbsp;am not getting how to use LB in this setup. Can you throw some light /share some document on how to use it on custom managed VPC.</P><P></P> Tue, 20 Dec 2022 06:20:42 GMT Manimkm08 2022-12-20T06:20:42Z https://community.databricks.com/t5/data-engineering/jobs-are-failed-with-aws-insufficient-free-addresses-in-subnet/m-p/16063#M10278 <P>We have assigned 3 dedicated subnets (one per AZ ) to the Databricks workspace each with /24 CIDR but noticed that all the jobs are running into a single subnet which causes&nbsp;</P><P>AWS_INSUFFICIENT_FREE_ADDRESSES_IN_SUBNET_FAILURE.</P><P></P><P>Is there a way to segregate the jobs across all the available subnets instead of using the same one all the time ?</P><P></P><P><B>Error Message:</B></P><P><B>Unexpected state for cluster (1217-000012-7mijsxio): AWS_INSUFFICIENT_FREE_ADDRESSES_IN_SUBNET_FAILURE(CLIENT_ERROR): aws_api_error_code:InvalidParameterValue,aws_error_message:Not enough free addresses in subnet subnet-0bd28fa8a819c24f1</B></P><P></P> Mon, 19 Dec 2022 07:41:19 GMT https://community.databricks.com/t5/data-engineering/jobs-are-failed-with-aws-insufficient-free-addresses-in-subnet/m-p/16063#M10278 Manimkm08 2022-12-19T07:41:19Z https://community.databricks.com/t5/data-engineering/jobs-are-failed-with-aws-insufficient-free-addresses-in-subnet/m-p/16064#M10279 <P>@Mani Srini​&nbsp;you should not get this error, if below recommendations are followed</P><P></P><OL><LI>have you created 2 private subnets with different AZ</LI></OL><P>please re-validate <A href="https://docs.databricks.com/administration-guide/cloud-configurations/aws/customer-managed-vpc.html#:~:text=By%20default%2C%20clusters%20are%20created%20in%20a%20single,own%20VPC%2C%20a%20feature%20known%20as%20customer-managed%20VPC." alt="https://docs.databricks.com/administration-guide/cloud-configurations/aws/customer-managed-vpc.html#:~:text=By%20default%2C%20clusters%20are%20created%20in%20a%20single,own%20VPC%2C%20a%20feature%20known%20as%20customer-managed%20VPC." target="_blank">Customer-managed VPC | Databricks on AWS</A> subnets section</P> Mon, 19 Dec 2022 16:15:57 GMT https://community.databricks.com/t5/data-engineering/jobs-are-failed-with-aws-insufficient-free-addresses-in-subnet/m-p/16064#M10279 karthik_p 2022-12-19T16:15:57Z https://community.databricks.com/t5/data-engineering/jobs-are-failed-with-aws-insufficient-free-addresses-in-subnet/m-p/16066#M10281 <P>@karthik p​&nbsp;Have configured one subnet per AZ(total 3). Have followed the same steps as mentioned in the document. Is there a way to check whether the Databricks uses all the subnets or not?</P><P>@Debayan Mukherjee​&nbsp;am not getting how to use LB in this setup. Can you throw some light /share some document on how to use it on custom managed VPC.</P><P></P> Tue, 20 Dec 2022 06:20:42 GMT https://community.databricks.com/t5/data-engineering/jobs-are-failed-with-aws-insufficient-free-addresses-in-subnet/m-p/16066#M10281 Manimkm08 2022-12-20T06:20:42Z https://community.databricks.com/t5/data-engineering/jobs-are-failed-with-aws-insufficient-free-addresses-in-subnet/m-p/16065#M10280 <P>Hi, this is a failure when AWS subnet has been exhausted and there is no IP available for new services. </P><P><A href="https://aws.amazon.com/premiumsupport/knowledge-center/subnet-insufficient-ips/" target="test_blank">https://aws.amazon.com/premiumsupport/knowledge-center/subnet-insufficient-ips/</A></P> Mon, 19 Dec 2022 16:47:00 GMT https://community.databricks.com/t5/data-engineering/jobs-are-failed-with-aws-insufficient-free-addresses-in-subnet/m-p/16065#M10280 Debayan 2022-12-19T16:47:00Z