Can't create AWS p3 instance

[jjsnlee]

Hi, I'm trying to create a `p3.2xlarge` in my workspace, but the cluster fails to instantiate, specifically getting this error message: `No zone supports both the driver instance type [p3.2xlarge] and the worker instance type [p3.2xlarge]` (though I am just creating a Single Node instance). I can create other clusters with other GPU instance types (eg, g6 class).

I can confirm that I have Limit capacity for P instances (76 vCPUs in the region, us-east-1). Does anyone have any clues on how I can debug this issue?

[Walter_C]

Can you please share the complete error showed in the event logs when trying to run the cluster? does the availability zone available have the capacity for this instances?

[jjsnlee]

Here is an example (this was when I tried a `p3.8xlarge`, and I set the Availability Zone = Auto for the cluster)

```In zone us-east-1c, insufficient capacity for the following node type: p3.8xlarge. AWS Error: Failure happened when talking to AWS, AWS API error code: Unsupported AWS error message: Your requested instance type (p3.8xlarge) is not supported in your requested Availability Zone (us-east-1c). Please retry your request by not specifying an Availability Zone or choosing us-east-1a, us-east-1d, us-east-1e, us-east-1f. In zone us-east-1d, insufficient capacity for the following node type: p3.8xlarge. AWS Error: Failure happened when talking to AWS, AWS API error code: Unsupported AWS error message: Your requested instance type (p3.8xlarge) is not supported in your requested Availability Zone (us-east-1d). Please retry your request by not specifying an Availability Zone or choosing us-east-1a, us-east-1e, us-east-1f.```

The odd thing is it gives that message once, and then just gives a more succinct message in the Event Log: `No zone supports both the driver instance type [p3.8xlarge] and the worker instance type [p3.8xlarge]` afterwards. And every time I try to start the cluster it continues with that message, as if it doesn't try again across the different AZ's. 

[Walter_C]

I tried on my internal environment setting it on us-east-1a or us-east-1b and it has worked on my side, can you try to force them to be created on any of this 2, is it the same exact issue?

[tyagip1]

Facing the same issue. Tried specifying the Availability zone as eu-west-1a, eu-west-1b and auto. But event logs throws the same error:

[Walter_C]

Can you please provide me an screenshot of the complete configuration you are currently setting up for this cluster?

[tyagip1]

"first_on_demand": 1,
"node_type_id": "p3.2xlarge"
"node_type_id": "p3.2xlarge"
"data_security_mode": "SINGLE_USER",
"runtime_engine": "STANDARD",
"effective_spark_version": "16.0.x-gpu-ml-scala2.12",

[jjsnlee]

I've been having a lot better luck loading the g class, which has some good GPU options (g6e) so I think I'm going to give up on these p3 instances. But just some posts on Reddit make it seem like AWS is trying to direct traffic away from these? Though this post suggests they are trying to that via costs, not availability...

[Walter_C]

I understand thanks for sharing.

[Walter_C]

This is failing to you with same error?

[tyagip1]

Yes, That's correct!

[Walter_C]

I was able to start a cluster with same exact configuration on my internal environment with no issues, I have selected east-1a as the AZ to deploy.

By any chance have you engaged AWS support on this?