cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

samruddhi
by New Contributor
  • 1123 Views
  • 1 replies
  • 0 kudos

Issue while creating Workspace in databricks using AWS

I am trying to configure databricks with AWS, I have configured the cloud resources as described in this https://docs.databricks.com/administration-guide/account-api/iam-role.html#language-Databricks%C2%A0VPC I have selected "Your VPC Default" as the...

image.png
  • 1123 Views
  • 1 replies
  • 0 kudos
Latest Reply
Abishek
Valued Contributor
  • 0 kudos

@samruddhi ChitnisCan you please check the below troubleshooting guide : Credentials configuration error messages: Malformed request: Failed credential configuration validation checksThe list of permissions checks in the error message indicate the li...

  • 0 kudos
amitca71
by Contributor II
  • 2361 Views
  • 6 replies
  • 0 kudos

Customer-managed VPC with empty default security group

Hi,I use self managed VPC. when i use security group the jobs clusters are being created.if i make the default security group empty and create customized security group (required by soc2 to have empty default), with same definition of ingress (allow...

  • 2361 Views
  • 6 replies
  • 0 kudos
Latest Reply
jose_gonzalez
Moderator
  • 0 kudos

Hi @Amit Cahanovich​,Just a friendly follow-up. Did any of the responses help you to resolve your question? if it did, please mark it as best. Otherwise, please let us know if you still need help.

  • 0 kudos
5 More Replies
438037
by New Contributor
  • 670 Views
  • 0 replies
  • 0 kudos

Databricks VPC - EKS VPC security groups

Hi,We have a databricks deployment in our AWS account in a dedicated VPC which we created a VPC peering to our EKS VPC, in the EKS main security group we added a rule that opens all TCP ports from the Databricks VPC and now it's working. Once I try t...

  • 670 Views
  • 0 replies
  • 0 kudos
ao1
by New Contributor III
  • 969 Views
  • 2 replies
  • 2 kudos

VPC status is BROKEN

Hi,AllWhen I check cloud resources, the VPC status is BROKEN.However, the cluster is runnning without any problems.What is the BROKEN state?And how can I get it healthy?Regards.

  • 969 Views
  • 2 replies
  • 2 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 2 kudos

Hi @aoi inanaga​, Please restart your cluster and check how it's working.

  • 2 kudos
1 More Replies
junaid
by New Contributor
  • 878 Views
  • 1 replies
  • 0 kudos

Issue in creating workspace - Custom AWS Configuration

Hello,We have tried to created new Databricks Workspace using "Custom AWS Configuration" and we have given our own VPC (Customer managed VPC) and tried but workspace failed to launch. There are Documents Available for resolve this error , According t...

  • 878 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @Junaid Ahmed​, If you are having network setup issues, you could choose to create the workspace with a Databricks-managed VPC rather than a customer-managed VPC.ImportantYou must choose whether to provide a customer-managed VPC when creating your...

  • 0 kudos
Anonymous
by Not applicable
  • 4060 Views
  • 9 replies
  • 2 kudos

Resolved! Issue in creating workspace - Custom AWS Configuration

We have tried to create new workspace using "Custom AWS Configuration" and we have given our own VPC (Customer managed VPC) and tried but workspace failed to launch. We are getting below error which couldn't understand where the issue is in.Workspace...

  • 4060 Views
  • 9 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

@Mitesh Patel​ - As Atanu thinks the issue may be resolved, I wanted to check in with you, also. How goes it?

  • 2 kudos
8 More Replies
Anonymous
by Not applicable
  • 1668 Views
  • 2 replies
  • 3 kudos

Issue in creating workspace - Custom AWS Configuration

We have tried to create new workspace using "Custom AWS Configuration" and we have given our own VPC (Customer managed VPC) and tried but workspace failed to launch. We are getting below error which couldn't understand where the issue is in.Workspace...

  • 1668 Views
  • 2 replies
  • 3 kudos
Latest Reply
Mitesh_Patel
New Contributor III
  • 3 kudos

I'm also getting the same issue. I'm trying to create a E2 workspace using Terraform with Customer-managed VPC in us-east-1 (using private subnets for 1a and 1b). We have 1 network rule attached to our subnets that looks like this:  Similar question ...

  • 3 kudos
1 More Replies
cig0
by New Contributor II
  • 3492 Views
  • 6 replies
  • 2 kudos

Resolved! AWS VPC peering connection: can't make Databricks VPC reach our services on the accepter VPC

Hi,We followed this document (https://docs.databricks.com/administration-guide/cloud-configurations/aws/vpc-peering.html) describing how to establish a connection between two (or more) VPC in AWS, but so far we haven't been able to communicate with t...

  • 3492 Views
  • 6 replies
  • 2 kudos
Latest Reply
jose_gonzalez
Moderator
  • 2 kudos

Hi @Martin Cigorraga​ ,If Huaming's fully answered your question, would you be happy to mark their answer as best so that others can quickly find the solution?

  • 2 kudos
5 More Replies
MoJaMa
by Valued Contributor II
  • 1043 Views
  • 1 replies
  • 0 kudos
  • 1043 Views
  • 1 replies
  • 0 kudos
Latest Reply
MoJaMa
Valued Contributor II
  • 0 kudos

Yes.Currently supported paths are:BYO VPC -> New BYO VPCDatabricks-created VPC -> New Databricks-created VPC

  • 0 kudos
aladda
by Honored Contributor II
  • 1458 Views
  • 1 replies
  • 0 kudos
  • 1458 Views
  • 1 replies
  • 0 kudos
Latest Reply
aladda
Honored Contributor II
  • 0 kudos

For an optimal processing experience Databricks segregates the Spark application traffic from the management traffic to avoid Network contention. Spark application traffic communications between the Driver-Executor and the Executors themselves where ...

  • 0 kudos
MoJaMa
by Valued Contributor II
  • 721 Views
  • 1 replies
  • 0 kudos
  • 721 Views
  • 1 replies
  • 0 kudos
Latest Reply
MoJaMa
Valued Contributor II
  • 0 kudos

That’s only available at Premium and Enterprise SKUs in AWS.See the "Enterprise Security" section here:https://databricks.com/product/aws-pricing

  • 0 kudos
Anonymous
by Not applicable
  • 1000 Views
  • 1 replies
  • 0 kudos

Failed E2 Workspace Error Message

My workspace ended up in a FAILED workspace state with one of the following messages:INVALID_STATE: The maximum number of VPCs has been reached.INVALID_STATE: The maximum number of VPC endpoints has been reached.INVALID_STATE: The maximum number of a...

  • 1000 Views
  • 1 replies
  • 0 kudos
Latest Reply
sajith_appukutt
Honored Contributor II
  • 0 kudos

The maximum number of xxx has been reached indicates that you have hit the soft limits for some of the AWS resources in that region. These are mostly soft limits and you could file a request to AWS support team to increase this

  • 0 kudos
Labels