topic issue with azure databricks workspace after we disable public network access in Data Engineering

issue with azure databricks workspace after we disable public network access

Ravikumashi — Fri, 12 Apr 2024 13:59:01 GMT

Hi All,

We had azure databricks workspaces created thru terraform with public network access enabled to true and everything was working great. recently we have disabled the public network access and started to face issues.

terraform is uanble to add user/admins to workspace and giving us weird errors.

1. cannot read service principal: invalid charecter '<' begining of the value

2. cannot read workspace config: invalid charecter '<' begining of the value

we are using azurerm provider version 3.35.0 and databricks provider 1.7.0

And also we have private end points enabled for azure databricks workspace and it should go by private network.

Any help will be greatly appreaciated.

Re: issue with azure databricks workspace after we disable public network access

koushiknpvs — Sat, 13 Apr 2024 17:27:21 GMT

Were you able to add the needed up addresses to the list in the networking settings?

Re: issue with azure databricks workspace after we disable public network access

Ravikumashi — Mon, 15 Apr 2024 08:38:30 GMT

we have created a private endpoint for our azure databricks workspace is this not enough. Do we need to also explictly set up setwork security rule to allow traffic from private endpoint to databricks workspace.

Also required NSG rules for workspace is set to 'No Azure Databricks Rules' do we need to change this to 'All Rules' ?.

Could you elaborate on this please ?.

Re: issue with azure databricks workspace after we disable public network access

Ravikumashi — Mon, 15 Apr 2024 18:57:43 GMT

we use the following code to create private endpoint and on UI we can see the private endpoint connection status as approved.

resource "azurerm_private_endpoint" "example" {
  name                = "example-endpoint"
  location            = azurerm_resource_group.example.location
  resource_group_name = azurerm_resource_group.example.name
  subnet_id           = azurerm_subnet.endpoint.id

  private_service_connection {
    name                           = "example-privateserviceconnection"
    private_connection_resource_id = azurerm_private_link_service.example.id
    is_manual_connection           = false
  }
}

After we disable public network access databricks provider is unable to add users/service principal to the azure daatbricks workspace.

@koushiknpvs

Re: issue with azure databricks workspace after we disable public network access

Mohamednazeer — Wed, 04 Sep 2024 15:29:10 GMT

Do you got any solution for this issue? I am also facing the same issue.