Databricks Community

AlbertWang

Hi all,I created an Azure Databricks workspace with the below featuresDeployed in my Azure virtual network (VNet injection)Enabled Secure cluster connectivityEnabled the Back-end Private LinkI did not create and use a NAT. I found that I still can ac...

AlbertWang · 08-08-2024

The problemAfter setting up Unity Catalog and a managed Volume, I can upload/download files to/from the volume, on Databricks Workspace UI.However, I cannot access the volume from notebook. I created an All-purpose compute, and run dbutils.fs.ls("/Vo...

AlbertWang · 08-07-2024

We have set up Azure Databricks with Unity Catalog (Metastore).Used Managed Identity (Databricks Access Connector) for connection from workspace(s) to ADLS Gen2ADLS Gen2 storage account has Storage Blob Data Contributor and Storage Queue Data Contrib...

AlbertWang

@Volker @kunalmishra9 Tested the below element is not necessary.num_workers: 0

AlbertWang

@Volker @kunalmishra9 As answered by Ganesh Chandrasekaran , the below solution works. I tested. Btw, you need to keep both spark_conf and custom_tags. job_clusters: - job_cluster_key: job_cluster new_cluster: spark_v...

AlbertWang

Hi @szymon_dybczak Thank you for your reply.Do you mean "If you choose SCC option, there should be NAT gateway created"?I checked there was no NAT created. Moreover, I have no Azure Firewall, Load Balancer, and NAT gateway, but my compute still can c...

AlbertWang · 08-08-2024

I found the reason and a solution, but I feel this is a bug. And I wonder what is the best practice.When I enable the ADSL Gen2's Public network access from all networks as shown below, I can access the volume from a notebook.However, if I enable the...

AlbertWang · 08-08-2024

I found the reason and a solution, but I feel this is a bug. And I wonder what is the best practice.When I enable the ADSL Gen2's Public network access from all networks as shown below, I can access the volume from a notebook.However, if I enable the...

Databricks Community

User Stats

User Activity

Confusion about SCC, NAT, and Internet connection (Azure)

Azure Databricks Unity Catalog - Cannot access Managed Volume in notebook

Azure Databricks Unity Catalog - cannot access managed volume in notebook

Re: Asset Bundles cannot run job with single node job cluster

Re: Asset Bundles cannot run job with single node job cluster

Re: Confusion about SCC, NAT, and Internet connection (Azure)

Re: Azure Databricks Unity Catalog - cannot access managed volume in notebook

Re: Azure Databricks Unity Catalog - Cannot access Managed Volume in notebook