Connecting an Azure Databricks to Azure Gen 2 storage stuck on "Running Command..."

jborn
New Contributor III

‎01-11-2024 09:14 AM

I recently had an Azure Databricks setup done behind a VPN.  I'm trying to connect to my Azure Storage Account Gen 2  Using the following code I haven't been able to connect and keep getting stuck on reading the file.  What should I be checking?  

 

#initialize the source path and create the bronze df 
SourceFilePath = f"abfss://" + PersistedContainer + "@" + AzureBlobStorageAccountName + ".dfs.core.windows.net/" + PersistedDirectory + "/" + PersistedFilename
print(SourceFilePath)
#Path prints out as expected
spark.conf.set(f"fs.azure.account.key." + AzureBlobStorageAccountName + ".dfs.core.windows.net","Key removed")
dfBronze = spark.read.load(SourceFilePath,format='csv',header = True).cache()

 

 

When I get to the spark.read.load line I get stuck on "Running Command..." 
Any commands I can use to get debug information on what is happening?
Using the shell command below resolves to the correct IP.

 

%sh nslookup AzureBlobStorageAccountName.dfs.core.windows.net​

 

0 Kudos