Databricks job fails while creating table.

[RajeshRK]

Hi Team,

The Databricks job fails with the below error while creating EXTERNAL table.

com.simba.spark.jdbc41.internal.apache.http.wire - Error running query: MetaException(message:Got exception: org.apache.hadoop.fs.azure.AzureException com.microsoft.azure.storage.StorageException: Public access is not permitted on this storage account.)

MetaException(message:Got exception: org.apache.hadoop.fs.azure.AzureException com.microsoft.azure.storage.StorageException: Public access is not permitted on this storage account.), Query: CREATE EXTERNAL TABLE 

Can you please suggest if there is any permission missing in WASB storage?

Regards,

Rajesh.

[Vivian_Wilfred]

Hi @Rajesh Kannan R​ , Can you verify the networking configuration on your storage account? Check the Firewall rule to see if public access is enabled for all networks. Enable all network and see if it resolves your issue.

If you have restricted public access then the workspace has to be NPIP and you must have all the required workspace traffic to be whitelisted.

https://docs.microsoft.com/en-us/azure/databricks/security/secure-cluster-connectivity

[RajeshRK]

@Vivian Wilfred

Hi Vivian,

Thank you for your quick update. Let me check the settings and update you.

[RajeshRK]

@Vivian Wilfred​ 

Hi Vivian,

As I checked the network settings for the storage account and it is set to "Enabled from all networks".

Regards,

Rajesh.

[Vivian_Wilfred]

@Rajesh Kannan R​ , try enabling the Blob public access to see if it resolves the issue. Look for this setting in Overview > Properties > Data lake Storage >

[RajeshRK]

@Vivian Wilfred​ 

Hi Vivian,

It is working when Blob public access is enabled, but as security concerns, our customers don't want to enable Blob public access. Is there any other way we can publish to databricks table without enabling blob public access?

[RajeshRK]

@Vivian Wilfred​ 

Hi Vivian,

Can you please let me know is there any other way to create a table without enabling blob public access?

Regards,

Rajesh.

[Vivian_Wilfred]

Hi @Rajesh Kannan R​ , Allow Blob Public Access means that a storage account allows public blob/container access levels on the containers. If this is disabled then it means all containers must be private. When containers are private then you must authenticate with key, sp, etc. Please validate the authentication for these containers.

https://azure.microsoft.com/en-in/updates/choose-to-allow-or-disallow-blob-public-access-on-azure-st...

[RajeshRK]

@Vivian Wilfred​ 

Hi Vivian,

Thank you for the update. I will check and get back to you.

Regards,

Rajesh.

[RajeshRK]

@Arjun Kaimaparambil Rajan​ 

Hi Arjun,

We use sasToken to authenticate blob storage.

Regards

Rajesh.

[Vartika]

Hey there @Rajesh Kannan R​ 

Hope all is well!

Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. 

We'd love to hear from you.

Thanks!