Unsupported cloud file system schema 'wasbs' error in custom catalog, but not in hive_metastore

[DianGermishuiz1]

When trying to run a create table USING CSV that pulls data from an azure blob storage path in a custom catalog I created, I get an error stating Unsupported cloud file system schema 'wasbs'.

However, when I run this code in the hive_metastore catalog, it runs fine.

What permissions do I need to set on my custom catalog to allow the code to succeed?

The blob storage path is provided in the Databricks SQL Analyst training course - https://partner-academy.databricks.com/learn/course/1035/play/5293/getting-started-course-setup;lp=7...

Image of the error:

[Debayan]

Hi, this has to be abfss:// (The legacy Windows Azure Storage Blob driver (WASB) has been deprecated. ABFS has numerous benefits over WASB; see Azure documentation on ABFS. For documentation for working with the legacy WASB driver, see Connect to Azure Blob Storage with WASB (legacy).)

[DianGermishuiz1]

Hello,

Tried changing the path, however i get a new error

"No parent external location found for path 'abfs://courseware@dbacademy.dfs.core.windows.net/data-analysis-with-databricks/v01/flights/departuredelays.csv'"

Is it perhaps that the files are still stored in a standard blob storage and not in a Gen2 Data Lake ?

[Anonymous]

Hi @Dian Germishuizen​ 

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. 

We'd love to hear from you.

Thanks!

[DianGermishuiz1]

Hi @Vidula Khanna​ ,

No resolution as yet.

Tried changing the path, however i get a new error

"No parent external location found for path 'abfs://courseware@dbacademy.dfs.core.windows.net/data-analysis-with-databricks/v01/flights/departuredelays.csv'"

Is it perhaps that the files are still stored in a standard blob storage and not in a Gen2 Data Lake ?

[Hemant · Accepted solution]

Hello @Dian Germishuizen,

It appears that the wasbs or abfss protocol is not functioning properly in the SQL editor. However, you can still utilize these protocols to create a mount point in the data engineer persona using a notebook. Once the mount point is created, you can use it in the path option to create the table.

I hope this helps! Let me know if you have any further questions.

[DianGermishuiz1]

Hi Hemant,

Thank you for the reply. I will try this out and see how it goes.

@Vidula Khanna​  can you please pass this message on to Databricks Product / Support team to update the SQL Analyst learning path training to indicate that you can only do it in the default hive_metastore catalog, not a custom catalog.

Also to just fix whatever is causing the problem in custom catalogs.

Thank you

[youssefmrini]

WASB Has been deprecated.