topic Re: Unsupported Azure Scheme: abfss in Data Engineering

Unsupported Azure Scheme: abfss

vroste — Thu, 09 Feb 2023 12:45:18 GMT

Using Databricks Runtime 12.0, when attempting to mount an Azure blob storage container, I'm getting the following exception:

`IllegalArgumentException: Unsupported Azure Scheme: abfss`

dbutils.fs.mount(
    source="abfss://container@my-storage-account.dfs.core.windows.net/",
    mount_point="/mnt/my-mount-point"
  )

I can succesfully list files using `dbutils.fs.ls`, it's just the mounting that does not work.

Re: Unsupported Azure Scheme: abfss

-werners- — Thu, 09 Feb 2023 13:25:47 GMT

don't you have to pass some config like fs.azure.account.auth.type etc?

Re: Unsupported Azure Scheme: abfss

vroste — Thu, 09 Feb 2023 13:30:38 GMT

I have done that, following https://docs.databricks.com/dbfs/mounts.html#mount-adls-gen2-or-blob-storage-with-abfs. Without credentials configs I get other errors.

Re: Unsupported Azure Scheme: abfss

-werners- — Thu, 09 Feb 2023 13:37:41 GMT

you did not activate Unity by any chance?

Re: Unsupported Azure Scheme: abfss

vroste — Thu, 09 Feb 2023 13:39:28 GMT

Not as far as I can tell, unless it's activated by default or something

Re: Unsupported Azure Scheme: abfss

-werners- — Thu, 09 Feb 2023 13:51:29 GMT

it is not activated by default.

I still believe it has something to do with your config as I can mount and unmount without any issues on 12.1

Do you pass all the necessary configuration as explained in the help?

configs = {"fs.azure.account.auth.type": "OAuth",
           "fs.azure.account.oauth.provider.type": "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider",
           "fs.azure.account.oauth2.client.id": <SP id>,
           "fs.azure.account.oauth2.client.secret": dbutils.secrets.get(scope = <SCOPE> , key = <KEY>),
           "fs.azure.account.oauth2.client.endpoint": "https://login.microsoftonline.com/<TENANT>/oauth2/token"}
  
dbutils.fs.mount(
  source = "abfss://container@storage.dfs.core.windows.net",
  mount_point = "/mnt/mount",
  extra_configs = configs)

I use databricks secrets to fetch a secret from Azure Key Vault.

Re: Unsupported Azure Scheme: abfss

vroste — Thu, 09 Feb 2023 14:26:14 GMT

You are right! I tweaked some configs and it works now. The exception message was confusing.

Thanks!

Re: Unsupported Azure Scheme: abfss

AdamRink — Mon, 13 Mar 2023 16:17:35 GMT

What configs did you tweak, having same issue?

Re: Unsupported Azure Scheme: abfss

acardosolima — Thu, 04 Jan 2024 02:09:21 GMT

In my case, I copied the same parameters used to access ADLS using a service principal, so client_id, client_secret and client_endpoint had ".<storage-account>.dfs.core.windows.net" in it. When removed, the code worked properly.