Hello, I'm trying to mount my Databricks to my Azure gen 2 data lake to read in data from the container, but I get an error when executing this line of code:
dbutils.fs.mount(
source = "abfss://resumes@choisysresume.dfs.core.windows.net/",
mount_point = "/mnt/resumeSharepointdata", extra_configs = configs)
The error I get is: (Note: The "redacted" is me taking out sensitive information)
---------------------------------------------------------------------------
ExecutionError Traceback (most recent call last)
File <command-739828508581633>:10
1 #Granting Access and mounting the input to the storage
3 configs = {"fs.azure.account.auth.type": "OAuth",
4 "fs.azure.account.oauth.provider.type": "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider",
5 "fs.azure.account.oauth2.client.id": "redacted",
6 "fs.azure.account.oauth2.client.secret": "redacted",
7 "fs.azure.account.oauth2.client.endpoint": "https://login.microsoftonline.com/redacted/oauth2/token",
8 "fs.azure.createRemoteFileSystemDuringInitialization": "true"}
---> 10 dbutils.fs.mount(
11 source = "abfss://resumes@redacted.dfs.core.windows.net/",
12 mount_point = "/mnt/resumeSharepointdata", extra_configs = configs)
File /databricks/python_shell/dbruntime/dbutils.py:362, in DBUtils.FSHandler.prettify_exception_message.<locals>.f_with_exception_handling(*args, **kwargs)
360 exc.__context__ = None
361 exc.__cause__ = None
--> 362 raise exc
ExecutionError: An error occurred while calling o392.mount.
: java.lang.NullPointerException
at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.oauth2.AzureADAuthenticator.getTokenCall(AzureADAuthenticator.java:320)
at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.oauth2.AzureADAuthenticator.getTokenCall(AzureADAuthenticator.java:287)
at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.oauth2.AzureADAuthenticator.getTokenUsingClientCreds(AzureADAuthenticator.java:110)
at com.databricks.backend.daemon.dbutils.DBUtilsCore.verifyAzureOAuth(DBUtilsCore.scala:1092)
at com.databricks.backend.daemon.dbutils.DBUtilsCore.verifyAzureFileSystem(DBUtilsCore.scala:1104)
at com.databricks.backend.daemon.dbutils.DBUtilsCore.createOrUpdateMount(DBUtilsCore.scala:988)
at com.databricks.backend.daemon.dbutils.DBUtilsCore.$anonfun$mount$1(DBUtilsCore.scala:1036)
at com.databricks.logging.UsageLogging.$anonfun$recordOperation$1(UsageLogging.scala:555)
at com.databricks.logging.UsageLogging.executeThunkAndCaptureResultTags$1(UsageLogging.scala:650)
at com.databricks.logging.UsageLogging.$anonfun$recordOperationWithResultTags$4(UsageLogging.scala:671)
at com.databricks.logging.UsageLogging.$anonfun$withAttributionContext$1(UsageLogging.scala:412)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at com.databricks.logging.AttributionContext$.withValue(AttributionContext.scala:158)
at com.databricks.logging.UsageLogging.withAttributionContext(UsageLogging.scala:410)
at com.databricks.logging.UsageLogging.withAttributionContext$(UsageLogging.scala:407)
at com.databricks.backend.daemon.dbutils.FSUtils.withAttributionContext(DBUtilsCore.scala:68)
at com.databricks.logging.UsageLogging.withAttributionTags(UsageLogging.scala:455)
at com.databricks.logging.UsageLogging.withAttributionTags$(UsageLogging.scala:440)
at com.databricks.backend.daemon.dbutils.FSUtils.withAttributionTags(DBUtilsCore.scala:68)
at com.databricks.logging.UsageLogging.recordOperationWithResultTags(UsageLogging.scala:645)
at com.databricks.logging.UsageLogging.recordOperationWithResultTags$(UsageLogging.scala:564)
at com.databricks.backend.daemon.dbutils.FSUtils.recordOperationWithResultTags(DBUtilsCore.scala:68)
at com.databricks.logging.UsageLogging.recordOperation(UsageLogging.scala:555)
at com.databricks.logging.UsageLogging.recordOperation$(UsageLogging.scala:525)
at com.databricks.backend.daemon.dbutils.FSUtils.recordOperation(DBUtilsCore.scala:68)
at com.databricks.backend.daemon.dbutils.FSUtils.recordDbutilsFsOp(DBUtilsCore.scala:132)
at com.databricks.backend.daemon.dbutils.DBUtilsCore.mount(DBUtilsCore.scala:1030)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:380)
at py4j.Gateway.invoke(Gateway.java:306)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:195)
at py4j.ClientServerConnection.run(ClientServerConnection.java:115)
at java.lang.Thread.run(Thread.java:750)
What is the problem here? How could I go about fixing it?
