topic Re: Is DBFS going to be deprecated? in Get Started Discussions

Is DBFS going to be deprecated?

ArvindDige — Tue, 02 Jul 2024 13:31:12 GMT

Is DBFS going to be deprecated? As I am using /dbfs/FileStore/tables/ location where a jar file is stored, and I am copying this jar file to /databricks/jars locations.

My concerns is as DBFS root and mounts are deprecated, is that mean in coming days the dbfs is going to be removed from Databricks itself? Please find attached screenshot(dbfs.png) about for reference. If this is the case then I have to migrate my source path (/dbfs/FileStore/tables/) of Jar to some other location.

One of the scenario I am testing is storing the jar in Storage Account and copying it into /databricks/jars/ location, but it is not allowing me to copy the same jar in /databricks/jars location. Can someone help me on this scenario.

Re: Is DBFS going to be deprecated?

raphaelblg — Tue, 02 Jul 2024 14:42:36 GMT

Hi @ArvindDige,

I would let the .jar file in your workspace files or in the Storage Account. When you say "it is not allowing me to copy the same jar in /databricks/jars location" what specifically do you mean? are you getting any errors?

Re: Is DBFS going to be deprecated?

ArvindDige — Wed, 03 Jul 2024 09:26:49 GMT

Hi Raphael,

I am trying below init script to achieve this task, PFA

And getting error as below,

Cluster scoped init script abfss://container@storage.dfs.core.windows.net/init_script.sh failed: Failure to initialize configuration for storage account storage.dfs.core.windows.net: Invalid configuration value detected for fs.azure.account.key, Caused by: Invalid configuration value detected for fs.azure.account.key.

Apart from this my main question is, does databricks going to remove support for dbfs? can you please answer this as well.

Thanks,

Arvind Dige