Notebook path can't be in DBFS?

User16790091296 — Thu, 24 Jun 2021 15:15:30 GMT

Some of us are working with IDEs and trying to deploy notebooks (.py) files to dbfs. the problem I have noticed is when configuring jobs, those paths are not recognized.

notebook_path:

If I use this :

dbfs:/artifacts/client-state-vector/0.0.0/bootstrap.py.

Only absolute paths are currently supported. Paths must begin with '/'."

If I use this;

/dbfs/artifacts/client-state-vector/0.0.0/bootstrap.py or /artifacts/client-state vector/0.0.0/bootstrap.py

I get Notebook not found.

what could be the issue here?

I see from the Databricks's architecture that Notebooks are in Microsoft managed subscription whereas DBFS is in Customer's subscription. Could that be the reason (that Notebook task is only able to pick from the microsoft managed subscription)? e.g. The folders I created on the workspace level where I have some notebooks, do not show up in the DBFS browser, so I am beginning to think that could be the reason.

Re: Notebook path can't be in DBFS?

User16752239289 — Thu, 08 Jul 2021 21:49:49 GMT

The issue is that the python file saved under DBFS not as a workspace notebook.

When you given /artifacts/client-state vector/0.0.0/bootstrap.py, the workspace will search the notebook(python file in this case) under the folder that under Workspace tab in the UI. If you prefer, this way, you can either create a folder under Workspace tab or upload to any exist folder and then import your python file there.

The folder "/dbfs/artifacts/client-state-vector/0.0.0/bootstrap.py" is only valid to be used in cluster. The /dbfs is mount point that each cluster has and allow the cluster to access the DBFS filesystem as local file.

topic Notebook path can't be in DBFS? in Data Engineering

Notebook path can't be in DBFS?

Re: Notebook path can't be in DBFS?