topic Read notebooks programmatically from databricks python notebook in Get Started Discussions

Read notebooks programmatically from databricks python notebook

evgvain — Wed, 20 Dec 2023 08:25:38 GMT

I need programmatically read the content of the specific notebook in my databricks workspace from another notebook in the same workspace. Following the databricks documentation, I extract the path to my notebook and then list all other notebooks in the directory. This works as expected.

notebook_path = os.path.dirname(dbutils.entry_point.getDbutils().notebook().getContext().notebookPath().getOrElse(None)) for f in dbutils.fs.ls(f"file:/Workspace{notebook_path}"): print(f)

For example: FileInfo(path='file:/Workspace/Users/***/Notebook1', name='Notebook1', size=0, modificationTime=0)

Then I try to read a specific notebook:

with open(f"/Workspace{notebook_path}/Notebook1", "r") as _f: c = _f.read()

I get the following error:

OSError: [Errno 95] Operation not supported File /databricks/python/lib/python3.10/site-packages/IPython/core/interactiveshell.py:282, in _modified_open(file, *args, **kwargs) 275 if file in {0, 1, 2}: 276 raise ValueError( 277 f"IPython won't let you open fd={file} by default " 278 "as it is likely to crash IPython. If you know what you are doing, " 279 "you can use builtins' open." 280 ) --> 282 return io_open(file, *args, **kwargs)

Re: Read notebooks programmatically from databricks python notebook

evgvain — Wed, 20 Dec 2023 10:29:24 GMT

I was able to do this by using databricks.sdk.WorkspaceClient. However, I looks like a workaround. I would prefer to read the content of notebooks directly.

databricksURL = dbutils.notebook.entry_point.getDbutils().notebook().getContext().apiUrl().getOrElse(None) myToken = dbutils.notebook.entry_point.getDbutils().notebook().getContext().apiToken().getOrElse(None) w = WorkspaceClient(host=databricksURL, token=myToken) export_response = w.workspace.export(path, format=ExportFormat.JUPYTER)

Any advice appriciated.

Re: Read notebooks programmatically from databricks python notebook

jenshumrich — Wed, 13 Mar 2024 10:28:16 GMT

Thanks a lot for sharing your code snippet. Helped me a lot to copy some notebooks into our fileshare.

from databricks.sdk import WorkspaceClient

from databricks.sdk.service import workspace

import base64

def notebook_reader(file_location😞

databricksURL = dbutils.notebook.entry_point.getDbutils().notebook().getContext().apiUrl().getOrElse(None)

myToken = dbutils.notebook.entry_point.getDbutils().notebook().getContext().apiToken().getOrElse(None)

w = WorkspaceClient(host=databricksURL, token=myToken)

export_response = w.workspace.export(file_location, format=workspace.ExportFormat.SOURCE)

return base64.b64decode(export_response.content)