Programatic access to Files in Repos

[Jiri_Koutny]

Hi, we are testing the new Files support in Databricks repos. Is there a way how to programmatically read notebooks?

Thanks

[Jiri_Koutny]

Thanks @Kaniz Fatma​  for quick reply. Looking forward to the answers 🙂

[Jiri_Koutny]

We definitely work with Python files in Databricks Repos, not with local files.

[Sandeep]

@Jiri Koutny​ Alternatively, using export API, you can download it and use

curl -n -o example.scala \

'https://<databricks-instance>/api/2.0/workspace/export?path=/Users/user@example.com/ScalaExampleNotebook&direct_download=true'

Documentation https://docs.databricks.com/dev-tools/api/latest/workspace.html#export

[Jiri_Koutny]

Hi Sandeep, is it really possible to download files (not notebooks) from Repos using the export API? Last time I tried, only notebooks were exported and all files were skipped...

[User16871418122 · Accepted solution]

Hi @Jiri Koutny​ these files anyway should be synced to your remote repository (git, bitbucket, GitLab etc). The APIs from version control tools Git API for example might help you achieve what you want.

https://stackoverflow.com/questions/38491722/reading-a-github-file-using-python-returns-html-tags

#!/usr/bin/env python3
import base64
import requests
 
 
url = 'https://api.github.com/repos/{user}/{repo_name}/contents/{path_to_file}'
req = requests.get(url)
if req.status_code == requests.codes.ok:
    req = req.json()  # the response is a JSON
    # req is now a dict with keys: name, encoding, url, size ...
    # and content. But it is encoded with base64.
    content = base64.decodestring(req['content'])
else:
    print('Content was not found.')