Programatic access to Files in Repos

[Jiri_Koutny]

Hi, we are testing the new Files support in Databricks repos. Is there a way how to programmatically read notebooks?

Thanks

[Kaniz]

Hi @ Jiri Koutny! My name is Kaniz, and I'm the technical moderator here. Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question first. Or else I will get back to you soon. Thanks.

[Jiri_Koutny]

Thanks @Kaniz Fatma​  for quick reply. Looking forward to the answers 🙂

[Kaniz]

Hi @Jiri Koutny​ , Just wanted to understand if you need help with Python Repo files or just local python file? Can you please be more specific?

[Jiri_Koutny]

We definitely work with Python files in Databricks Repos, not with local files.

[Kaniz]

Thank you for sharing this.

[Sandeep]

@Jiri Koutny​ Alternatively, using export API, you can download it and use

curl -n -o example.scala \

'https://<databricks-instance>/api/2.0/workspace/export?path=/Users/user@example.com/ScalaExampleNotebook&direct_download=true'

Documentation https://docs.databricks.com/dev-tools/api/latest/workspace.html#export

[Jiri_Koutny]

Hi Sandeep, is it really possible to download files (not notebooks) from Repos using the export API? Last time I tried, only notebooks were exported and all files were skipped...

[User16871418122 · Accepted solution]

Hi @Jiri Koutny​ these files anyway should be synced to your remote repository (git, bitbucket, GitLab etc). The APIs from version control tools Git API for example might help you achieve what you want.

https://stackoverflow.com/questions/38491722/reading-a-github-file-using-python-returns-html-tags

#!/usr/bin/env python3
import base64
import requests
 
 
url = 'https://api.github.com/repos/{user}/{repo_name}/contents/{path_to_file}'
req = requests.get(url)
if req.status_code == requests.codes.ok:
    req = req.json()  # the response is a JSON
    # req is now a dict with keys: name, encoding, url, size ...
    # and content. But it is encoded with base64.
    content = base64.decodestring(req['content'])
else:
    print('Content was not found.')