Git Integration with Databricks Query Files and Azure DevOps

dbengineer516
New Contributor III

I’ve been trying to develop a solution for our team to be able to have Git integration between Databricks and Azure DevOps. However, the “query” file type/workspace item on Databricks can’t be committed and pushed to a Git repo, only regular file types and notebooks. My thought was to automate a process/create a Python script that would be able to go into a workspace, along with its subfolders, and change over query files into .sql files so that they can be pushed to a Git repo. I’ve tried accomplishing this by using get /api/2.0/workspace/list, but this doesn’t report query object types, so then I tried using get /api/2.0/preview/sql/queries to first list all query files and then specify a parent folder in a function to pass along to convert query files in a specified parent folder. However, after trying this on my home directory first, the API call would only return back certain query files and walk through some subfolders and not others. I was wondering if there is a better and more efficient solution for this and how to go about accomplishing this so that we can easily commit query files to a Git repo/convert query files into .sql files so that they can be committed.

Thank you!