Hi all,
Just wanted to ask if there is an announcement regarding changes on Databricks REST API?
Because I had weird experiences in using the REST API.
Specifically for Query History API
Last Aug 29, 2024.
I created a script to pull get request in the endpoint. I saw a key in the response named:
'query_source': {'driver_info': {'driver_name': '', 'version_number': '2.0', etc..'},
I can get all the response but not with that specific key. (use case is to determine where it is used ex:NB, Genie space, etc.)
I checked the query params, but there is nothing related to query_source. I even also enabled metrics, still no luck.
But then with my playful mind, I added this on my parameters: "include_query_source": 'true', - which is not part of the documentation - and you guess it, it worked.
Now as I am writing this, the 'query_source' is now missing in the response sample in the documentation. But still works.
It is better if we have and announcement regarding the changes of the endpoints because this might cause issues in the future when we decide to create a workflow for this kind of use case.