I am running dbt on a databricks job. It saves all documentation: manifest.json, run_results.json, etc in "Download Artifacts" in a job. I am not able to find out a way to read those in codes, transform and save on databricks. 

Tried job API. The artifacts API: api/2.1/jobs/runs/get-output does return a link to read or download these artifacts, but it requires task ID as an input. No other API on databricks gives Task ID as output. There is no complete link for me to get these artifacts today.

Can someone pl help?