- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-02-2025 01:22 AM
In a workflow, is there a way to access task artifacts from within the run?
I have a job with a task TasksA, which is a dbt task that creates some artifacts. I want to store these artifacts, but the job artifacts seems to be saved in a location I cannot access?
So for now I have created a TaskB that takes in job.run_id as a parameter, uses the databricks cli to read job to get the tasks, then reads the task output which gives an artifact link, and then I download the artifacts from that link and save it to my catalog.
This feels like an overkill?? Is there a way that TaskB (within the same job as TaskA) can access the artifacts created from TaskA in an easier way than using the cli like that?
Any improvements to my current workflow is a win
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-02-2025 01:37 AM
Hi @rokata ,
I think you're doing it in correct way. As of now, there's no other way to download artifacts from the job run.
In documentation they're using the same approach as you:
dbt-databricks/docs/databricks-workflows.md at main · databricks/dbt-databricks · GitHub
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-02-2025 03:00 AM
Thank you for the response, and for the link!
It feels like it should be simpler, but if the docs go that way I guess that is the correct way.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-03-2026 06:15 AM
is there a better way to access artifacts now?