cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Capture Automatically Added tags

databricksdev
New Contributor II

Can we capture automatically added tags (ex: RunName) from azure data bricks job cluster to parameters or custom tags in azure data factory

1 REPLY 1

Kaniz
Community Manager
Community Manager

Hi @databricksdev

Azure Databricks applies default tags to each cluster, including Vendor, Creator, ClusterName, and ClusterId. In addition, it applies two default tags on job clusters: RunName and JobId1. However, these tags are only applied to DBU usage reports for clusters launched from pools and do no...1.

As for Azure Data Factory, it can execute a Databricks notebook against the Databricks jobs cluster ...2. However, automatically capturing added tags from Azure Databricks job clusters to parameters or custom tags in Azure Data Factory is not directly supported.

You might consider creating a custom solution to capture these tags. For example, you could potentially use Azure Databricks APIs to retrieve the tags from a running job and then use Azure Data Factory’s Web Activity or Azure Functions to store these tags in a place where they can be accessed by Azure Data Factory.