Arti
New Contributor III

You can use the cluster node initiaization script to grab the environment variable DB_CLUSTER_ID. Refer here https://docs.databricks.com/user-guide/clusters/init-scripts.html#environment-variables.

Also from the notebook , the following also works :

spark.conf.get("spark.databricks.clusterUsageTags.clusterName")

spark.conf.get("spark.databricks.clusterUsageTags.clusterId")

The entire list of spark properties is available in UI in Spark UI --> Environment tab.

Hope this helps!

View solution in original post