kfoster
Databricks Partner

pipeline id for the run can be found using:

spark.conf.get("pipelines.id")