Hi @yvesbeutler here is a sample way I did it using databricks asset bundles for notebook tasksresources:
jobs:
chained_jobs:
name: chained-jobs
tasks:
- task_key: main
notebook_task:
notebook_path: /W...
Hi @HariPrasad1 here is a way to get the job list (note: works for non-serverless clusters)from dbruntime.databricks_repl_context import get_context
cluster_id = spark.conf.get("spark.databricks.clusterUsageTags.clusterId")
workspaceUrl = spark.conf...
Hi @HariPrasad1, if I understand correctly, is it to programmatically retrieve the Spark UI URL (which has access to the list of jobs) in a notebook when a job is running?
