DBR 11.3 LTS - Specify spark.app.id for python notebook task
Hi, I use Terraform databricks_job to submit a job with many tasks. IFAIK, each task is a separated Spark Application with a specific applicationId (https://spark.apache.org/docs/3.1.3/api/python/reference/api/pyspark.SparkContext.applicationId.html)...
- 607 Views
- 0 replies
- 0 kudos