Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-16-2022 11:49 AM
I have several users doing data analysis on Databricks Spark notebooks, everything is smooth, now I want to make sure that the checkpointdir is configured on the cluster start, so every user doesn't had to set it on the Notebook (ending up in a lot of similar paths)
To do this:
SparkContext.checkpointFile(directory: String)I had to have a context the ideal will be to set it here, on cluster configuration:
How can I achieve this?
Thanks!
Labels:
- Labels:
-
Checkpoint
-
Databricks notebook