Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-07-2025 10:37 AM
You're right to be concerned about users reactivating the cluster after your scheduled termination job. Looking more closely at Databricks cluster policies, there's no direct way to restrict cluster activation based on time of day - this is a limitation in the current Databricks platform.
While you can't directly set time restrictions in policies, you can implement a workaround:
- Create a cluster policy with an init script requirement:
{ "spark_conf.spark.databricks.cluster.profile": { "type": "fixed", "value": "singleNode" }, "custom_tags.BusinessHours": { "type": "fixed", "value": "true" }, "init_scripts": { "type": "fixed", "value": [{"dbfs": {"destination": "dbfs:/scripts/business-hours-check.sh"}}] } }
2. Create an init script that checks the current time and terminates if outside business hours:
bash
#!/bin/bash # Define business hours (in 24hr format) START_HOUR=8 END_HOUR=18 # Get current hour CURRENT_HOUR=$(date +%H) CURRENT_DAY=$(date +%u) # 1-5 is Monday-Friday # Check if outside business hours (or weekend) if [ $CURRENT_DAY -gt 5 ] || [ $CURRENT_HOUR -lt $START_HOUR ] || [ $CURRENT_HOUR -ge $END_HOUR ]; then echo "Outside business hours. Terminating cluster." # Signal termination through the API curl -X POST -H "Authorization: Bearer $DATABRICKS_TOKEN" \ https://<databricks-instance>/api/2.0/clusters/delete \ -d "{\"cluster_id\": \"$DATABRICKS_CLUSTER_ID\"}" fi
LR