Databricks Community

Sugumar_Sriniva · 02-22-2022

Dear connections,I'm unable to run a shell script which contains scheduling a Cron job through init script method on Azure Data bricks cluster nodes.Error from Azure Data bricks workspace:"databricks_error_message": "Cluster scoped init script dbfs:/...

Sugumar_Sriniva · 03-23-2022

@Atanu Sarkar ,I m using the below shell script for trying to schedule the crontab in databricks node.crontab-setup-for-log-cleanup.shsudo -H -u root bash -c 'echo "$(echo '* */3 * * * sh /dbfs/FileStore/driver-logs-cleanup.sh' ; crontab -l 2>&1)" |...

Sugumar_Sriniva · 02-26-2022

Yes I'm able to make a cron entry in the running cluster's spark driver node through ssh.

Sugumar_Sriniva · 02-23-2022

Hi werners,I need to perform the cleanup of azure data bricks driver logs (std.out, std.err, log4j) from dbfs path every hour. to achieve this I'm trying to schedule one Cron job on data bricks driver node so that logs can be deleted every one hour....

Databricks Community

User Stats

User Activity

Data bricks cluster creation is failing while running the Cron job scheduling script through init script method from azure data bricks.

Re: Data bricks cluster creation is failing while running the Cron job scheduling script through init script method from azure data bricks.

Re: Data bricks cluster creation is failing while running the Cron job scheduling script through init script method from azure data bricks.

Re: Data bricks cluster creation is failing while running the Cron job scheduling script through init script method from azure data bricks.