topic Re: Data bricks cluster creation is failing while running the Cron job scheduling script through init script method from azure data bricks. in Data Engineering

Data bricks cluster creation is failing while running the Cron job scheduling script through init script method from azure data bricks.

Sugumar_Sriniva — Tue, 22 Feb 2022 11:36:53 GMT

Dear connections,

I'm unable to run a shell script which contains scheduling a Cron job through init script method on Azure Data bricks cluster nodes.

Error from Azure Data bricks workspace:

"databricks_error_message": "Cluster scoped init script dbfs:/<shell-script> failed: Script exit status is non-zero"

Looking for a method to achieve the same.

Re: Data bricks cluster creation is failing while running the Cron job scheduling script through init script method from azure data bricks.

Anonymous — Tue, 22 Feb 2022 18:22:54 GMT

Hello @Sugumar Srinivasan - Welcome to the community! It's nice to meet you. My name is Piper, and I'm a moderator for Databricks.

Let's give the other members a chance to answer your question. We'll come back to this if we need to. 🙂

Re: Data bricks cluster creation is failing while running the Cron job scheduling script through init script method from azure data bricks.

-werners- — Wed, 23 Feb 2022 07:53:28 GMT

I do not completely understand what you are trying to do?

Are you trying to schedule a cron job on the cluster workers?

Re: Data bricks cluster creation is failing while running the Cron job scheduling script through init script method from azure data bricks.

Sugumar_Sriniva — Wed, 23 Feb 2022 13:08:16 GMT

Hi werners,

I need to perform the cleanup of azure data bricks driver logs (std.out, std.err, log4j) from dbfs path every hour. to achieve this I'm trying to schedule one Cron job on data bricks driver node so that logs can be deleted every one hour. While using below script in init, the azure databricks cluster creation is failing.

The shell script contains below content:

dbfs:/FileStore/crontab-setup-for-log-cleanup.sh

sudo -H -u root bash -c 'echo "$(echo "* */5 * * * sh /dbfs/FileStore/driver-logs-cleanup.sh" ; crontab -l 2>&1)" | crontab -'

Re: Data bricks cluster creation is failing while running the Cron job scheduling script through init script method from azure data bricks.

-werners- — Wed, 23 Feb 2022 13:15:19 GMT

Does the script work while the cluster is running, to exclude a issue with the script itself?

Re: Data bricks cluster creation is failing while running the Cron job scheduling script through init script method from azure data bricks.

Tasootoor1998 — Sat, 26 Feb 2022 10:17:14 GMT

DataBricks Cluster Creation is failing while running the cron job scheduling script through init Script Method from Azure Data Bricks

Re: Data bricks cluster creation is failing while running the Cron job scheduling script through init script method from azure data bricks.

Sugumar_Sriniva — Sat, 26 Feb 2022 17:56:09 GMT

Yes I'm able to make a cron entry in the running cluster's spark driver node through ssh.

Re: Data bricks cluster creation is failing while running the Cron job scheduling script through init script method from azure data bricks.

Atanu — Tue, 08 Mar 2022 14:12:35 GMT

the issue is definitely the init script.Please cross check the init script or you can post it here if no sensitive info. we can cross verify.

Re: Data bricks cluster creation is failing while running the Cron job scheduling script through init script method from azure data bricks.

Atanu — Wed, 16 Mar 2022 05:07:02 GMT

@Sugumar Srinivasan are you able to check the init script to rectify the issue?

Re: Data bricks cluster creation is failing while running the Cron job scheduling script through init script method from azure data bricks.

Sugumar_Sriniva — Wed, 23 Mar 2022 07:04:24 GMT

@Atanu Sarkar ,

I m using the below shell script for trying to schedule the crontab in databricks node.

crontab-setup-for-log-cleanup.sh

sudo -H -u root bash -c 'echo "$(echo '* */3 * * * sh /dbfs/FileStore/driver-logs-cleanup.sh' ; crontab -l 2>&1)" | crontab -'

driver-logs-cleanup.sh

#!/bin/bash

#Find the the older stdout, stderr and log4j files and deletes it accordingly.

find /dbfs/cluster-logs/<db-cluster-id>/driver/ -name "stdout--*" -exec rm -f {} \;

find /dbfs/cluster-logs/<db-cluster-id>/driver/ -name "stderr--*" -exec rm -f {} \;

find /dbfs/cluster-logs/<db-cluster-id>/driver/ -name "log4j""-""*"".gz" -exec rm -f {} \;

Thanks & Regards,

Sugumar Srinivasan.

Re: Data bricks cluster creation is failing while running the Cron job scheduling script through init script method from azure data bricks.

jose_gonzalez — Mon, 11 Apr 2022 18:20:51 GMT

Hi @Sugumar Srinivasan ,

Just a friendly follow-up. Are you still having issues with your init script or not any more? Please let us know.

Re: Data bricks cluster creation is failing while running the Cron job scheduling script through init script method from azure data bricks.

User16764241763 — Wed, 13 Apr 2022 16:09:44 GMT

Hello @Sugumar Srinivasan Could you please enable cluster log delivery and inspect the INIT script logs in the below path dbfs:/cluster-logs/<clusterId>/init_scripts path.

https://docs.databricks.com/clusters/configure.html#cluster-log-delivery-1