topic Re: Global init script not running in Data Engineering

Global init script not running

vjani — Tue, 06 Aug 2024 22:00:54 GMT

Hello Databricks Community,

I am trying to connect databricks with datadog and have added datadog agent script in global init but it did not worked. Just to make sure if init script is working or not I have added below two lined of code in global init script.

#!/bin/bash echo " ============== init_script ========="

After that I have restarted cluster and I do not see any output in Cluster > my-cluster> "Driver Logs" nor I am seeing any INIT_SCRIPTS_STARTED in "Event Log". What am I missing? How do I check if script files were executed or not?

Runtime:

14.3 LTS (includes Apache Spark 3.5.0, Scala 2.12)

Re: Global init script not running

szymon_dybczak — Wed, 07 Aug 2024 11:12:09 GMT

Hi @vjani ,

Global init scripts only run on clusters configured with single user or legacy no-isolation shared access mode, so Databricks recommends configuring all init scripts as cluster-scoped init scripts and managing them across your workspace using cluster policies.
So the question is, what access mode you are using in your cluster?

Use global init scripts | Databricks on AWS

Re: Global init script not running

Retired_mod — Thu, 08 Aug 2024 06:37:19 GMT

Hi @vjani, Thanks for reaching out! Please review the response and let us know if it answers your question. Your feedback is valuable to us and the community.

If the response resolves your issue, kindly mark it as the accepted solution. This will help close the thread and assist others with similar queries.

We appreciate your participation and are here if you need further assistance!

Re: Global init script not running

vjani — Fri, 09 Aug 2024 20:05:05 GMT

Thanks Slash for the reply. That seems to be a reason. I was following https://docs.datadoghq.com/integrations/databricks/?tab=driveronly and missed that configuration.

Re: Global init script not running

szymon_dybczak — Fri, 09 Aug 2024 21:06:58 GMT

No problem, glad that it was helpful to you.