- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-06-2022 10:55 PM
Hi Team,
Is it possible to have a startup notebook in databricks similar to init sql script (
This need to run on start of every cluster )
- Labels:
-
Notebook
-
Start
-
Startup Notebook
Accepted Solutions
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-08-2022 02:40 PM
@somanath Sankaran , Running a notebook while the cluster is starting won't be possible. If your requirement is for auditing, you can configure the audit logs.
https://docs.databricks.com/administration-guide/account-settings/audit-logs.html
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-07-2022 03:43 AM
There is a way to do that using terraform but it will be complicated... https://github.com/databrickslabs/terraform-provider-databricks
If you need something less complicated: for jobs cluster is easy just set task on which all others will depend.
Another way which I think of is to set the azure function which runs every minute and checks for new running clusters and then triggers their notebook.
In Azure using event grid and logic apps you can also monitor new VM deployment ad than trigger notebook.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-07-2022 04:30 AM
@Hubert Dudek thanks but it will be helpful if Databricks provides something out of the box
as I am planning to set some startup tasks for auditing purpose (for eg secret detection etc ) .
Another easy way is to have %run on top of every notebook but devs can escape by not executing it
Also I need it on interactive cluster
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-08-2022 02:40 PM
@somanath Sankaran , Running a notebook while the cluster is starting won't be possible. If your requirement is for auditing, you can configure the audit logs.
https://docs.databricks.com/administration-guide/account-settings/audit-logs.html
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-18-2022 12:29 AM
@Kaniz Fatma I think we dont have the option to do the same in Databricks
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-26-2022 05:18 AM
@Kaniz Fatma sure will do

