cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Capture events such as Start, Stop and Terminate of cluster.

sonalitotade
New Contributor II

Hi,

I am using databricks with AWS.

I need to capture events such as Start, Stop and Terminate of cluster and perform some other action based on the events that happened on the cluster.

Is there a way I can achieve this in databricks?

2 REPLIES 2

daniel_sahal
Honored Contributor III

@sonali totade​ 

I see two ways of doing that:

  • Creating a custom app that will monitor Clusters API (list and get) and store their status somewhere, then trigger an action on new or status change,
  • Enable audit logging (https://docs.databricks.com/administration-guide/account-settings/audit-logs.html) and use CloudWatch or any other log analysis tool to create custom events.

sonalitotade
New Contributor II

Hi Daniel,

thanks for the response

I would like to know if we can capture the event logs as shown in the image below when an event occurs on the cluster.

image

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.