cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

AWS + Cluster with custom docker image = export metrics to CloudWatch?

ivanychev
Contributor II

Is there a working setup on setting up metrics export to CloudWatch while using custom docker images for cluster creation?

I've tried to set up the CloudWatch agent manually, but launching `amzon-cloudwatch-agent-ctl` in the bootstrap script fails with

`sysv-init is not supported`

It is probably caused by the fact that it's launched in a Docker container, and it's not using `systemd`.

Sergey
3 REPLIES 3

Atanu
Databricks Employee
Databricks Employee

I am not sure, if this works with custom docker (probably not.)

But should work with normal cluster.

Integration Steps is adding as a text file.Please take a look @Sergey Ivanychevโ€‹ 

Thanks! My question is specifically about custom docker. I managed to set up CloudWatch for a normal cluster myself.

Sergey

Atanu
Databricks Employee
Databricks Employee

We do not support gangila with custom docker too. but let me cross verify if we are supporting cloudwatch for the same. Sorry for the inconvenience @Sergey Ivanychevโ€‹ 

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group