topic AWS + Cluster with custom docker image = export metrics to CloudWatch? in Data Engineering

AWS + Cluster with custom docker image = export metrics to CloudWatch?

ivanychev — Wed, 25 May 2022 01:26:16 GMT

Is there a working setup on setting up metrics export to CloudWatch while using custom docker images for cluster creation?

I've tried to set up the CloudWatch agent manually, but launching `amzon-cloudwatch-agent-ctl` in the bootstrap script fails with

`sysv-init is not supported`

It is probably caused by the fact that it's launched in a Docker container, and it's not using `systemd`.

Re: AWS + Cluster with custom docker image = export metrics to CloudWatch?

Atanu — Thu, 26 May 2022 09:59:14 GMT

I am not sure, if this works with custom docker (probably not.)

But should work with normal cluster.

Integration Steps is adding as a text file.Please take a look @Sergey Ivanychev

Re: AWS + Cluster with custom docker image = export metrics to CloudWatch?

ivanychev — Thu, 26 May 2022 10:01:17 GMT

Thanks! My question is specifically about custom docker. I managed to set up CloudWatch for a normal cluster myself.

Re: AWS + Cluster with custom docker image = export metrics to CloudWatch?

Atanu — Thu, 26 May 2022 10:06:49 GMT

We do not support gangila with custom docker too. but let me cross verify if we are supporting cloudwatch for the same. Sorry for the inconvenience @Sergey Ivanychev