topic Re: Help trying to calculate a percentage in Data Engineering

Help trying to calculate a percentage

DanVartanian — Thu, 20 Jan 2022 12:05:33 GMT

The image below shows what my source data is (HAVE) and what I'm trying to get to (WANT).

I want to be able to calculate the percentage of bad messages (where formattedMessage = false) by source and date.

I'm not sure how to achieve this in DatabricksSql. Any help appreciated.

DanVartanian — Thu, 20 Jan 2022 15:40:21 GMT

Thank you so much

-werners- — Fri, 21 Jan 2022 08:28:02 GMT

you could use a windows function over source and date with a sum of messagecount. This gives you the total per source/date repeated on every line.

Then apply a filter on formattedmessage == false and divide messagecount by the sum above.

DanVartanian — Fri, 21 Jan 2022 14:09:00 GMT

Thank you, I was able to get it following your instructions😀