topic Re: Does Databricks persist the alert results history? in Data Engineering

Does Databricks persist the alert results history?

Nick_Hughes — Wed, 10 Nov 2021 10:32:35 GMT

Hi, this seems like a really basic feature to have, an alert is generated and spits out the email, but the URL doesn't take you to the events list that happened at that time but just the query in the editor (via the alert config screen). We're really struggling to make use out of this feature with no persisted history and no way seemingly to format the alert in to a table of results. Please can I request the persisting of historic alert events as a feature request or can someone let me know if im just not seeing it? Thank you

Re: Does Databricks persist the alert results history?

Prabakar — Wed, 10 Nov 2021 10:35:03 GMT

Hi @Nick Hughes , for the feature request you can raise it in our ideas portal so that it will have direct visibility to the product team and develop in the future.

Re: Does Databricks persist the alert results history?

Nick_Hughes — Wed, 10 Nov 2021 10:57:04 GMT

@Prabakar Ammeappin does this mean it's definitely not a current feature and history is lost?

Re: Does Databricks persist the alert results history?

Hubert-Dudek — Wed, 10 Nov 2021 10:59:42 GMT

You mean alerts for jobs runs?

If yes you can write own script to handle that as API for jobs have information related to alerts like timeout or status failed https://docs.databricks.com/dev-tools/api/latest/jobs.html#operation/JobsRunsList

On Azure when you run databricks notebooks via Data Factory it have really nice feature to manage failure state and you can create own alerts (via Logic Apps)

Re: Does Databricks persist the alert results history?

Nick_Hughes — Wed, 10 Nov 2021 11:09:28 GMT

Hiya,

No, not for job runs, for the business alerting feature off scheduled queries

Re: Does Databricks persist the alert results history?

Prabakar — Wed, 10 Nov 2021 11:22:55 GMT

@Nick Hughes Do you mean the dbsql queries and alerts?

Re: Does Databricks persist the alert results history?

Nick_Hughes — Wed, 10 Nov 2021 11:33:50 GMT

This one @Prabakar Ammeappin -https://docs.databricks.com/sql/user/alerts/index.html

Re: Does Databricks persist the alert results history?

Hubert-Dudek — Wed, 10 Nov 2021 11:53:58 GMT

aaa now I get it. The best option would be that Databricks add SQL alerts history to SQL API https://docs.databricks.com/sql/api/index.html

Re: Does Databricks persist the alert results history?

Prabakar — Wed, 10 Nov 2021 13:04:33 GMT

@Nick Hughes For Alerts, I don't think we have a persistent history. The SQL API mentioned by @Hubert Dudek works for the SQL query history like status, time range, endpoint, user id. But it won't give you the history of the alerts that you set. So for the SQL alert history, I would suggest you raise a feature request.

https://docs.databricks.com/sql/api/query-history.html#queryfilter

Re: Does Databricks persist the alert results history?

Prabakar — Wed, 10 Nov 2021 18:50:31 GMT

@Nick Hughes I was looking at our ideas portal and could see an API feature was requested for the Alerts (DB-I-4289). To create an API for the alerts and pull data, we might need to keep the alert history persistent. So this feature should suffice your requirement.

You can refer to the below post:

https://community.databricks.com/s/question/0D53f00001SAWH5CAP/is-there-an-alerting-api-please