Does Databricks have alerts / thresholds in place for cost monitoring?

[sajith_appukutt · Accepted solution]

As a Databricks account owner or account admin, you can configure daily delivery of billable usage logs in CSV file format to an AWS S3 storage bucket, where you can make the data available for further analytics.

A simple pipeline for configuring budget thresholds and alerting could look like this

1. Configure billable usage logs delivery to an s3 bucket.
2. Configure autoloader to load this to delta table
3. Configure dashboards in Databricks SQL for reporting
4. Configure alerts in Databricks SQL by scheduling a query that checks a threshold

[aladda]

You can also use tags to setup a chargeback mechanism within your organization for distributed billing - https://docs.databricks.com/administration-guide/account-settings/usage-detail-tags-aws.html