AWS DATABRICKS COMPUTE COSTS IN USAGE DASHBOARDS

[margarita_shir]

Good afternoon!

I know that the built-in Databricks usage dashboards currently only display DBU (Databricks Unit) usage. Our Databricks workspaces are running on AWS, and the clusters provision EC2 instances automatically.

I would like to include the underlying AWS EC2 costs alongside DBU usage in these dashboards.

• What is the recommended approach for capturing these EC2 costs?

• What is the proper way to connect this data to Databricks and incorporate it into the workspace usage dashboards?

Any guidance, best practices, or example setups would be greatly appreciated.

Thank you!

[Hubert-Dudek]

There is a plan to include EC2 costs in cloud_infra_cost in the system tables. Not sure about its status, but the table is already visible.

[dbxdev]

I guess the cloud_infra_cost table is still in private preview .

Other way would be . 

You can ingest AWS Cost and Usage Report in a S3 bucket that gets loaded into a UC table and join it to Databricks system tables . If you have your EC2 instance tagged appropriately and it propagates in CUR( AWS Cost and Usage Report  ) that will further help you with getting the right metrics

[dbxdev]

Another thread on cloud_infra_cost - 
https://community.databricks.com/t5/administration-architecture/cloud-infra-costs/td-p/96544