topic Cannot see 'Databricks Lakehouse Platform" on prod AWS Cost Explorer in Get Started Discussions

Cannot see 'Databricks Lakehouse Platform" on prod AWS Cost Explorer

andresalvati — Wed, 13 Sep 2023 17:52:49 GMT

Hi,

Currently, we have two different AWS accounts: dev and prod. We also have two different workspaces: one for dev and another for prod. The strange thing is that prod costs are being added to the dev account on AWS Cost Explorer ("Databricks Lakehouse Platform" service). I don't see any "Databricks Lakehouse Platform" costs on prod.

Can someone explain how Databricks connects and reports costs to AWS? Is it a Databricks config issue?

Thanks.

Re: Cannot see 'Databricks Lakehouse Platform" on prod AWS Cost Explorer

Walter_C — Wed, 13 Sep 2023 21:10:33 GMT

Databricks uses tags and AWS CloudTrail logs to connect and report costs to AWS.

Tags can be used to monitor costs and attribute Databricks usage to different business units and teams.

AWS CloudTrail logs can be used to calculate the exact cost of API requests for a Databricks file system (DBFS) S3 bucket.

Re: Cannot see 'Databricks Lakehouse Platform" on prod AWS Cost Explorer

andresalvati — Thu, 14 Sep 2023 11:17:38 GMT

Yes. However, I think AWS Cost Explorer is still powerful because you can compare Databricks against all the other services. Why am I seeing prod and dev aggregated on dev?