cancel
Showing results for 
Search instead for 
Did you mean: 
Warehousing & Analytics
Engage in discussions on data warehousing, analytics, and BI solutions within the Databricks Community. Share insights, tips, and best practices for leveraging data for informed decision-making.
cancel
Showing results for 
Search instead for 
Did you mean: 

DLT pipeline run cost

Chhaya
New Contributor III

Hi team,

I am looking for a way to find DBU cost for DLT clusters, does it get stored anywhere I have been looking into event_logs but did not find information related to cost. it does have cluster resource utilization details.

here is what I found, could somebody assist here ?

dlt_Cost 

I'm looking to monitor cost for the compute used by DLT pipelines

1 ACCEPTED SOLUTION

Accepted Solutions

karthik_p
Esteemed Contributor

@Chhaya Vishwakarma​ DBU pricing won't be visible in event log, when you go to your created DLT Job --> Settings --> under Compute (Summary option)--> shows DBU/hr pricing .

If you are creating new job--> u can see pricing summary near compute itself

usually DLT cluster prcicing will depend on whether u enabled photon or not , without photon you see different price and with photon u will see different pricing

View solution in original post

4 REPLIES 4

karthik_p
Esteemed Contributor

@Chhaya Vishwakarma​ DBU pricing won't be visible in event log, when you go to your created DLT Job --> Settings --> under Compute (Summary option)--> shows DBU/hr pricing .

If you are creating new job--> u can see pricing summary near compute itself

usually DLT cluster prcicing will depend on whether u enabled photon or not , without photon you see different price and with photon u will see different pricing

Chhaya
New Contributor III

thank you Karthik, does overwatch supports DLT . I am looking at more automated, and granular costing/performance monitoring

karthik_p
Esteemed Contributor

@Chhaya Vishwakarma​ As far as i know SAT and overwatch are meant to provide recommendations to environment (DBR's/VPC security etc...), you can do one this post your cost pricing data to any storage and perform query filtering based on requirement and can be analyzed and automated

Anonymous
Not applicable

Hi @Chhaya Vishwakarma​ 

Thank you for posting your question in our community! We are happy to assist you.

To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?

This will also help other community members who may have similar questions in the future. Thank you for your participation and let us know if you need any further assistance! 

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!