DLT pipeline run cost

[Chhaya]

Hi team,

I am looking for a way to find DBU cost for DLT clusters, does it get stored anywhere I have been looking into event_logs but did not find information related to cost. it does have cluster resource utilization details.

here is what I found, could somebody assist here ?

 

I'm looking to monitor cost for the compute used by DLT pipelines

[karthik_p · Accepted solution]

@Chhaya Vishwakarma​ DBU pricing won't be visible in event log, when you go to your created DLT Job --> Settings --> under Compute (Summary option)--> shows DBU/hr pricing .

If you are creating new job--> u can see pricing summary near compute itself

usually DLT cluster prcicing will depend on whether u enabled photon or not , without photon you see different price and with photon u will see different pricing

[Chhaya]

thank you Karthik, does overwatch supports DLT . I am looking at more automated, and granular costing/performance monitoring

[karthik_p]

@Chhaya Vishwakarma​ As far as i know SAT and overwatch are meant to provide recommendations to environment (DBR's/VPC security etc...), you can do one this post your cost pricing data to any storage and perform query filtering based on requirement and can be analyzed and automated

[Anonymous]

Hi @Chhaya Vishwakarma​ 

Thank you for posting your question in our community! We are happy to assist you.

To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?

This will also help other community members who may have similar questions in the future. Thank you for your participation and let us know if you need any further assistance!