Lakeflow SDP expectations

IM_01 — Mon, 06 Apr 2026 18:25:44 GMT

Hi,

Is there a way to get number of warned records, dropped records , failed records for each expectation I see currently it gives aggregated count

Re: Lakeflow SDP expectations

Ashwin_DSA — Mon, 06 Apr 2026 19:05:16 GMT

Hi @IM_01,

You can’t change the UI to break out those numbers, but you can get per-expectation counts from the DLT (Lakeflow) event log. Each expectation entry records passed_records and failed_records; for EXPECT rules failed_records = warned rows, and for EXPECT … DROP ROW rules failed_records = dropped rows. Expectations configured with FAIL UPDATE don’t emit aggregate metrics.

Here is a sample query you can run. Just replace the DLT table name where it says my_dlt_table

WITH exploded AS ( SELECT timestamp, explode( from_json( details:flow_progress:data_quality:expectations, 'array<struct<name:string,dataset:string,passed_records:long,failed_records:long>>' ) ) AS e FROM event_log(TABLE(my_dlt_table)) WHERE details:flow_progress:data_quality IS NOT NULL ) SELECT timestamp, e.name AS expectation_name, e.dataset, e.passed_records, e.failed_records FROM exploded ORDER BY timestamp DESC, expectation_name;

I tested it for a sample table and it returned the split. I'm guessing this is what you want to see?

You can also take a look at the documentation here for exploring data quality / expectations metrics from the event log.

Hope this helps.

If this answer resolves your question, could you mark it as “Accept as Solution”? That helps other users quickly find the correct fix.

topic Lakeflow SDP expectations in Data Engineering

Lakeflow SDP expectations

Re: Lakeflow SDP expectations