Left Outer Join returns an Inner Join in Delta Live Tables

dofre — Wed, 06 Mar 2024 10:31:08 GMT

In our Delta Live Table pipeline I am simply joining two streaming tables to a new streaming table.

We use the following code:

@Dlt.create_table() def fact_event_faults(): events = dlt.read_stream('event_list').withWatermark('TimeStamp', '4 hours') files = dlt.read_stream('file_list').withWatermark('Trigger_DateTime', '4 hours') event_faults = events.join(files, ( (events.TimeStamp >= (files.Trigger_DateTime - f.expr('INTERVAL 1 SECONDS'))) & (events.TimeStamp <= (files.Trigger_DateTime + f.expr('INTERVAL 1 SECONDS'))) & (events.Unique_Installation_ID == files.Unique_Installation_ID) ), how='leftouter') return event_faults

When running this, the event_list table has 12K records, the file_list has 4000 records but the event_faults table only has 76 records.

In an left outer join I would expect at least 12K records in the resulting table.

Executing the same query in a notebook without streaming and without watermarking, returns the expected 12K+ rows.

What am I missing here?

Re: Left Outer Join returns an Inner Join in Delta Live Tables

ragonzalez — Thu, 12 Feb 2026 00:04:06 GMT

did you ever get this resolved? struggling with a similar problem

topic Re: Left Outer Join returns an Inner Join in Delta Live Tables in Get Started Discussions

Left Outer Join returns an Inner Join in Delta Live Tables

Re: Left Outer Join returns an Inner Join in Delta Live Tables