topic Re: What is difference between streaming and streaming live table in Data Engineering

What is difference between streaming and streaming live table

Kroy — Fri, 12 Jan 2024 12:38:45 GMT

Can anyone explain in layman what is difference between Streaming and streaming live table ?

Re: What is difference between streaming and streaming live table

Lakshay — Fri, 12 Jan 2024 13:43:51 GMT

Data Loads are broadly classified into two types:

1. Batch load: When the source data is received on schedule, the data load is called as batch load. This is because we have a defined number of rows/ volume of data that needs to be processed at a time.

2. Streaming load: When the source data is received continuously, the data load is called as Streaming load. In this case, there is no fix volume of data that needs to be processed at a time. The data needs to be processed as soon as it is received.

As the two type of loads are quite different, the architectural requirement for the two is also different. A streaming table is just a Delta table that has extra support for a Streaming data load.

Re: What is difference between streaming and streaming live table

Kroy — Sat, 13 Jan 2024 14:37:32 GMT

@Lakshay - Sorry this I Know the difference between batch and streaming load, and I know purpose of streaming table, but here I am looking at difference between streaming and steaming live table

Re: What is difference between streaming and streaming live table

Lakshay — Mon, 15 Jan 2024 13:52:11 GMT

Hi @Kroy , Sorry I didn't understand your question. I thought you were asking the difference between streaming load and table.

Re: What is difference between streaming and streaming live table

Kroy — Thu, 18 Jan 2024 12:20:01 GMT

Thanks @Retired_mod - but how streaming table is different from streaming live table ?

Re: What is difference between streaming and streaming live table

CharlesReily — Thu, 18 Jan 2024 13:22:24 GMT

Streaming, in a broad sense, refers to the continuous flow of data over a network. It allows you to watch or listen to content in real-time without having to download the entire file first. A "Streaming Live Table" might refer to a specific type of data presentation or visualization where data in a table format is updated and displayed in real-time as it changes.

Re: What is difference between streaming and streaming live table

Kroy — Thu, 18 Jan 2024 16:43:14 GMT

What I read about DLT is that , there are three type of tables we can create with keyword

1. Streaming table

2.Live table

3.Streaming live table

I can relate the difference between streaming live table and live table is that - live table is type of materlised view and streaming live table process data from last process record

what I am not getting is what is difference between streaming table and streaming live table?

Re: What is difference between streaming and streaming live table

Atit — Tue, 16 Apr 2024 14:55:29 GMT

@Kroy 1. Streaming table and 3.Streaming live table are the same; the former is the current syntax; the latter is previous, but is still supported.