cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

What is difference between streaming and streaming live table

Kroy
Contributor

Can anyone explain in layman what is difference between Streaming and streaming live table ?

1 ACCEPTED SOLUTION

Accepted Solutions

Atit
New Contributor III
New Contributor III

@Kroy  1. Streaming table and 3.Streaming live table are the same; the former is the current syntax; the latter is previous, but is still supported.

View solution in original post

8 REPLIES 8

Lakshay
Esteemed Contributor
Esteemed Contributor

Data Loads are broadly classified into two types:

1. Batch load: When the source data is received on schedule, the data load is called as batch load. This is because we have a defined number of rows/ volume of data that needs to be processed at a time.

2. Streaming load: When the source data is received continuously, the data load is called as Streaming load. In this case, there is no fix volume of data that needs to be processed at a time. The data needs to be processed as soon as it is received.

As the two type of loads are quite different, the architectural requirement for the two is also different. A streaming table is just a Delta table that has extra support for a Streaming data load.

Kroy
Contributor

@Lakshay  - Sorry this I Know the difference between batch and streaming load, and I know purpose of streaming table, but here I am looking at difference between streaming and steaming live table

Lakshay
Esteemed Contributor
Esteemed Contributor

Hi @Kroy , Sorry I didn't understand your question. I thought you were asking the difference between streaming load and table.

Kaniz_Fatma
Community Manager
Community Manager

Hi @Kroy

A Live Table is like a traditional materialized view that always reflects the results of the query that defines it, including when the query defining the table or view is updated, or an input data source is updated. It may be entirely computed when possible to optimize computation resources and time.

 

A Streaming Live Table, on the other hand, processes data that has been added only since the last pipeline update. Streaming tables and views are stateful; if the defining query changes, new data will be processed b....

 

In simpler terms, a Live Table is like a constantly updated scoreboard that reflects all the scores till now, while a Streaming Live Table is like a ticker that shows only the latest scores since the last update.

Thanks @Kaniz_Fatma  - but how streaming table is different from streaming live table ?

CharlesReily
New Contributor III

Streaming, in a broad sense, refers to the continuous flow of data over a network. It allows you to watch or listen to content in real-time without having to download the entire file first.  A "Streaming Live Table" might refer to a specific type of data presentation or visualization where data in a table format is updated and displayed in real-time as it changes.

What I read about DLT is that , there are three type of tables we can create with keyword

1. Streaming table

2.Live table 

3.Streaming live table

I can relate the difference between streaming live table and live table is that - live table is type of materlised view and streaming live table process data from last process record

what I am not getting is what is difference between streaming table and streaming live table?

Atit
New Contributor III
New Contributor III

@Kroy  1. Streaming table and 3.Streaming live table are the same; the former is the current syntax; the latter is previous, but is still supported.

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!