DLT refresh time for combination of streaming and non streaming tables?
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Sunday
@dlt.table
def joined_table():
dim_df = spark.read.table("dim_table") # Reloads every batch
fact_df = spark.readStream.table("fact_stream")
return fact_df.join(dim_df, "id", "left")
1 REPLY 1
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Sunday
the question is default DLT pipeline refresh time is 5seconds but if I use combination of streaming and non streaming data then will it still be 5 seconds?

