Databricks Community

IM_01

Hi,I was going through the documentation on quarantining records. Initially I thought that partitioning is not supported for temporary tables however I came cross the following@DP.table( temporary=True, partition_cols=["is_quarantined"], ) @dp.ex...

IM_01

Hi,I was using window function row_number(),min,sum in the code, then the Lakeflow SDP pipeline was failing with the error: NON_TIME_WINDOW_NOT_SUPPORTED_IN_STREAMING - Window function is not supported on streaming dataframeswhat is the recommended a...

IM_01

Hi,A column was deleted on the source table, when I ran LSDP it failed with error DELTA_STREAMING_INCOMPATIBLE_SCHEMA_CHANGE_USE_LOG : Streaming read is not supported on tables with read-incompatible schema changes( e.g: rename or drop or datatype ch...

IM_01 · 01-30-2026

HiI see there is a way to store rules in table & use them in python while implementing LDPs how to use the generate/ read rules dynamically in SQL way of implementing LDPs. Could you please help me with this#DLT

IM_01

Thanks Ashwin

IM_01

@Ashwin_DSA So if we use view as a streaming source it can only reference a streaming table that are part of sdp pipeline or delta tables created outside the pipeline but not materialized viewsAnd even a temporary view can be used as streaming sourc...

IM_01

Hi @Ashwin_DSA Thanks for the response But in case of materialized view as a source for streaming table it should not work right but I see the pipeline is running without error. could you please help with this as the refresh rate can be full/incrme...

IM_01

@Ashwin_DSA could you please help with above sorry forgot to mention you above & unable to edit it 

IM_01

Hi Thanks for the response & for sharing the article.I was going through the article, had a doubt - view is a logical construct so it gives results when we read it and as per my knowledge a append-only source can be used as a source for streaming t...

Databricks Community

User Stats

User Activity

Partition cols for a temporary table in Lakefow SDP

Structured streaming error- NON_TIME_WINDOW_NOT_SUPPORTED_IN_STREAMING

Lakeflow SDP failed with DELTA_STREAMING_INCOMPATIBLE_SCHEMA_CHANGE_USE_LOG

how to use rules dynamically in LDP

Re: Partition cols for a temporary table in Lakefow SDP

Re: Partition cols for a temporary table in Lakefow SDP

Re: Partition cols for a temporary table in Lakefow SDP

Re: Partition cols for a temporary table in Lakefow SDP

Re: Partition cols for a temporary table in Lakefow SDP