Iceberg native table Streaming in databricks

xwu
Databricks Partner

Hi ! 

I’ve been exploring the new Managed Iceberg tables integration and noticed a potential discrepancy between the documentation and actual behavior regarding streaming/incremental workloads.

According to the official limitations, managed Iceberg tables do not currently support streaming or incremental batching. This seems particularly tied to declarative LakeFlow Streaming pipelines.

 
 

xwu_5-1773939524300.png

 
 
 
 
 

Capture d’écran 2026-03-19 175915.png

 

However, during my testing, I found a workaround:

  1. Pre-declare the table in Unity Catalog (UC).

  2. Execute the pipeline using pure Structured Streaming in Spark (decoupled from SQL Iceberg table creation syntax).

By following a specific syntax (similar to the one discussed in this community thread), the streaming process actually works.

My questions for the community and the Databricks team:

  • Is this "pure structure streaming" approach considered a supported workaround, or is it an unintended behavior (loophole) that might be restricted in future updates?

  • Given that the documentation explicitly lists streaming as a limitation, should we avoid this pattern for production workloads?

Thanks ! 

#Lakeflow #Declaratif pipeline #Streaming